Replying to @david_chisnall@infosec.exchange
@david_chisnall I think people were aware that transistor size will be a limiting factor in this example, so the limit was well known. Also this is physical limitation not a software one, LLMs are mostly just information (compression/code).