The training data for LLMs is still orders of magnitudes larger than their weights, memorizing every address, email, etc in the training data and accurately recalling them all is near impossible

They still hallucinate all the time when it comes to relatively common knowledge, let alone some random person’s address

I think the proof is in the pudding on this one, unfortunately

Privacy was dead long before AI. AI makes it much worse though