They are not only using human written data any more. Reinforcement learning through human feedback (RLHF) is a big part of it now, that’s the AI running through a problem multiple times with a human picking the best attempt and then using that best attempt as training data going forwards. The model collapse stuff from a few years ago was from an AI repeatedly ingesting its own input with no guidance over many training iterations.