If you’re into something like physicalism, then human intelligence is also just an emergent quality and we have about as much free will as the machines.
Edit history
If you’re into something like physicalism, then human intelligence is also just an emergent quality.
If you’re into something like panpsychism instead, then everything is conscious, including the LLM.
(there are other options too, but I think those two are the most compelling ones)