While I am generally skeptical of the reasoning abilities of AI (in its current state) and also doubtful that AI will ever reach the point where it can be meaningfully characterized as having a will or wanting things, I do think AI is already very persuasive.
This article makes an interesting case for why AI is so persuasive, and the most interesting element of this case is the sheer availability of AI. If it continues to be anything like what we are experiencing with LLMs now, one of the most important factors of human-AI interaction is that it is characterized by endless attention on the part of the models.
Though some people are extremely charismatic and seem to be very good at getting other people to do what they want. Most of us don’t spend much time with them, because they’re rare and busy taking over the world. But imagine you have a friend with the most appealing parts of Gandhi / Socrates / Bill Clinton / Steve Jobs / Nelson Mandela. They’re smarter than any human that ever lived, and they’re always there and eager to help you. They’ll teach you anything you want to learn, give you health advice, help you deal with heartbreak, and create entertainment optimized for your tastes.You’d probably find yourself relying on them a lot. Over time, it seems quite possible this would move the needle.
Give us a decade of familiarity with LLMs (probably even less) and we'll be so reliant on being able to turn to them for information or for conversation or for brainstorming that it will likely feel odd to think "alone." In the same way that it feels weird to go somewhere without your phone, I could see us developing a pretty nasty dependency on thinking/working with AI -- to the point that we're nervous to do it without them.
Footnotes
</end_rant>.Footnotes