Apple conducted a fascinating study to determine whether the best AI models available today can truly "reason". Not just solve tasks, but think through them.

They tested several LLMs on logic puzzles ranging from easy to very complex and gave them the opportunity to work through them step by step. The result? Even the best models often fail to think reliably. They either overcomplicate easy problems or simply give up on hard ones.

Most striking of all, when complexity increased, the models didn’t try harder — they thought less. In some tasks, such as the classic Tower of Hanoi puzzle, even when the correct algorithm was provided in the prompt, the models struggled to reach the goal.

The conclusion is that what appears to be 'reasoning' is often just a probabilistic illusion. AI can simulate thinking, but it lacks the understanding, planning and persistence of humans.

I’m putting together a short write-up in plain English with practical tips on how to use GPT-like tools more effectively in light of this.