US EditionApple Study Reveals AI Reasoning Models Collapse on Complex Problems
- On June 6, Apple published a paper titled 'The Illusion of Thinking' revealing that large reasoning models fail at complex logic tasks like Tower of Hanoi.
- The research arose from testing reasoning-optimized models on puzzles and benchmarks, showing their accuracy collapses as complexity increases despite access to correct algorithms.
- Apple found that models generate hallucinations up to 48% of the time and lack generalizable problem-solving skills, with performance dropping to zero beyond certain complexity thresholds.
- The authors found that model performance sharply declines and eventually stops altogether once problems surpass a specific complexity level, indicating that current models depend more on pattern recognition than on genuine reasoning.
- These findings challenge claims about near-term artificial general intelligence and imply fundamental limits in large reasoning models that urge more rigorous scientific analysis.
65 Articles
65 Articles
New Apple study challenges whether AI models truly “reason” through problems
In early June, Apple researchers released a study suggesting that simulated reasoning (SR) models, such as OpenAI's o1 and o3, DeepSeek-R1, and Claude 3.7 Sonnet Thinking, produce outputs consistent with pattern-matching from training data when faced with novel problems requiring systematic thinking. The researchers found similar results to a recent study by the United States of America Mathematical Olympiad (USAMO) in April, showing that these …
A paper from employees of the Group raises doubts that AI will achieve a "thinking ability" comparable to humans in the foreseeable future
Apple’s study reveals that some of the most sophisticated models fail shockingly when faced with difficult logical tasks, leaving open the question about the real potential of AI
Coverage Details
Bias Distribution
- 43% of the sources lean Left, 43% of the sources are Center
To view factuality data please Upgrade to Premium
Ownership
To view ownership data please Upgrade to Vantage
