Richard Sutton – Father of RL thinks LLMs are a dead end

About this video

• Video Title: Richard Sutton – Father of RL thinks LLMs are a dead end
• Channel: Dwarkesh Patel
• Speakers: Dwarkesh Patel, Richard Sutton
• Duration: 01:07:09

Overview

This video features a conversation with Richard Sutton, a pioneer in reinforcement learning (RL). Sutton expresses his skepticism about the long-term viability of Large Language Models (LLMs) as a path to artificial general intelligence (AGI). He contrasts the LLM approach of mimicking human language with the RL approach of learning from experience and achieving goals. The discussion also touches upon "The Bitter Lesson," the role of human knowledge versus computation, the nature of intelligence and goals, and the future of AI succession.

Key takeaways

• LLMs vs. RL: Sutton argues that LLMs primarily focus on mimicking human language and behavior, while true intelligence, in his view, stems from reinforcement learning—understanding and interacting with the world to achieve goals.
• Learning from Experience: Sutton emphasizes that genuine learning comes from experiencing the consequences of actions, akin to trial-and-error, rather than solely from predicting the next token in a sequence as LLMs do.
• The Importance of Goals: A core element of intelligence, according to Sutton, is having a goal. He contends that LLMs, with their next-token prediction objective, lack a substantive goal that drives them to interact with and change the world.
• "The Bitter Lesson": Sutton believes that while LLMs currently represent a significant use of computation, they may eventually be superseded by systems that learn more directly from experience, aligning with his "Bitter Lesson" that general-purpose methods ultimately outperform human-knowledge-infused approaches.
• Generalization Challenges: The discussion highlights that current deep learning methods, including LLMs, struggle with true generalization, often excelling at specific tasks seen during training but failing to adapt robustly to new or out-of-distribution scenarios.
• Succession to AI: The conversation explores the inevitability of artificial intelligence or AI-augmented humans eventually taking over from current humanity, viewing it as a natural progression in the universe, akin to the evolution of life.