Demis Hassabis on shipping momentum, better evals and world models

About this Video

• Video Title: Demis Hassabis on shipping momentum, better evals and world models
• Channel: Google for Developers
• Speakers: Demis Hassabis, Logan Kilpatrick
• Duration: 00:31:10

Introduction

This Google for Developers video features an interview with Demis Hassabis, CEO of Google DeepMind. The conversation centers around DeepMind's recent progress, focusing on advancements in large language models (like Gemini), the development of world models (Genie 3), the need for improved AI benchmarks (Kaggle Game Arena), and the future direction of AI research, particularly concerning the shift from models to systems and the pursuit of Artificial General Intelligence (AGI).

Key Takeaways

• Unprecedented Momentum: DeepMind is experiencing rapid progress and releasing new AI technologies at a fast pace.
• Agent-Based Systems & Deep Think: DeepMind's focus is on creating agent-based systems capable of complex reasoning and planning ("deep thinking"), building on their earlier work with game-playing AI.
• Genie 3 & World Models: Genie 3, a world model, demonstrates an impressive ability to understand and generate consistent representations of the physical world, opening possibilities for interactive entertainment and robotics.
• Need for Better Benchmarks: Current AI benchmarks are becoming saturated; Kaggle Game Arena provides a novel, self-scaling benchmark for evaluating AI capabilities across various games. The focus is shifting towards more nuanced evaluations that assess aspects beyond simple scores, such as consistency, safety, and reasoning skills.
• Shift from Models to Systems: The conversation highlights a shift from simply creating large language models to developing AI systems that effectively integrate models with tools and reasoning capabilities. This approach incorporates tool use as a crucial scaling dimension for expanding AI capabilities.
• Omni Model Vision: DeepMind's long-term goal is to create an "omni model" that integrates various capabilities (language, vision, world modeling) into a single, general-purpose AI system.