Is AI Unexplainable in Principle? | Roman Yampolskiy

About this video

• Video Title: Is AI Unexplainable in Principle? | Roman Yampolskiy
• Channel: Closer To Truth
• Speakers: Roman Yampolskiy, Robert Lawrence Kuhn (host, implied)
• Duration: 00:09:54

Overview

This video discusses the concept of AI unexplainability, a key risk alongside unpredictability and uncontrollability. Roman Yampolskiy explains why large language models are inherently black boxes, the limitations of current methods to understand them, and the potential for AI explanations to be plausible lies. He contrasts incomprehensibility with unverifiability and explores the implications of AI being fundamentally unexplainable.

Key takeaways

• The Unexplainability Triad: AI poses risks not only from being unpredictable and uncontrollable but also from being unexplainable.
• Black Box Nature of LLMs: Large language models, with billions of neurons and trillions of connections, are too complex for humans to fully comprehend their decision-making processes. The model itself, with all its weights, is the only true explanation, but it's not interpretable by humans.
• Lossy Compression and Plausible Lies: Attempts to explain AI decisions often involve "lossy compression," which simplifies the information but can lead to inaccuracies or incomplete pictures. This can result in AI generating "plausible lies"—explanations that sound reasonable but don't reflect the actual reasoning, similar to how split-brain patients confabulate.
• Limitations of Mechanistic Interpretability: While research aims to understand AI at a more fundamental level (mechanistic interpretability), current capabilities are limited to identifying which neurons fire or connecting clusters of neurons to concepts, not a full understanding of the decision-making process.
• The Catch-22 of AI Safety: There's a paradox where AI needs to be understood to be made safe, but if AI becomes too complex for humans to understand, we might need AI itself to understand and control AI, leading to a cycle of increasing complexity.
• Incomprehensibility vs. Unverifiability: Incomprehensibility means an explanation is beyond human understanding due to complexity or intelligence gap. Unverifiability relates to the difficulty of verifying the correctness of mathematical proofs or software, which often relies on imperfect peer review or probabilistic methods, leading to an infinite regress of verifiers.
• Permanent Unexplainability: The unexplainability of AI might be a permanent, principled issue rather than a temporary technical one. This lack of understanding is surprising to many and means AI systems function like black boxes, studied by scientists without a full grasp of their inner workings.