12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer

About this video

• Video Title: 12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer
• Channel: AI Engineer
• Speakers: Dex Horthy
• Duration: 16:56

Overview

This video discusses the patterns and principles for building reliable Large Language Model (LLM) applications, drawing parallels to the 12-Factor App methodology. The speaker, Dex Horthy, shares insights gained from interviewing over 100 founders and engineers, emphasizing that successful LLM applications often integrate existing software engineering practices rather than relying solely on the "agentic" capabilities of LLMs. The talk outlines key factors for building robust agents, focusing on areas like prompt engineering, control flow, state management, and human-AI collaboration.

Key takeaways

• Agentic Development Challenges: Many initial attempts at building LLM agents, especially for complex tasks like DevOps, prove difficult to refine beyond 70-80% functionality, often leading to developers discarding their work or spending excessive time on prompt engineering.
• 12-Factor Agents Concept: Inspired by the 12-Factor App methodology, the speaker proposes a set of principles for building reliable LLM applications. This framework aims to provide structure and best practices for developers.
• LLMs as Software Components: The core idea is that LLM applications are fundamentally software. Key LLM capabilities, like turning natural language into JSON, are seen as valuable components but not the entirety of an agent.
• Tool Use as Deterministic Code: The use of tools by LLMs is reframed not as a magical AI process, but as an LLM outputting JSON which is then processed by deterministic code.
• Owning Control Flow and State: Reliable agents require developers to own and manage their control flow and execution state, similar to traditional software, enabling features like pausing, resuming, and robust error handling.
• Context Engineering is Crucial: Since LLMs are stateless functions, the quality and relevance of the input (tokens) in the context window are paramount to achieving reliable and high-quality outputs. This involves careful prompt and memory management.
• Micro-Agents and Hybrid Workflows: The most effective current approaches often involve breaking down tasks into smaller, focused "micro-agents" integrated within larger, more deterministic workflows, rather than attempting to create a single, all-encompassing agent.
• Human-AI Collaboration: Designing agents to collaborate with humans, allowing for natural language interaction and decision-making at key points, is a critical factor for building user-friendly and effective applications.