Scaling Agents for Gen AI Products - Anju Kambadur, Bloomberg Head of AI Engineering

About this Video

• Video Title: Scaling Agents for Gen AI Products - Anju Kambadur, Bloomberg Head of AI Engineering
• Channel: AI Engineer
• Speakers: Anju Kambadur
• Duration: 00:19:38

Introduction

This video features Anju Kambadur, Bloomberg's Head of AI Engineering, discussing the challenges and strategies of scaling agents for generative AI products within a financial context. She focuses on the practical considerations and solutions Bloomberg employs, emphasizing the need for robustness and reliability in financial applications.

Key Takeaways

• Bloomberg's approach to building AI products: Initially, Bloomberg built its own large language model (LLM), but later pivoted to leveraging existing LLMs and building on top of them due to the rapid advancements in open-source models.
• Challenges in building agentic AI for finance: The speaker highlights challenges such as ensuring precision, comprehensiveness, speed, and data protection in financial applications. The inherent fragility of LLMs and the compounding errors in agentic architectures are emphasized.
• Strategies for scaling agentic AI: The speaker advocates for building robust APIs with well-defined inputs and outputs, reducing stochasticity, and implementing comprehensive monitoring and remediation workflows. She also suggests building "guard rails" into agents to prevent unsafe or inaccurate outputs (e.g., avoiding giving financial advice).
• Architectural considerations for scaling: The talk addresses the importance of adaptable Arc structures, initially favoring rapid iteration with collapsed software stacks and later transitioning to more structured approaches as product design matures and scales. The speaker recommends starting with a flexible, collapsed structure and then evolving towards a more defined architecture as understanding of the product and agent capabilities improves.
• Semi-agentic approach: Due to the limitations of current technology and the need for reliability in finance, Bloomberg employs a semi-agentic approach, combining autonomous elements with human oversight and "guard rails" to ensure accuracy and safety.