The Hard Truth About AI Agents: Lessons from Running Agents in Production

🎥 Recorded live at the MLOps World | GenAI Summit 2025 — Austin, TX (October 8, 2025) Session Title: The Hard Truth About AI Agents: Lessons Learned from Running Agents in Production Speaker: Hannes Hapke, Principal Machine Learning Engineer, Digits Talk Track: Agents in Production Abstract: Every event shows slick AI agent demos — but few show what happens at 3 AM when agents go rogue. In this candid and practical session, Hannes Hapke (Digits, formerly at Google) shares the hard-earned lessons from deploying real, customer-facing agents that handle sensitive financial data. You’ll hear what actually breaks in production, how to prevent expensive errors, and what patterns keep mission-critical systems stable. Hannes dives into architecture for reliability and observability, how to design guardrails without crippling performance, and why evaluation metrics in development often fail to predict real-world reliability. This talk is a field guide to the messy reality of AI agents in production, drawn from the insights in his new O’Reilly publication, Generative AI Design Patterns (co-authored with Dr. Valliappa Lakshmanan). What you’ll learn: • Why impressive agent demos fail in production—and how to bridge that gap • Architectural principles that actually improve reliability at scale • Practical monitoring techniques to catch issues before customers do • How to build functional guardrails without stifling your agents • Real-world lessons on trust, risk, and long-term system resilience.