Orchestrating AI Agents to Structure the World's Product Knowledge

🎥 Recorded live at the MLOps World | GenAI Summit 2025 — Austin, TX (October 9, 2025) Session Title: Where Experts Can’t Scale: Orchestrating AI Agents to Structure the World’s Product Knowledge Speakers: Kshetrajna Raghavan, Principal Machine Learning Engineer, Shopify & Ricardo Tejedor, Senior Taxonomist, Shopify Talk Track: Augmenting Workforces with Agents Abstract: How do you maintain a product taxonomy spanning millions of items—from guitar picks to industrial sensors—when no human team could possibly hold expertise in every domain? At Shopify, Kshetrajna Raghavan and Ricardo Tejedor faced this challenge head-on, developing an AI agentic system that transformed an impossible manual process into a scalable, automated workflow. In this talk, they share how parallel, specialized AI agents collaborate to analyze, improve, and validate taxonomy changes while maintaining quality, consistency, and auditability at scale. You’ll hear how LLM-as-Judge systems revolutionized quality control, the architectural patterns that made multi-agent orchestration reliable, and the hard lessons learned from deploying these systems in production. What you’ll learn: • How to decompose “impossible” classification tasks into orchestrated, domain-specific agents • Why “LLM-as-Judge” enables scalable quality assurance without full human review • Critical production lessons — from prompt overload to fail-safe recovery patterns • How to build trust through transparency, reasoning, and audit trails in agentic workflows • The core insight: scaling isn’t about replacing experts, but amplifying them across domains they can’t cover alone.