From Traces To Action: Auto-Instrumenting LLMs for Observability...

From Traces To Action: Auto-Instrumenting LLMs for Observability With OpenTelemetry & OpenSearch - Aditya Soni, Forrester Research & Anshika Tiwari, Amazon Web Services Large Language Models (LLMs) power some of today’s most advanced AI applications-from chatbots to intelligent copilots-yet monitoring their complex behavior remains a major challenge. This session demonstrates how OpenTelemetry’s auto-instrumentation capabilities, combined with OpenSearch’s powerful analytics, provide end-to-end observability for LLM-based systems. You’ll learn how to automatically collect rich telemetry data-traces, metrics, and logs-from your LLM applications with minimal manual effort using OpenTelemetry and libraries like OpenLIT. See how this data flows into OpenSearch, enabling real-time analysis, anomaly detection, and performance monitoring tailored specifically for AI workloads. By the end of this talk, you will get the tools and best practices to build scalable, AI-driven observability pipelines that keep your LLM applications reliable and performant.