Leveraging PyTorch for Generative AI in Distributed Edge Clouds

Leveraging PyTorch for Generative AI in Distributed Edge Clouds - Tina Tsou, TSC Chair of InfiniEdge AI (LF Edge) and TSC Member of OPEA (LF AI & Data) Generative AI, particularly large language models (LLMs), has rapidly evolved, becoming instrumental in transforming industries. However, the integration and deployment of these models on distributed edge clouds present unique challenges and opportunities. In this talk, we will explore how PyTorch enables efficient development, deployment, and management of generative AI models on geo-distributed edge infrastructures. We will discuss the architecture, key optimizations, and practical insights drawn from deploying PyTorch-based LLMs in real-world edge scenarios. Attendees will gain valuable knowledge on scalability techniques, performance benchmarking, and best practices for ensuring efficient inference and real-time responsiveness at the edge.