Unlock the Future of Generative AI: TorchTitan's Latest...

Lightning Talk: Unlock the Future of Generative AI: TorchTitan’s Latest Breakthroughs - Tianyu Liu & Jiani Wang, Meta We’re excited to share the latest advancements in TorchTitan (https://github.com/pytorch/torchtitan), a PyTorch native platform that’s revolutionizing rapid experimentation and large-scale training of generative AI models. With its widespread adoption across top research institutions and industry labs, TorchTitan is poised to take AI innovation to new heights. Join us as we unveil cutting-edge modeling architectures, including Mixture of Experts (MoE), multimodal LLMs, and diffusion models, and showcase transformative infrastructure techniques like long context support, advanced quantization, fault-tolerant / semi-sync training, and compiler-first parallelisms. With its lean implementation and flexible extension points, TorchTitan empowers researchers and developers to push the boundaries of what’s possible in generative AI. Get ready to experience the future of AI/ML infrastructure today.