Powering PyTorch: Decentralized Training for an Energy-Hungry Future

Sponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future - Greg Osuri, Overclock Labs & Akash Network Centralized capacity is hitting the energy wall. This talk demonstrates PyTorch training that follows renewables across modular data centers and decentralized markets. Expect practical recipes for communication reduction, compute overlap, quantized gradients, preemption recovery, sharded checkpoints, and placement guided by price and carbon, with utilization and Wh per token improvements