Dynamic Shape Recompilations

Lightning Talk: Dynamic Shape Recompilations - Bob Ren, Meta Recompilations are a critical aspect of the PT2 programming model, particularly when considering the trade-offs between specialization and runtime performance. This talk explores the challenges associated with dynamic shapes recompilations, where optimizations intended to enhance performance can inadvertently lead to significant recompilation overheads. Such issues often arise when developers implement optimizations that, while improving specific metrics like QPS, introduce unpredictability in recompilation frequency. This talk aims to educate users on these trade-offs and highlight future work focused on advancing the pareto frontier through innovations like multi-graph and multi-kernel level dispatch.