PyTorch’s New Tools Span the Full LLM Lifecycle
In a session recorded at AI Infra @ Scale 2024, software engineers Wanchao Liang and Evan Smothers, joined by Meta research scientist Kimish Patel, walked through the latest PyTorch features aimed at the entire LLM workflow. The talk highlighted progress in three core areas: large-scale training, memory-efficient fine-tuning, and on-device inference.
The speakers opened by addressing why memory-efficient fine-tuning matters, then detailed common architectural and algorithmic approaches that allow models to be adapted on consumer-grade hardware. From there, the conversation shifted to the difficulties of moving large models to edge devices. They explained how techniques like quantization are key to making those on-device deployments practical, alongside other deployment-focused optimizations.
