Speaker: Meghana Madhyastha
Performance Engineer @Parasail, Previously Fundamental AI Research Lab @Meta
Meghana is currently a performance engineer at Parasail working to make LLM serving run faster and more efficiently in our fleet. Her work spans different parts of the stack from writing low-level CUDA kernels to distributed KVCache management, and more recently, teaching agents how to do some of these things. Prior to parasail, she spent a year at the Fundamental AI Research lab (FAIR) at Meta working on writing efficient CUDA kernels that leverage structured sparse hardware primitives. Prior to that, she earned a PhD in computer science from Johns Hopkins university researching how to alleviate data movement and I/O bottlenecks across the memory hierarchy.
Session
Efficient GPU Agents: Infra for Autonomous LLM Perf Optimization
Parasail serves open-weight LLMs on distributed clusters of GPUs managed by Kubernetes. The typical performance optimization loop is to reproduce a problem or measure a regression, iterate on code or config, set up and run benchmarks, and decide whether the change is worth shipping.