Speaker: Dianhuan Lin
Staff Engineer @Datadog
Dr. Diane Lin is a Tech Lead at Datadog, where she leads product development on self-evolving AI agents for the modern SOC. She was previously CTO and Co-founder of Culminate (acquired by Datadog), where her AI security analysts augmented human teams to place top 3 at DEFCON's CTRL+ALT+DETECT competition (formerly OpenSOC), a real-world benchmark for AI-driven threat detection under pressure.
Prior to founding Culminate, Diane was Director of Machine Learning at Zscaler, leading the development of ML-based large-scale threat detection models. Before that, she was a founding scientist on the Amazon Alexa question-answering team and a senior researcher at robotics startup Vicarious (acquired by Google DeepMind). She holds a PhD in Machine Learning from Imperial College London with research affiliations at MIT's CSAIL and is an inventor on multiple patents and an active mentor in the AI and cybersecurity communities.
Session
LLM Flip-Flops: From Reliability Risk to a Hill-Climbing Signal
Run the same task through the same LLM twice, and you may get two materially different answers. This behavior is often dismissed as the stochastic nature of LLMs, but it creates a serious product risk.