Jay Yuan

Safely releasing AI applications while continuously patching vulnerabilities

AI applications still need regular security patches and dependency updates, but those software changes can also affect the data processing, model interaction, and evaluation paths around an LLM. In our releases, traditional tests and pass/fail evaluation outcomes could remain broadly stable while the underlying evaluation score distributions shifted. That raised a practical question: is the movement a real behavioural regression, normal LLM variability, or variation in the evaluator itself—and when is there enough evidence to stop a release?

This session shares how I approached that problem by bringing behavioural regression testing into the pull request workflow for AI-impacting changes. I’ll cover repeated evaluation runs, behavioural ranges and score distributions, green/amber/red signals, and release-level historical tracking, as well as what worked, what didn’t, and where engineering judgment is still required when deciding whether probabilistic software is safe to ship.