How GitHub uses AI agents to stress test new features

Sameen Karim explains how GitHub uses a swarm of AI agents to stress test new features before release, with agents continuously probing for edge cases and attempts to circumvent rules to help harden the system.

Overview

Before shipping new features (including stacked pull requests), GitHub runs agent-driven testing where many AI agents operate continuously to:

• Try to circumvent rules and constraints in the system
• Discover edge cases that typical test suites might miss
• Stress test code paths and system behavior under adversarial or unexpected usage

The clip frames this as “the era of agent-driven testing,” highlighting AI agents as a way to expand coverage beyond conventional scripted tests and manual QA by having autonomous agents explore the product’s behavior around the clock.