How we optimize GitHub Copilot for every new model

Julia Kasper, Product Manager at Microsoft, presents the evaluation harness behind GitHub Copilot’s coding agent at GitHub Universe 2026. The session focuses on integrating models from OpenAI, Claude, and Gemini while maintaining quality across different clients and configurations.

The harness behind model integration

The session’s central takeaway is that the harness is the product: supporting a new model involves more than making it available for selection. GitHub Copilot’s evaluation framework continuously tests, measures, and helps optimize the coding agent across a matrix of clients, models, and configurations.

Kasper covers Copilot’s approach to model choice, its model integration strategy, and the workflow used to bring new models into the product.

Evaluating quality and catching regressions

The framework combines several evaluation methods:

These methods support continuous improvement of the harness as models and configurations change.

Session chapters