Agents in production: Observe evaluate and optimize with Foundry | BRK251

Deepraj Khot, Han Che, and Divya Chirasani present a practical continuous-improvement workflow for agents after deployment. The session combines Foundry demonstrations with a Tata Consultancy Services (TCS) case study.

From production behavior to continuous improvement

Deploying an agent is the start of an ongoing process: observing its behavior, evaluating its outputs, investigating unexpected issues, and improving performance. Because AI agents are non-deterministic, teams need evidence from actual runs as well as structured evaluations to assess whether they remain trustworthy in production.

The session connects several parts of this workflow:

Evaluation and debugging demonstrations

The demonstrations cover traces, datasets, synthetic data generation, and offline evaluations. The presenters also introduce Foundry evaluation features, including rubric evaluators and decision models.

The production workflow extends evaluation to live traffic, helping teams identify quality gaps that emerge after deployment. Production insights and continuous analysis then provide evidence for debugging, while Agent Optimizer workflows connect those findings to quality improvements.

Monitoring across an agent fleet

Beyond investigating individual runs, the session addresses observability across multiple agents and cost management. The TCS case study illustrates token reduction and quality improvements using Foundry observability.

Session chapters

Resources

This foundational breakout session, BRK251, is part of the Agent Platform track at Microsoft Ignite 2026 and is presented in English (US).