Any agent, any cloud: Standardized tracing with Foundry+OpenTelemetry | DEM341
Hanchi Wang demonstrates how Azure AI Foundry and OpenTelemetry can standardize GenAI tracing across agent frameworks and clouds, so teams can instrument model and tool calls, debug failures, and use trace-driven evaluation to optimize latency and cost.
Overview
This Build 2026 demo focuses on solving fragmented telemetry for AI agents shipped across different clouds and agent frameworks by using a consistent tracing approach.
Key ideas covered:
- Problem: Teams build and run agents across multiple frameworks and clouds, but observability is inconsistent, making it harder to diagnose:
- Failures
- Latency regressions
- Cost spikes
- Approach: Use Azure AI Foundry together with OpenTelemetry standards for GenAI tracing to get consistent, framework-agnostic and cloud-agnostic traces.
- Workflow: A practical loop for operating agents in production:
- Instrument model calls and tool calls
- Debug issues using unified traces
- Evaluate behavior using trace-based evaluation
- Optimize performance and cost based on what traces reveal
What the demo shows
- Simple setup steps to instrument model and tool calls so they emit standardized traces.
- How unified traces help with diagnosing failures and understanding execution details (including tool execution and metadata in an example involving querying a PDF).
- Discussion of the RAG pattern and how tracing supports debugging RAG flows.
- How consistent observability supports production operations, including:
- Monitoring and debugging with unified traces
- Visualization and optimization using trace data
Session context
- Event: Microsoft Build 2026
- Session: DEM341
- Level: Intermediate
- Topic area: Responsible AI