Hong Gao introduces new Azure Monitor and AI Foundry integration, providing enterprise-grade observability for generative AI systems. The post highlights unified dashboards, AI telemetry, and OpenTelemetry-powered insights.

End-to-End Observability for Generative AI: Azure Monitor and AI Foundry Integration

Author: Hong Gao

Introduction

Monitoring and trusting your systems requires new strategies in the age of Generative AI. Unlike traditional applications, GenAI apps are dynamic, making choices, planning, and integrating tool invocations. Conventional observability tools focused on servers and microservices fall short for these new workloads.

What’s New: Azure Monitor and AI Foundry Integration

At Microsoft Ignite, the next phase of integrating Azure Monitor with AI Foundry was announced—features designed specifically to tackle observability challenges in GenAI and LLM-based applications and agents.

Key Capabilities

  • Agent Overview Dashboard: Unified dashboards in Grafana and Azure show multiple GenAI agents’ metrics, including:
    • Success rate
    • Grounding quality
    • Safety violations
    • Latency
    • Cost per outcome
    • Ability to track regressions after model/prompt changes

  • AI-Tailored Trace View: Every AI agent decision is readable as a “story” (plan → reasoning → tool calls → guardrail checks). This makes it possible to identify performance or safety issues rapidly.

  • AI-Aware Trace Search: Search, filter, and sort millions of runs using GenAI-specific attributes such as model IDs, grounding scores, or cost—pinpoint critical events efficiently.

  • Low-Code Agent Monitoring: Foundry’s visual interface enables automatic observability of agents. No coding needed—track reliability, safety, and cost from day one.

  • Full-Stack Visibility: All evaluations, traces, and red-teaming results are visible in Azure Monitor, allowing agent signals to be correlated with infrastructure KPIs and telemetry from other services.

Video Demonstration

A demonstration video showing these features in action is available: 2025_IgniteAct3Video.mp4

Learn More

OpenTelemetry Innovation

These new features leverage the latest OpenTelemetry extensions, described in this Azure AI Foundry blog post. Microsoft is actively contributing to OpenTelemetry agent standards, making it possible to capture multi-agent orchestration traces, LLM reasoning context, and custom evaluation signals. This enables interoperability across Azure Monitor, Foundry, and tools like Datadog, Arize, and Weights & Biases, providing customers with consistent monitoring across cloud and hybrid AI scenarios.

Built for Enterprise Scale

By building on open standards and deep Azure integrations, organizations can apply robust governance, compliance, and quality assurance disciplines to AI-powered workloads—just as they do for traditional applications.

Conclusion

Generative AI transforms what it means to operate and monitor software. With these innovations, Microsoft aims to give customers reliable, transparent, and compliant ways to run AI solutions at enterprise scale.

This post appeared first on “Microsoft Tech Community”. Read the entire article here