Taming the Data Estate with Copilot and Azure Service Monitor | MVP Unplugged
Justin Garrett (Microsoft Developer Relations) talks with Azure MVP Magnus Mårtensson about managing large, complex data estates with AI-assisted observability in Azure, including Azure Monitor, Log Analytics, Service Groups, and Azure Copilot workflows.
Full summary based on transcript
What problem this episode focuses on
The discussion centers on enterprise observability challenges as organizations scale across:
- Hybrid environments
- Multi-cloud platforms
- Legacy systems
Key pain points include monitoring application health, handling large volumes of diagnostic data, and reducing time-to-resolution during incidents.
Azure Monitor and Log Analytics for observability
Magnus explains how Azure Monitor and Log Analytics are used to:
- Collect and centralize telemetry (logs/metrics) across resources and subscriptions
- Query and investigate operational data using Kusto Query Language (KQL)
- Support incident investigation and ongoing health monitoring
Reference docs:
- Azure Monitor documentation: https://learn.microsoft.com/azure/azure-monitor
Managing diagnostic/log data at scale
The episode highlights practical considerations when log volumes grow:
- Keeping diagnostic data useful (signal vs noise)
- Organizing telemetry so investigations don’t become manual, repetitive work
- Using platform features to speed up analysis and reduce operational overhead
Azure Service Groups (preview) and application mapping
Magnus introduces Azure Service Groups (preview) as a way to help Azure understand application architecture by grouping related resources/services.
- Purpose: represent how an application is composed (not just a flat list of resources)
- Benefit: improves how health and dependencies can be reasoned about during investigations
Reference docs:
- Azure Service Groups documentation: https://learn.microsoft.com/azure/governance/service-groups
Health models and resilient operations
The conversation covers using health models to:
- Track application/service health in a structured way
- Improve how teams reason about failures and dependencies
- Support more consistent operational practices across large estates
AI-assisted investigation and faster root cause analysis
A major theme is using AI to accelerate investigations by:
- Analyzing logs and telemetry for patterns
- Detecting anomalies
- Suggesting likely causes and recommended next steps
- Reducing the time spent manually writing and refining queries
Azure Copilot in VS Code: natural language to KQL
The episode describes how Azure Copilot can be used inside tools like Visual Studio Code to:
- Ask questions about operational issues in natural language
- Generate KQL queries from those questions
- Iterate on investigations faster (especially for engineers who don’t write KQL daily)
Related tooling:
- Azure Tools for VS Code: https://code.visualstudio.com/docs/azure/gettingstarted
Azure Advisor recommendations with AI
Magnus and Justin discuss using Azure Advisor insights (with AI assistance) to:
- Prioritize recommendations
- Optimize performance and cost n- Make operational decisions faster across subscriptions
Resources mentioned
- Free Microsoft Foundry trial: https://aka.ms/devrelft
- MVP Unplugged playlist: https://youtube.com/playlist?list=PLlrxD0HtieHhclud3yVB88znZPKCZYX_8&si=4HoycKJyUcl1qwV-