Weekly ML Roundup: Governed agents, private data paths, ops lift

by TechHub

This week in Machine Learning, the focus shifts from prototypes to production guardrails, with Fabric data agents now generally available in Copilot Studio and new preview work on advanced DAX generation for semantic models. Fabric governance tightened with workspace outbound access controls, tenant-wide networking policy auditing, and connection recency signals that help reduce stale and single-owner risk. On data movement and interoperability, teams get clearer paths for private connectivity (Snowflake, Eventstream) and a GA transition from ODBC to ADBC that is worth validating early, while platform operations mature through capacity telemetry in Real-Time Hub and new CI/CD guidance to standardize deployments.

This Week's Overview

Fabric data agents and Copilot Studio integrations

Building on last week's push toward governed agent experiences in Fabric (including tighter data source routing controls and MCP-based access patterns), Microsoft Fabric data agents are now generally available in Microsoft Copilot Studio through a tool-based integration (Fabric IQ Data MCP), giving copilots and agents a governed way to query Fabric data without baking credentials and SQL into prompt flows. For teams trying to standardize “chat over enterprise data,” this pushes the integration point toward managed tools and centrally controlled data access, with more flexible publishing targets like Teams and Microsoft 365 Copilot.

On the development side, this also signals a clearer path for agent orchestration: Copilot Studio can call into Fabric data agents as tools, and you can keep enforcement (permissions, auditing, data boundaries) in Fabric rather than duplicating it per bot. If you are already building internal assistants, this is a good moment to review which data experiences should be implemented as Fabric agents vs custom endpoints, then align them with your tenant governance model.

Advanced DAX generation for semantic models also moved into preview for Fabric data agents, using an agentic, iterative workflow to turn natural-language questions into DAX with better instance value resolution. Practically, this targets the brittle part of “NL → BI metric” workflows: ambiguous names, inconsistent dimension values, and DAX measures that work in one report but not another. If you rely on Power BI semantic models, expect this to reduce manual DAX wrangling over time, but plan to validate results and add guardrails for business-critical metrics while the feature is still in preview.

Fabric governance, networking controls, and admin visibility

A cluster of updates this week focused on tightening control over how Fabric workspaces connect to the outside world and how admins audit those settings at scale, extending last week's security-and-operations thread around programmable controls (like CMK APIs) and identity-driven connections. The common theme is making “secure by default” more realistic in multi-workspace tenants by adding workspace-level enforcement, better visibility, and API coverage.

Workspace Outbound Access Protection (OAP) (Preview)

Workspace Outbound Access Protection (OAP) entered preview for Fabric Operations Agent and Fabric Maps, bringing a workspace-level default-deny stance for outbound connectivity with allow-list rules. That matters when workspaces start to accumulate connectors, custom integrations, and agent-style tooling, where outbound calls can become a data exfiltration path if left unmanaged.

For developers and platform teams, OAP pushes networking decisions into a policy surface you can standardize per environment (dev/test/prod) rather than relying on conventions. The preview also adds visibility via notifications and activity logs, which should help when troubleshooting why an artifact suddenly cannot reach an external dependency.

Admin API for tenant-wide networking policy auditing (GA)

Microsoft also shipped GA for the Fabric “List Networking Communication Policies” Admin API, which provides tenant-wide retrieval of workspace inbound/outbound networking policy settings. This closes a practical governance gap: without an inventory API, it is hard to prove compliance or even answer basic questions like “which workspaces allow outbound internet access” during an audit.

If you run Fabric at scale, this is a cue to add continuous checks into your admin automation, then feed results into your compliance dashboards or SIEM workflows. Pairing this API with workspace provisioning standards can help ensure policy drift gets detected quickly.

Connection recency signals and APIs for cleanup and ownership risk

Fabric introduced “connection recency” signals and previewed Connection Recency APIs, aimed at identifying stale, duplicate, or risky single-owner connections across a tenant, which pairs naturally with last week's focus on removing secrets from integrations (for example, workspace identity-based connections) by making it easier to find and retire the connections most likely to fail during hardening. This is the kind of operational governance that usually becomes urgent only after a credential rotation breaks pipelines or a single user leaves and takes the only working connection with them.

The guidance includes using the Fabric REST APIs (plus a companion Python notebook) to find connections that have not been used recently and to reduce single-owner risk through role assignments and Microsoft Entra ID group ownership. Treat this like hygiene for your data estate: fewer duplicated connections and fewer personal credentials tends to mean fewer outages during security hardening.

Data movement and interoperability upgrades (Power BI/Fabric, Snowflake, streaming)

Several changes landed around how data gets into Fabric (securely), how connectors work under the hood, and how teams should prepare for service-level transitions, continuing last week's emphasis on production-friendly connectivity decisions (including gateway patterns and identity-based streaming connections). Taken together, they point to Microsoft standardizing connectivity and hardening private networking patterns as Fabric takes on more production workloads.

ODBC to ADBC driver transition in Power BI and Microsoft Fabric (GA)

The ODBC to ADBC (Apache Arrow Database Connectivity) transition is now generally available in Power BI and Fabric, with a clear validation path and a timeline for default enablement and service cutover. This is an under-the-hood change, but it can affect query behavior, performance, and edge-case connector compatibility, especially for organizations with many legacy data sources and custom gateway setups.

Microsoft is explicitly asking teams to start validating now, with options to test per connection, override at the workspace level, or control via a tenant setting. If you own shared datasets or enterprise gateways, plan a staged rollout: test critical connectors first, confirm query folding and refresh behavior, and watch for differences tied to Power Query settings like Implementation="2.0".

Private connectivity from Fabric to Snowflake

Microsoft published an end-to-end pattern for moving data between Fabric Data Factory (Pipelines and Copy Jobs) and Snowflake using private networking. The topology combines workspace-level Private Link, Trusted Workspace Access to an Azure Storage staging account, and Snowflake storage integration configured with USE_PRIVATELINK_ENDPOINT.

For teams under strict networking requirements, the value is reducing public internet exposure while keeping a workable ingestion flow for large batch transfers. This is also a useful reference when you need to align Fabric workspace identity, staging storage controls, and Snowflake-side policies into one reviewed architecture.

Streaming private operational data into Fabric Eventstream

A separate guide walked through streaming private sources (like a factory MQTT broker) into Fabric Eventstream using Azure virtual network injection and a Fabric streaming virtual network data gateway, which builds on last week's Eventstream security direction by showing the networking side of keeping ingestion private end-to-end. The key point is that operational sources can remain off the public internet while still feeding near real-time analytics in Eventhouse and Lakehouse.

For architects building OT (operational technology) to IT pipelines, this is a practical pattern to study because it combines identity (workspace identity and RBAC) with networking controls. It also sets up a cleaner path to event-driven alerting and downstream transformations once the stream is inside Fabric.

Fabric operations: capacity, telemetry, and CI/CD practice

This week included updates that make Fabric easier to run like a platform, following last week's observability improvements for streaming workloads (per-Eventstream monitoring controls and dashboard KPIs) with more capacity-level telemetry and automation guidance. These are the pieces that usually decide whether pilots survive contact with production.

Capacity Operation Events in Real-Time Hub (Preview)

Capacity Operation Events are now in preview in Fabric Real-Time Hub, providing near real-time, operation-level telemetry for capacity consumption. Instead of only seeing that a capacity is throttling, admins can narrow down which specific operations and artifacts are driving usage and performance issues.

This should help teams move from reactive “capacity is slow” tickets to targeted remediation, like optimizing a particular workload, rescheduling heavy refreshes, or fixing a runaway query pattern. The preview uses Real-Time Hub building blocks such as Eventhouse and Microsoft Activator, which also hints at where Microsoft expects admins to build automated responses.

New CI/CD learning resources for Fabric automation

Microsoft published new Microsoft Learn resources for Fabric CI/CD, spanning conceptual guidance and end-to-end tutorials. The content covers provisioning with Terraform, deployments with Fabric CLI and fabric-cicd, Azure DevOps pipeline examples, and Bulk Import/Export API patterns.

If your Fabric environments still rely on manual workspace setup and click-driven releases, this is a direct prompt to standardize. Treat the tutorials as a baseline, then extend them with environment promotion rules, secret management, and policy enforcement (for example, networking policies and connection governance) so deployments stay consistent.

Data modeling and warehousing patterns in Fabric (medallion, dbt, migration)

A set of guides and GA tooling updates focused on how teams structure data in Fabric over time: medallion-layer practices, dbt orchestration patterns, and migrating SQL workloads into Fabric-native capabilities, continuing last week's theme of moving from “it works” patterns to repeatable, governed warehouse operations (including medallion guidance and DW tuning levers). The message is less about new features and more about operationalizing common architectures with clearer playbooks.

Medallion best practices for Fabric Data Warehouse

Microsoft shared best practices for implementing a medallion architecture in Fabric Data Warehouse, with guidance aligned to how teams actually operate pipelines. Recommendations include batch writes, making Silver transformations rerunnable, and modeling Gold for consumption, plus using Fabric DW defaults effectively.

The post also calls out concrete mechanics like using COPY INTO and CTAS (create table as select), then monitoring by layer with tools like Query Insights. If you have mixed Lakehouse/Warehouse estates, this is a useful reference for deciding where each layer should live and how to keep reprocessing safe.

dbt job patterns in Fabric (Preview guidance)

New guidance outlines four common dbt job patterns in Fabric across Lakehouse and Warehouse, framed as medallion-architecture options. It also explains how to choose between native scheduling and Fabric pipelines when you need orchestration, dependencies, or enterprise controls.

For teams standardizing on dbt, the practical takeaway is to decide early which Fabric surface is your primary transformation target, then keep orchestration consistent to avoid splitting operational visibility across tools. Even though the post is framed as preview guidance, the patterns help reduce “every team does it differently” drift.

Fabric Migration Assistant for SQL database (GA)

The Fabric Migration Assistant for SQL database is now generally available, including a new Validate step that checks a DACPAC for compatibility before provisioning SQL database in Fabric. The workflow covers assessment, schema deployment with Copilot-powered fix suggestions, and then data copy using Fabric Copy Jobs (powered by Data Factory).

This is aimed at turning migration into a repeatable pipeline rather than a one-off project. If you have many SQL databases to evaluate, the Validate step is useful as an early filter for effort and risk, and it can help you standardize what “ready for Fabric SQL” means across teams.

Azure AI infrastructure and platform positioning

Microsoft continued to frame enterprise AI as a production engineering problem that spans infrastructure, tooling, data, governance, and consistent operations, which aligns with last week's focus on making agent access and data operations more governed and repeatable across Fabric and adjacent SQL surfaces. The Azure view emphasizes an integrated stack across compute and storage, AI Foundry tooling, Fabric for data and analytics, Purview for governance, and data services like Azure SQL Database and Azure Cosmos DB.

For developers, the practical implication is that “multi-model AI in production” increasingly depends on how well your platform handles identity, networking, data access, and lifecycle management, not just model quality. The post also points to modernization workflows that can use GitHub Copilot for .NET and Java, tying application refactoring to AI adoption rather than treating them as separate initiatives.

On the infrastructure side, Azure highlighted MLPerf Storage v3.0 results for Azure Managed Lustre, focusing on sustained training throughput and strong checkpoint write/recovery rates across tiers and file-system sizes up to about 25 PiB. If you are training or fine-tuning large models (the post references workloads like UNET3D and Llama3), this is a reminder that storage and checkpointing performance can be a bottleneck, and that evaluating the storage layer is part of serious capacity planning.

Other Machine Learning News

Microsoft published a step-by-step demo kickoff for building a retail assistant on Azure AI Foundry and Fabric, starting with a private-network Bicep template and moving through Fabric capacity, a Lakehouse loaded with AdventureWorks, and an ontology validated via a Fabric Data Agent, which fits as a practical follow-on to last week's agent governance work by showing how these pieces come together in an end-to-end build. It is a useful reference if you want a concrete “from zero to working agent over enterprise data” build, especially when you need private networking from day one.

Azure Databricks troubleshooting guidance also landed for handling capacity incidents, with a practical split between regional VM SKU shortages, subscription quota limits, and Databricks platform ceilings. The mitigations (off-peak retries, changing VM families, instance pools, and Azure Capacity Reservation Groups) are worth turning into runbooks if you support shared analytics platforms.

Finally, Fabric Maps can now be embedded into Real-Time Dashboards in preview as a referenced tile that reuses the map item's published configuration. This keeps layers, filters, styling, and queries managed in one place, which should reduce dashboard drift when geospatial elements are reused across multiple real-time views.