Weekly ML Roundup - Fabric Runtime 2.0, agents, and RAG

by TechHub

This week in ML and data engineering, Microsoft Fabric pushed further on standardization and operability with Runtime 2.0 (Spark 4.1 and Delta Lake 4.2), a GPU query acceleration preview, and more policy-driven OneLake governance, including a preview mirror of Google Lakehouse Runtime Catalog metadata for Iceberg tables. On the workflow side, Fabric Warehouse CI/CD moved closer to familiar database project patterns with DacFx and a VS Code Schema Compare path, while agent updates (including GPT-5.1 references and MCP-based connections) continued the shift from assisted authoring to tool-connected automation that still needs validation and logging. Rounding out the week, new guidance focused on practical cost loops in Azure Databricks and on building AI-ready apps with vector search and RAG in Azure SQL Database, plus a broader view of how Work IQ, Foundry IQ, and Fabric IQ frame RAG as the grounding layer for enterprise agents.

This Week's Overview

Microsoft Fabric August 2026: Runtime, governance, and AI-assisted workflows

Fabric Runtime 2.0 and performance features (GPU query acceleration preview)

Fabric's August 2026 update lands with Runtime 2.0, bringing Apache Spark 4.1 and Delta Lake 4.2 into the managed runtime for Fabric data engineering and data science workloads, following last week's Runtime 2.0 GA milestone by putting the new baseline into the monthly feature summary context. For teams standardizing Spark and Delta versions across environments, this reduces version drift and makes it easier to validate notebooks and pipelines against a known runtime baseline.

On the performance side, Fabric highlighted a GPU query acceleration preview, which targets faster execution for eligible query patterns. If you are evaluating it, treat it like any other accelerator rollout: validate query shapes, measure end-to-end latency (not just engine time), and confirm cost behavior under realistic concurrency.

OneLake integration updates, including mirrored Google Lakehouse Runtime Catalog (preview)

This month's Fabric summary also calls out OneLake resource instance rules, continuing last week's governance-through-operability thread (for example, identity-filtered SQL audit logs and OneLake-first patterns) by making OneLake governance and data placement more predictable at scale. The practical takeaway is that OneLake management is increasingly policy-driven, so platform teams should review how new rules interact with existing capacity and workspace organization.

In preview, Fabric can now mirror Google Lakehouse Runtime Catalog metadata into OneLake, enabling zero-copy access to supported Apache Iceberg tables across Fabric experiences (SQL, Power BI, and Spark-based engineering and data science). This is aimed at reducing duplication when your lakehouse spans platforms, but it also means you need to re-check assumptions around permissions, lineage, and what “mirrored metadata” implies for discovery and auditing.

Warehouse CI/CD improvements and schema workflow (DacFx + VS Code Schema Compare)

Fabric Warehouse CI/CD got more attention in the August roundup, including improvements that lean on familiar SQL tooling like DacFx plus a VS Code Schema Compare workflow, and it complements last week's push to make governance and cost attribution actionable by making schema changes easier to review, promote, and audit consistently. For teams trying to treat warehouse schema as a first-class artifact, this points toward tighter “database project”-style deployment patterns rather than one-off manual changes.

The near-term developer impact is fewer bespoke scripts for drift detection and promotion, especially when multiple squads share a Fabric Warehouse and need a consistent review-and-deploy path. If you already use schema compare in other Microsoft SQL ecosystems, this is a signal that Fabric is aligning with those established practices.

Data agents and agent integrations (GPT-5.1 and MCP-based connections)

August's Fabric update also highlights a set of data agent enhancements, including support that mentions GPT-5.1 and integrations built around MCP (Model Context Protocol), building on last week's agent-assistance previews in Fabric Data Agent by shifting from “how to author safely” toward broader model and tool connectivity. The direction is clear: Fabric wants agent-style experiences to be able to reach across more tools and data surfaces while still staying grounded in governed enterprise data.

For developers, the practical question is where the boundaries are between “agent assistance” and production automation. If you plan to operationalize agent outputs (for example, generating queries, summarizing results, or triggering actions), build in validation steps and logging so outputs can be reviewed and traced like any other automated workflow.

Fabric data access and migration: safer gateways, identity preservation, and controlled public exposure

On-premises data gateway v3000.330 (August 2026)

The August 2026 on-premises data gateway release (v3000.330) focuses on Power BI Desktop compatibility and a set of improvements across security, authentication, diagnostics, and reliability, and it follows last week's July gateway release with another nudge to keep hybrid connectivity components in lockstep with Desktop and service changes. Gateway upgrades are easy to postpone, but this release is a reminder that keeping the gateway current is often tied directly to keeping refresh and connectivity stable as Power BI Desktop and related services evolve.

The release notes highlight attention to CVEs and third-party dependencies, which matters for organizations that treat the gateway as a controlled boundary into on-prem networks. If you run multiple gateway clusters, plan a staged rollout so you can validate authentication changes and diagnostic behavior before upgrading every environment.

Fabric Data Warehouse: IDENTITY columns with IDENTITY_INSERT (GA)

Fabric Data Warehouse now generally supports IDENTITY columns with IDENTITY_INSERT and reseed operations, which is a concrete win for migration and hybrid warehousing scenarios, and it pairs naturally with last week's warehouse cost-attribution guidance by making it easier to move existing workloads into Fabric without breaking key-based reporting and downstream query patterns. This specifically helps when you need to preserve existing surrogate keys rather than regenerate them during a move into Fabric.

In practical terms, you can bring in historical dimension and fact tables while maintaining key stability, which reduces downstream breakage in reports, semantic models, and ETL logic that assumes certain IDs. It also makes common enterprise patterns (like controlled key management and deterministic loads) easier to implement without workarounds.

Fabric Apps: anonymous data access (preview)

Fabric Apps introduced a preview feature to allow anonymous access to selected data and operations, but only under explicit controls: tenant admin approval, app opt-in, and data-model role permissions, extending last week's governance theme (identity filtering and security controls) into an explicit “public exposure” scenario. Done well, this supports “public portal” style experiences or broad internal distribution without requiring every viewer to be provisioned as a named user.

The security model matters more than the feature toggle here. Teams should treat this as an application security exercise: least-privilege roles in the data model, careful scoping of what operations are exposed, and monitoring to detect unexpected usage patterns.

Event-driven patterns in Fabric: from business events to automated reactions

Fabric's Business Events series continued with a focus on how to react to events using Fabric Activator and Eventhouse, and how those pieces map to broader Fabric and Azure eventing patterns, picking up from last week's Real-Time Hub capacity events by widening the same event-driven approach from platform telemetry into business workflows. The thread tying this together is operational analytics that does not stop at dashboards: events flow in, detections happen, and actions can follow.

For developers building near-real-time workflows, the key design decision is where to place your triggers and state. Activator-style patterns can reduce glue code for common reactions, while Eventhouse provides a place to land and analyze event streams, but you still need clear contracts for event schemas, retries, and idempotency when actions fire.

Azure SQL and Databricks guidance for ML workloads: cost controls and vector-first app foundations

Azure Databricks cost optimization playbook (discovery → tuning → observability)

A new guide lays out a three-phase approach to Azure Databricks cost optimization: start with cost discovery, move into cluster and Spark/SQL best practices, then operationalize ongoing cost observability, echoing last week's Fabric warehouse CU attribution theme by treating cost as an ongoing engineering loop rather than a one-time report. It calls out practical mechanisms teams can adopt, including tagging, dashboards, budgets, and using Unity Catalog system tables to understand usage patterns.

On the execution side, the guidance points developers toward the familiar knobs that usually dominate spend: cluster sizing and scheduling, query and job efficiency via Spark UI and SQL tuning, and using platform capabilities like Photon Engine and Delta Live Tables where they fit. The big takeaway is that cost work is iterative, so build feedback loops (alerts, reporting, chargeback/showback) rather than treating it as a one-time cleanup.

Azure SQL Database Foundations: Hyperscale, migration, and AI-ready patterns (vector search + RAG)

Microsoft published an Azure SQL Database Foundations video series with companion repos that spans Hyperscale basics, migration and optimization, scaling/performance tuning, and building AI-ready apps using vector search and retrieval-augmented generation (RAG) on operational data, which follows last week's RAG retrieval-quality guidance by reinforcing that your vector-capable datastore choice is just one part of the end-to-end retrieval pipeline. This is useful framing for teams trying to use the operational database as part of an AI architecture without immediately exporting everything into a separate vector store.

A related episode walks through using vector embeddings and vector indexes in Azure SQL Database Hyperscale to enable semantic search and retrieval patterns. If you are prototyping RAG against operational data, this helps you evaluate when “vectors in the database” is sufficient for your latency and relevance targets, and when you need a dedicated retrieval layer.

Other Machine Learning News

Agent building guidance this week focused less on model training and more on how agents connect to real enterprise work, continuing last week's Fabric agent tooling thread by zooming out to how the broader “IQ” lineup positions RAG as the common grounding layer across work, analytics, and app experiences. A Microsoft Developer video breaks down how Work IQ, Foundry IQ (Azure AI Foundry), and Fabric IQ relate, with RAG as the common pattern for grounding agent responses in organizational data and analytics.