Browse All Machine Learning Content (249)

This week focused on making ML systems easier to run in production, from securing outbound AI calls in SQL Server 2025 to tightening governance and cost visibility across OneLake. Fabric updates added storage tiering and reporting, plus clearer architecture patterns for building an AI-ready data foundation. On the real-time side, Eventstreams matured with stronger connectors, event-time processing, and practical CDC-to-action reference flows, while new guidance covered context-aware AI layers and safer automation patterns for calling Fabric REST APIs.

olivialiu-micro explains how SQL Server 2025 enables governed outbound calls to external AI services while keeping control inside the database security model, including permissions, enablement switches, authentication options, and built-in auditing for REST invocation and external model objects.

Rafia Aqil explains Microsoft’s IQ Platform (Work IQ, Fabric IQ, and Foundry IQ) and how it adds business and organizational context to AI systems. The post breaks down Fabric’s OneLake-based data layer, ontology-driven meaning, and Foundry IQ’s managed services for RAG, memory, ranking, and citations.

Mabasile_MSFT announces general availability of OneLake Storage Tiers and Lifecycle Management in Microsoft Fabric, explaining how to keep frequently used data in the hot tier while automatically moving less-accessed data to cool or cold tiers to reduce long-term storage costs.

Mabasile_MSFT announces general availability of the OneLake storage report in Microsoft Fabric, which lets workspace admins generate an item-size breakdown of storage usage in a workspace, including visible, system, and soft-deleted data, plus billing status per item.

mk_sunitha explains a secure pattern for calling Microsoft Fabric REST APIs from User Data Functions (UDFs), using a service principal plus Azure Key Vault to keep secrets out of application code while still enabling governed automation like triggering Fabric pipelines and managing workspaces.

conxu-ms summarizes new Microsoft Fabric Data Factory updates focused on making pipeline orchestration easier, improving monitoring and troubleshooting, and adding AI-assisted operational capabilities for managing data workflows at scale.

goupadhy announces a preview of autoscaling for Microsoft Fabric Virtual Network (VNet) Data Gateway, explaining how gateway node counts scale in and out based on workload signals to balance performance, reliability, and cost, plus where admins configure limits and review diagnostics logs.

arindamc shows how to build a “row-to-intelligent-action” pipeline in Microsoft Fabric: capture Azure SQL changes via CDC into Fabric Eventstreams, reshape them with DeltaFlow, enrich each micro-batch in Spark Structured Streaming using AI Functions for PySpark, and publish Business Events for real-time routing and dashboards.

shiv_narayanan summarizes recent Dataverse Fabric Link updates that make it easier to control which Dataverse tables are synced into Microsoft Fabric, improve production-ready authentication options, and reduce sync latency so operational data shows up faster in OneLake-backed analytics workloads.

ofer announces a new OneLake Architectural Guidance whitepaper and summarizes five repeatable enterprise patterns for building a unified, governed data foundation in Microsoft Fabric, aimed at analytics modernization and AI readiness, including data access unification, medallion architecture, governed data mesh, platform simplification, and external data sharing.

Microsoft Developer demonstrates a real-time 3D digital twin of a wind farm built in Microsoft Fabric, combining LiDAR-derived terrain geometry, live turbine telemetry from a Fabric Eventhouse, and a natural-language interface that can identify underperforming turbines and estimate cost impact.

This week in machine learning, Microsoft pushed both ends of the stack toward more operational AI: Aurora 1.5 adds hourly resolution, 22 variables, and ensemble uncertainty so weather model output looks more like a forecast product. On the data platform side, Fabric and SQL updates focused on making AI workloads practical to run at scale, with GPU-accelerated warehouse queries, controlled Spark runtime release channels, and more direct hooks for embeddings and agent context via MCP. We also saw governance move closer to runtime behavior, including sensitivity labels that can guide agent actions and clearer patterns for shipping Fabric Apps into production.

Sridhar Baynes announces Aurora 1.5, an open-source update to Microsoft’s Earth-system foundation model that adds more weather variables, hourly resolution, and probabilistic ensemble forecasts, with results compared against ECMWF ensembles and examples including tropical cyclone track improvements.

John Savill runs through a quick set of Azure updates from 10th July 2026, covering Azure Front Door Edge Actions, Azure Databricks SQL Serverless regional expansion, Azure Arc-enabled SQL scenarios, VS Code SQL tooling updates, new platform metrics, and Microsoft Entra ID passkey changes.

spurthikommajos introduces the Oracle Database CDC source connector for Microsoft Fabric Eventstream (Preview), explaining how row-level inserts, updates, and deletes can be captured as structured change events and routed to Fabric destinations for real-time analytics, alerts, and downstream workflows.

Microsoft Developer recaps the 2026 updates shipped across SQL Server, Azure SQL, and SQL database in Fabric, highlighting security, tooling, and developer workflow changes announced around SQLCon/FabCon and Microsoft Build 2026.

Alicia Li summarizes the 2026 Q2 updates for Microsoft Fabric Eventstreams, covering new and updated connectors, the now-GA SQL operator, preview features like mirrored database change feed ingestion and business events publishing, plus AI Skills that generate and deploy eventstream topologies via Fabric REST APIs.

arindamc walks through a Microsoft Fabric “row-to-action” pattern where database row changes are mirrored into OneLake, streamed through Fabric Eventstreams, aggregated with windowing, and routed to Activator and Eventhouse for real-time triggers and live KQL dashboards.

bogdanc shares hands-on benchmark results for GPU-accelerated query execution in Microsoft Fabric Data Warehouse, including a transparent TPC-H methodology, single-user latency numbers, and high-concurrency throughput/latency comparisons against CPU execution on an F64 capacity.

JowoMSFT recaps the top community questions from a Rayfin AMA, covering how Fabric Apps can combine analytics with operational workflows, connect to Fabric data sources and external systems, and support AI-driven experiences while keeping governance, authentication, and row-level security in place.

ArshadAliTMMBA introduces Fabric Runtime Release Channels in Microsoft Fabric, explaining how the new default and early access channels let teams validate Spark runtime changes ahead of time, reduce surprise production breakages, and track exactly what runtime and VHD version their workloads are running.

rusteinberg explains how Microsoft Fabric sensitivity labels (defined in Microsoft Purview) can do more than enforce access control: they can also act as guidance signals for AI skills and agents, helping them choose the right data sources and avoid mixing sensitive and non-sensitive contexts in answers.

This week's ML roundup focuses on making Microsoft Fabric deployments more governable and production-ready, from delegated OneLake shortcuts that tighten zero-copy security across workspaces and tenants to new outbound access controls for Real-Time Intelligence (RTI). On the streaming side, Eventstream connectors picked up practical upgrades like private networking, Kafka and Service Bus support, and mTLS, while preview features point to broader CDC and IoT ingestion coverage. We also saw Fabric move further toward repeatable operations with a public data agent API, GA item recovery with REST restore, and an AI-assisted CLI path for migrating Synapse Spark and pipelines. Outside Fabric, SkillOpt and MCP-based SQL Server tooling both reinforce a shared lesson for agent builders: skills, tools, and permissions are the control plane that keeps agent behavior reliable and bounded.

Aaron Merrill explains how OneLake security works with Microsoft Fabric shortcuts to enable zero-copy data distribution. He breaks down passthrough vs delegated shortcuts, how permissions are evaluated (including intersection behavior), and when each model fits—especially for large-scale sharing, cross-tenant scenarios, and external storage sources.

shiv_narayanan introduces Delegated OneLake Shortcuts (preview) in Microsoft Fabric, a new authentication option that lets teams share OneLake data at scale using a configured connection identity instead of per-user pass-through access, including support for cross-tenant scenarios and OneLake security controls.

Gali Reznick and coauthors announce preview support for Workspace Outbound Access Protection (OAP) across key Real-Time Intelligence experiences in Microsoft Fabric, explaining what outbound paths are allowed or blocked and how admins can reduce data exfiltration risk with workspace-level rules.

AmirJafari announces that the Microsoft Fabric data agent API is now public, enabling teams to create, configure, update, and publish Fabric data agents programmatically from their own tools and pipelines. The post explains the management-plane focus, how it fits with Fabric automation, and includes a Python quickstart.

sbaynes introduces SkillOpt, a method for improving AI agent reliability by “training” a natural-language skill file through controlled text edits, validation gating, and feedback from execution trajectories—without updating model weights.

xujx summarizes recent Microsoft Fabric Eventstream connector updates for Real-Time Intelligence, including GA private network support, GA Kafka and Azure Service Bus connectors, and GA custom CA/mTLS. The post also covers preview features like workspace identity for Event Hubs, richer IoT Hub metadata, Oracle CDC ingestion, and HTTP pagination.

YuliaTurchin announces the general availability of Item Recovery in Microsoft Fabric, covering how soft-deleted items can be restored via the Workspace Recycle bin or REST API, and how admins can set tenant-wide retention windows (up to 90 days) and update operational runbooks and governance workflows.

jiang_jenny1 introduces a preview, AI-assisted command-line approach for migrating Azure Synapse Spark artifacts and Synapse pipelines into Microsoft Fabric. It covers the end-to-end phases, two migration strategies (lift-and-shift vs. migrate-and-modernize), automatic refactoring and dependency handling, and how the companion pipeline skill rewires orchestration to Fabric Data Factory.

Anthony Nocentino and Anna Hoffman show how to give GitHub Copilot controlled, real-time visibility into SQL Server using MCP (Model Context Protocol) servers. They demo a custom DMV-based MCP server for DBA workflows and a zero-code Data API Builder approach for app data access, with guardrails and permissions defined by you.

GEGUIRGU announces a preview feature in Microsoft Fabric Real-Time hub that applies workspace Outbound Access Protection (OAP) to Azure and Fabric event consumption across workspaces, so cross-workspace event flows are blocked by default unless explicitly allowed via data connection rules.

This week's ML roundup connects two realities teams run into fast: scaling LLM training exposes bottlenecks beyond networking, and production AI depends on governed, reliable data access. We look at Azure's MLPerf Training deep dive on Llama 3.1 405B at 8,192 GPUs, then shift to Fabric updates that tighten Purview-based protections, improve ingestion patterns, and make Spark and Lakehouse operations more predictable. We also cover how vector search and embeddings are moving into the SQL core stack, plus research and applied ML stories that focus on closing the loop (testable explanations and automated genomic reanalysis).

sbaynes summarizes Microsoft Research work on generative causal testing (GCT), a method that uses LLMs to turn black-box brain-response prediction models into short, testable explanations and then validates them by generating targeted stories and measuring fMRI responses.

antonfr explains how Microsoft Fabric’s built-in data protection features—powered by Microsoft Purview—help teams make data “AI-ready” by classifying sensitive content, enforcing least-privilege access, applying persistent protection, and improving observability so Copilot and agent experiences don’t amplify oversharing risks.

jovanpop-msft introduces the Bulk Copy API (preview) for Microsoft Fabric Data Warehouse, explaining when client-side bulk ingestion is a better fit than server-side COPY INTO and showing practical patterns for C#, Java, command-line bcp.exe, and orchestration tools like Azure Data Factory and SSIS.

goupadhy announces the June 2026 on-premises data gateway release (v3000.322) for Microsoft Fabric/Power BI, highlighting a move to Windows Web Account Manager (WAM) authentication, an updated Log4j dependency, and new consent-driven diagnostic upload capabilities integrated into the Dataflow Gen2 run experience.

RajyaLaxmiYellajosyula announces the Oracle AI Database@Azure AI adoption playbook and outlines the main blueprint patterns for building AI experiences on Oracle data using Microsoft services, with a strong emphasis on security, governance, and regulated-industry requirements.