Browse Machine Learning News (147)

olivialiu-micro explains how SQL Server 2025 enables governed outbound calls to external AI services while keeping control inside the database security model, including permissions, enablement switches, authentication options, and built-in auditing for REST invocation and external model objects.

Mabasile_MSFT announces general availability of OneLake Storage Tiers and Lifecycle Management in Microsoft Fabric, explaining how to keep frequently used data in the hot tier while automatically moving less-accessed data to cool or cold tiers to reduce long-term storage costs.

Mabasile_MSFT announces general availability of the OneLake storage report in Microsoft Fabric, which lets workspace admins generate an item-size breakdown of storage usage in a workspace, including visible, system, and soft-deleted data, plus billing status per item.

mk_sunitha explains a secure pattern for calling Microsoft Fabric REST APIs from User Data Functions (UDFs), using a service principal plus Azure Key Vault to keep secrets out of application code while still enabling governed automation like triggering Fabric pipelines and managing workspaces.

conxu-ms summarizes new Microsoft Fabric Data Factory updates focused on making pipeline orchestration easier, improving monitoring and troubleshooting, and adding AI-assisted operational capabilities for managing data workflows at scale.

goupadhy announces a preview of autoscaling for Microsoft Fabric Virtual Network (VNet) Data Gateway, explaining how gateway node counts scale in and out based on workload signals to balance performance, reliability, and cost, plus where admins configure limits and review diagnostics logs.

arindamc shows how to build a “row-to-intelligent-action” pipeline in Microsoft Fabric: capture Azure SQL changes via CDC into Fabric Eventstreams, reshape them with DeltaFlow, enrich each micro-batch in Spark Structured Streaming using AI Functions for PySpark, and publish Business Events for real-time routing and dashboards.

shiv_narayanan summarizes recent Dataverse Fabric Link updates that make it easier to control which Dataverse tables are synced into Microsoft Fabric, improve production-ready authentication options, and reduce sync latency so operational data shows up faster in OneLake-backed analytics workloads.

ofer announces a new OneLake Architectural Guidance whitepaper and summarizes five repeatable enterprise patterns for building a unified, governed data foundation in Microsoft Fabric, aimed at analytics modernization and AI readiness, including data access unification, medallion architecture, governed data mesh, platform simplification, and external data sharing.

Sridhar Baynes announces Aurora 1.5, an open-source update to Microsoft’s Earth-system foundation model that adds more weather variables, hourly resolution, and probabilistic ensemble forecasts, with results compared against ECMWF ensembles and examples including tropical cyclone track improvements.

spurthikommajos introduces the Oracle Database CDC source connector for Microsoft Fabric Eventstream (Preview), explaining how row-level inserts, updates, and deletes can be captured as structured change events and routed to Fabric destinations for real-time analytics, alerts, and downstream workflows.

Alicia Li summarizes the 2026 Q2 updates for Microsoft Fabric Eventstreams, covering new and updated connectors, the now-GA SQL operator, preview features like mirrored database change feed ingestion and business events publishing, plus AI Skills that generate and deploy eventstream topologies via Fabric REST APIs.

arindamc walks through a Microsoft Fabric “row-to-action” pattern where database row changes are mirrored into OneLake, streamed through Fabric Eventstreams, aggregated with windowing, and routed to Activator and Eventhouse for real-time triggers and live KQL dashboards.

bogdanc shares hands-on benchmark results for GPU-accelerated query execution in Microsoft Fabric Data Warehouse, including a transparent TPC-H methodology, single-user latency numbers, and high-concurrency throughput/latency comparisons against CPU execution on an F64 capacity.

JowoMSFT recaps the top community questions from a Rayfin AMA, covering how Fabric Apps can combine analytics with operational workflows, connect to Fabric data sources and external systems, and support AI-driven experiences while keeping governance, authentication, and row-level security in place.

ArshadAliTMMBA introduces Fabric Runtime Release Channels in Microsoft Fabric, explaining how the new default and early access channels let teams validate Spark runtime changes ahead of time, reduce surprise production breakages, and track exactly what runtime and VHD version their workloads are running.

rusteinberg explains how Microsoft Fabric sensitivity labels (defined in Microsoft Purview) can do more than enforce access control: they can also act as guidance signals for AI skills and agents, helping them choose the right data sources and avoid mixing sensitive and non-sensitive contexts in answers.

Aaron Merrill explains how OneLake security works with Microsoft Fabric shortcuts to enable zero-copy data distribution. He breaks down passthrough vs delegated shortcuts, how permissions are evaluated (including intersection behavior), and when each model fits—especially for large-scale sharing, cross-tenant scenarios, and external storage sources.

shiv_narayanan introduces Delegated OneLake Shortcuts (preview) in Microsoft Fabric, a new authentication option that lets teams share OneLake data at scale using a configured connection identity instead of per-user pass-through access, including support for cross-tenant scenarios and OneLake security controls.

Gali Reznick and coauthors announce preview support for Workspace Outbound Access Protection (OAP) across key Real-Time Intelligence experiences in Microsoft Fabric, explaining what outbound paths are allowed or blocked and how admins can reduce data exfiltration risk with workspace-level rules.

AmirJafari announces that the Microsoft Fabric data agent API is now public, enabling teams to create, configure, update, and publish Fabric data agents programmatically from their own tools and pipelines. The post explains the management-plane focus, how it fits with Fabric automation, and includes a Python quickstart.

sbaynes introduces SkillOpt, a method for improving AI agent reliability by “training” a natural-language skill file through controlled text edits, validation gating, and feedback from execution trajectories—without updating model weights.

xujx summarizes recent Microsoft Fabric Eventstream connector updates for Real-Time Intelligence, including GA private network support, GA Kafka and Azure Service Bus connectors, and GA custom CA/mTLS. The post also covers preview features like workspace identity for Event Hubs, richer IoT Hub metadata, Oracle CDC ingestion, and HTTP pagination.

YuliaTurchin announces the general availability of Item Recovery in Microsoft Fabric, covering how soft-deleted items can be restored via the Workspace Recycle bin or REST API, and how admins can set tenant-wide retention windows (up to 90 days) and update operational runbooks and governance workflows.

jiang_jenny1 introduces a preview, AI-assisted command-line approach for migrating Azure Synapse Spark artifacts and Synapse pipelines into Microsoft Fabric. It covers the end-to-end phases, two migration strategies (lift-and-shift vs. migrate-and-modernize), automatic refactoring and dependency handling, and how the companion pipeline skill rewires orchestration to Fabric Data Factory.

GEGUIRGU announces a preview feature in Microsoft Fabric Real-Time hub that applies workspace Outbound Access Protection (OAP) to Azure and Fabric event consumption across workspaces, so cross-workspace event flows are blocked by default unless explicitly allowed via data connection rules.

sbaynes summarizes Microsoft Research work on generative causal testing (GCT), a method that uses LLMs to turn black-box brain-response prediction models into short, testable explanations and then validates them by generating targeted stories and measuring fMRI responses.

antonfr explains how Microsoft Fabric’s built-in data protection features—powered by Microsoft Purview—help teams make data “AI-ready” by classifying sensitive content, enforcing least-privilege access, applying persistent protection, and improving observability so Copilot and agent experiences don’t amplify oversharing risks.

jovanpop-msft introduces the Bulk Copy API (preview) for Microsoft Fabric Data Warehouse, explaining when client-side bulk ingestion is a better fit than server-side COPY INTO and showing practical patterns for C#, Java, command-line bcp.exe, and orchestration tools like Azure Data Factory and SSIS.

goupadhy announces the June 2026 on-premises data gateway release (v3000.322) for Microsoft Fabric/Power BI, highlighting a move to Windows Web Account Manager (WAM) authentication, an updated Log4j dependency, and new consent-driven diagnostic upload capabilities integrated into the Dataflow Gen2 run experience.

perirocha introduces the generally available sp_get_table_health_metrics stored procedure for Microsoft Fabric Lakehouse tables, showing how SQL analytics endpoint users can diagnose physical table health (like small files, deleted rows, and checkpoints) in T-SQL and trigger OPTIMIZE only when it’s actually needed.

analyticanna rounds up the Microsoft SQL updates shipped in the first half of 2026 across Azure SQL, SQL Server, and SQL database in Microsoft Fabric, with links to each GA/Preview announcement. Highlights include T-SQL regex features, Entra-based logins, TDE improvements, embeddings support, and new tooling in SSMS and VS Code (including GitHub Copilot features).

sbaynes explains Talos, an open-source pipeline for automated, iterative genomic reanalysis that re-checks stored sequencing results as public evidence changes, aiming to deliver more rare-disease diagnoses with a low false-positive rate and a sustainable review workload for clinical teams.

sunil_sabat explains how Microsoft Fabric Data Factory supports multi-cloud data integration and orchestration across platforms like Snowflake, Databricks, Google BigQuery, and Salesforce, and how OneLake features (Shortcuts, mirroring) and Purview lineage help teams build governed, production-ready analytics and AI data flows.

Santhosh_Ravin1 introduces Efficient Scaledown (Preview) for Microsoft Fabric Spark, explaining how remote shuffle storage and shuffle migration reduce recomputation during scale-down, improve resiliency, and cut compute costs, with concrete benchmark results and the Spark configuration needed to enable the feature.

Santhosh_Ravin1 explains how Microsoft Fabric’s Native Execution Engine (NEE) speeds up Spark workloads that use Python/Scala UDFs and nested data types. It covers why UDFs and complex types have historically forced costly serialization and row-based fallbacks, what NEE changes in the execution path, and the benchmarked performance gains.

yexu announces general availability of invoking Microsoft Fabric Copy jobs directly from Fabric Activator, enabling event-driven data movement that runs only when a condition is met (like a file landing in OneLake or a table update) instead of relying on fixed schedules.

Natalie Guevara explains how GitHub built Qubot, an internal GitHub Copilot-powered analytics agent that lets employees ask plain-language questions over warehouse data and get answers quickly, with results captured as markdown reports in pull requests and validated through an offline evaluation framework.

jovanpop-msft introduces preview features in Fabric Data Warehouse that add approximate string-matching functions and modern string operators for T-SQL. The post shows how to use distance and similarity scoring to find misspellings and variants (like “Hongkong” vs “Hong Kong”), plus new concatenation and Unicode helpers for clearer SQL.

Govind Kamtamneni explains how to build an outcome-driven “learning system” in Microsoft Foundry using OpenEnv environments, rubric-based evals, and a closed-loop optimizer. The post contrasts non-parametric harness tuning with parametric post-training (ECHO), and shows how Azure Container Apps sandboxes provide an isolated, enterprise-ready runtime for agent rollouts.