Weekly ML Roundup: Foundry Fine-Tuning and Governed OneLake Access

by TechHub

Foundry expanded public-preview fine-tuning support to additional models, including reinforcement-only support for MAI-Code-1.1-Flash, and introduced Multi-LoRA Managed Compute so compatible adapters can share base-model weights and GPU capacity, with hourly hardware charges that include idle time. OneLake table APIs now support native Iceberg writes, while OneLake storage support for Azure Databricks and Eventhouse enforcement of OneLake permissions reached general availability, broadening interoperability without removing access controls. Fabric's preview IQ Share enables read-only cross-tenant sharing without copying data, and new endpoint discovery and remote Model Context Protocol tools help agents find and query Fabric data. Copilot can now generate editable Real-Time Dashboards in preview, though teams still need to validate generated queries and measures before operational use. Together, these updates connect model customization and agent workflows with governed data access, while keeping deployment costs and permission boundaries explicit.

This Week's Overview

Fabric access controls follow the data

Fabric's access updates cover querying secured data, administering connections, and sharing items across tenants. Each has a different boundary: access to a Fabric object does not necessarily grant access to its underlying data or credentials.

Eventhouse enforces OneLake permissions

OneLake security support for Eventhouse external tables is generally available. Queries automatically enforce object-level, column-level, and row-level controls using the querying user's effective OneLake permissions. Creating an external table does not grant access to the secured data it references.

The Eventhouse endpoint can query Lakehouse data without requiring another copy, while preserving those access restrictions. Teams can define OneLake security roles and validate table access, column restrictions, and row filters with a representative data consumer before opening access more broadly.

Connection administration and policy enforcement

Preview admin connection APIs provide tenant-wide inventory and role-assignment operations through GET, POST, PATCH, and DELETE. Assigning an Owner does not transfer OAuth credentials, grant source-system permissions, or guarantee that dependent workloads will run, so ownership changes still require authentication and access checks.

Authentication and tenant allowlists apply to both existing and new cloud connections when enforcement is enabled. The tenant allowlist supports up to 100 Microsoft Entra tenant IDs, and policy changes can take up to one hour. Existing connections that become noncompliant may report Policy Evaluation Failure and go Offline.

Cross-tenant sharing without copying data

IQ Share is in preview for sharing entire Lakehouses, Warehouses, and Mirrored Items across Fabric tenants. Data stays in the producer's tenant, and recipients get read-only access through Fabric-managed shortcuts without needing a guest identity. Producers can revoke access, set expiration, monitor shares through OneLake Catalog, and use APIs and continuous integration and delivery support.

New and deleted tables appear automatically for consumers, but whole-item changes require refreshing the Share Subscription. Spark notebooks, semantic models, data agents, and ontologies currently need an intermediate Lakehouse and shortcuts rather than direct access to a Mirrored OneLake Item.

OneLake expands interoperable storage and table access

OneLake's updates broaden both direct writes and access to data managed elsewhere. The practical distinction is whether a workflow writes into OneLake, reads through shortcuts, or explicitly replicates data.

Native Iceberg writes

OneLake table APIs now support creating native Iceberg tables, committing updates, dropping tables, and requesting short-lived storage credentials. PyIceberg, Snowflake, and DuckDB can write directly to OneLake using standard Iceberg tooling. Optimistic concurrency checks reject conflicting commits rather than overwriting newer table states, giving writers an explicit conflict to handle.

Shared table metadata

The additional table metadata API is in preview and supports reading and writing descriptions and key-value tags for tables and columns. Metadata updates made through internal OneLake shortcuts apply to the target table, preserving shared context across different access paths.

Native storage for Azure Databricks

OneLake external-location support for Azure Databricks Unity Catalog is generally available from the Microsoft Fabric side. Supported Unity Catalog-managed tables, volumes, views, materialized views, and streaming tables can use underlying data stored in OneLake. OneLake Security is enabled by default for Azure Databricks Storage items.

Setup requires a capacity-backed Fabric workspace, an Azure Databricks Storage item, an approved identity, a storage credential, and a Unity Catalog external location. Teams must configure access separately in Azure Databricks and Fabric, including both Unity Catalog and OneLake Security permissions.

Read-only access to more Databricks objects

Fabric mirrored Azure Databricks catalogs now support materialized views and streaming tables alongside tables. Azure Databricks creates and manages the source objects, while Fabric reads them through shortcuts without copying or replicating the data. Source objects need external data access configuration, applicable prerequisites, and Unity Catalog permissions before catalog refresh or synchronization.

The mirrored catalog provides a SQL analytics endpoint for T-SQL queries and supports Power BI semantic models using Direct Lake. New mirrors enable automatic synchronization of future catalog changes for the selected schema by default.

Salesforce access through shortcuts

Salesforce Data 360 integration with OneLake is available in preview through shortcuts, allowing SQL and Spark queries against shared Data Lake Objects without importing a separate Lakehouse copy or requiring additional refresh. Production applications should use service principal authentication with an application registered in Microsoft Entra ID.

The preview supports only Data Lake Objects with the _dll extension. Data Model Objects, Calculated Insight Objects, and schema-level shortcuts are unsupported, so teams need to check object compatibility before choosing this integration.

Choosing between shortcuts and replication

Fabric's integration guidance distinguishes shortcuts for accessing externally stored data from pipelines for explicit movement, multi-step orchestration, and complex transformations. Snowflake managed tables and views can be replicated into OneLake, while Iceberg integration can access existing data through metadata mirroring and shortcuts. Azure Databricks catalog mirroring likewise exposes catalog structure and uses shortcuts rather than replication.

Open Mirroring provides another ingestion route: it accepts changes in a landing zone, after which Fabric converts and manages the data as Delta tables. For agent applications, Fabric data agents can connect to Copilot Studio or serve as tools for Foundry agents accessing governed enterprise data.

Foundry expands fine-tuning and shared adapter deployment

Foundry's previews expand the models available for customization and introduce shared deployment capacity for compatible adapters. Model support and deployment economics need separate checks before teams commit to a configuration.

More models for managed fine-tuning

Muse-Glimmer-30B, qwen3.8-27b, qwen3.6-35b-a3b, and gpt-oss-120b enter public preview with supervised and reinforcement fine-tuning support. MAI-Code-1.1-Flash is the first Microsoft model available for Managed Fine-Tuning on Foundry resources, but supports reinforcement fine-tuning only in this preview. That distinction limits which training approach teams can use with each model.

Shared GPU capacity for compatible adapters

Multi-LoRA Managed Compute enters public preview, allowing compatible model adapters to share base-model weights and GPU capacity. Adapters must match the same base model and version, and deployment requires a compatible LoRA-enabled template and accelerator quota.

Managed Compute charges hourly for hardware, including idle time, rather than charging per inference token. Shared deployment therefore needs a utilization-based cost assessment, not just a comparison of token prices.

Fabric Warehouse connects capacity diagnosis with query inspection

Fabric Warehouse updates address query investigation and text processing inside SQL. Following last week's workspace-level monitoring coverage, the new Capacity Metrics entry point carries the relevant time window into Warehouse Monitor.

Investigating expensive queries

Monitor in warehouse is generally available and opens Warehouse Monitor directly from Capacity Metrics with the relevant time window applied. Access requires the Capacity Metrics app and the Admin role in the workspace containing the warehouse. The Data Warehouse operations skill in Skills for Fabric can correlate Capacity Metrics with Query Insights for the same warehouse and time window.

The measurements need careful interpretation: CPU time records processor time allocated to execution, not elapsed duration or Capacity Units billed for an individual query. Result cache hit values distinguish cached result reuse (2), cache creation (1), and no applicable result caching (0). These distinctions help developers investigate execution behavior without treating CPU time as a per-query bill.

Regular expressions in T-SQL

Fabric Data Warehouse and the SQL analytics endpoint now support scalar regular expression functions in preview: REGEXP_LIKE, REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR, and REGEXP_COUNT. Developers can extract fields from inbound email text, identify malformed email addresses for review or quarantine, and clean, standardize, or mask matching text directly in T-SQL. This provides an in-database option for preparing text before downstream consumption.

Agents gain data discovery and dashboard generation tools

Fabric's agent-facing updates address finding the right data tools and turning data into editable operational views. Discovery and generation reduce manual setup, but generated queries still require validation.

OneLake endpoint discovery and remote MCP

OneLake offers a remotely hosted Model Context Protocol (MCP) server in preview, while retaining the local MCP option. Fabric Core MCP now includes tools for managing shortcuts and OneLake security. A preview endpoint discovery API returns available query engines, skills, MCP servers, and endpoints for Fabric items.

GitHub Copilot uses endpoint discovery to select engine-specific querying tools and run Spark analysis. The underlying data paths remain distinct: shortcuts query external data without copying it, while shortcut transforms convert files into tables and maintain them as files change.

Editable Real-Time Dashboards from prompts

Copilot can generate a Real-Time Dashboard from tables in a selected Eventhouse database through Generate with Copilot (Preview). Users describe their goals and refine the proposed dashboard plan through follow-up messages. Generated dashboards remain editable, including visual types and the queries behind individual tiles.

The stadium example compares transit demand with gate entries and queue lengths across stations, transit lines, gates, and zones. Before using generated dashboards for operational decisions, teams should verify the queries, data relationships, and meanings of measures.

Fabric connectors support application and pipeline workflows

Fabric's connector updates cover application access to data and pipeline execution of service operations. The supported actions differ by connection type, making connector selection part of application and workflow design.

Typed access from Fabric Apps

Fabric Apps supports connectors for Lakehouse SQL analytics endpoints, warehouses, SQL databases in Fabric, and Power BI semantic models. Lakehouse endpoint connectors support reads, warehouse and SQL database connectors support read, create, update, and delete operations, and semantic model connectors support executeQuery.

The CLI example adds a fabric-warehouse connector scoped to reads using workspace and item identifiers. Added connectors become part of the project configuration and are available through a generated connector interface.

Service operations inside pipelines

Fabric Actions invokes supported Fabric REST API operations directly from pipelines with built-in connection and credential handling. Authors can browse or search operations and configure required parameters without knowing the underlying endpoint, and the response becomes available through the activity output. Supported operations are expanding across data, analytics, workspace, and capacity resources.

Business Actions adds 25 third-party connectors spanning communications, customer relationship management, document generation, electronic signatures, security, and other business processes. Together, the preview capabilities let pipeline authors combine supported Fabric operations with external business actions.

Other Machine Learning News

Foundry Local's on-device coding setup connects to GitHub Copilot through Bring Your Own Key and a local API endpoint, requiring CLI 0.13.0 or later, an NVIDIA GPU with at least 24 GB of VRAM, and separately installed CUDA and compatible cuDNN runtimes. Four-bit quantization reduces Qwen3.8-27B to approximately 18 GB, while ONNX Runtime provides speculative decoding, key-value cache quantization, prefix caching, continuous batching, and CPU offloading.

In the author's Work IQ experiment, Jev answered first on 12 of 13 questions with a 5.6-second median versus 22.3 seconds for Work IQ ask, and scored 131 of 134 routing questions versus 128 each for Microsoft-Decision-1 and OpenAI's Decisions API. Treat those comparisons as exploratory: the revised questions were refined against Jev rather than held out, Microsoft-Decision-1 timings were per API call rather than full routing plan, and its end-to-end Work IQ races had not been rerun.

Fabric Eventstream's Reference Data Join enriches live events from OneLake Lakehouse Delta tables, including shortcuts, using visual or SQL inner and left outer joins with configurable reference-data refresh intervals. The Azure IoT Hub example adds shipment and route context to trailer telemetry before downstream delivery, but the larger payloads and duplicated reference values can increase transport and storage costs.

Organizations can assign users or groups to a designated Fabric capacity so eligible Copilot usage is billed separately from workspace workloads. F2 is the minimum SKU, but sizing should follow observed usage rather than the minimum alone.

Also relevant this week