Automagic Data Anomaly Detection in Fabric | Data Exposed MVP Edition

Matt Gordon and Anna Hoffman walk through a practical Microsoft Fabric setup for detecting data anomalies in near real time and responding automatically.

Overview

Scenario and goal

• The demo uses a “Global Glass” sporting event scenario.
• The goal is to automatically detect anomalies (for example, in ticket sales data or ETL pipelines) and take action when something looks wrong.

Streaming change events into Fabric

• The presenters show how to set up Change Event Streaming from Azure SQL / SQL Server into Microsoft Fabric.
• They then work with the resulting event stream in Fabric.

Core Fabric components used

• Eventstream (Fabric): used to ingest and route streaming events.
• Eventhouse (Fabric): introduced as the destination/engine for working with the streaming data.

Anomaly detection

• The video covers anomaly detection in Fabric and includes a segment focused on the code for detecting anomalies.

Alerting and automated actions

• The presenters demonstrate creating alerts using Activator in Fabric.
• They discuss follow-up actions such as:
  • Custom actions
  • Notebooks
  • Other automated responses

Tips for getting started

• Practical tips and tricks are shared, including a note on using managed identities.
• The approach is summarized as “SQL plus customization”.

Resources

• Fabric Updates Blog: Stream SQL Change Events to Microsoft Fabric Eventstream: https://community.fabric.microsoft.com/t5/Fabric-Updates-Blog/Stream-SQL-Change-Events-to-Microsoft-Fabric-Eventstream-with/ba-p/5172004
• Microsoft Learn tutorial: Stream SQL change events to Eventstream: https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/stream-sql-change-events-to-eventstream
• Microsoft Learn primer: Anomaly detection in Fabric (Eventhouse): https://learn.microsoft.com/en-us/fabric/real-time-intelligence/anomaly-detection?tabs=eventhouse