Agentic RAG and chat completion in the Microsoft SQL engine | Data Exposed

Bob Ward explains how Microsoft SQL (SQL Server, Azure SQL, and SQL DB in Fabric) can be used for chat completion and agentic RAG scenarios, including making REST calls to models, building prompts in SQL, and adding security controls like immutable logging, content moderation, and prompt-injection protection.

Overview

The episode shows how to use capabilities in Microsoft SQL to:

• Work with vectors in SQL for retrieval-augmented generation (RAG) scenarios
• Make REST calls from SQL to call model endpoints
• Process model responses inside SQL
• Integrate SQL-based AI workflows with Azure AI Foundry, Foundry Gateway, and Microsoft Teams

Topics covered (from the video chapters)

SQL has vectors (and more)

• Introduces vector capabilities in Microsoft SQL for AI/RAG-style workloads.

Making REST calls from Microsoft SQL

• Demonstrates calling external endpoints from SQL via REST.

Calling models and processing results with SQL

• Shows how to invoke model endpoints (chat completion) and return/process results in SQL.

Demo: building the prompt

• Walks through constructing prompts as part of the SQL-driven workflow.

Immutable logging with SQL Ledger

• Uses SQL Ledger to keep an immutable log of activity (useful for auditing and traceability).

Tips and tricks

• Shares practical guidance for building and operating this pattern.

Content safety and moderation with the gateway

• Uses a gateway layer to apply content safety/moderation controls.

Using Foundry SDK with SQL

• Covers integrating with the Azure AI Foundry SDK from the SQL-based workflow.

Preventing prompt injection with the gateway

• Uses the gateway to reduce prompt-injection risk.

Exposing a Foundry application from Microsoft Teams

• Shows how to surface the resulting Foundry application through Microsoft Teams.

More secure, less data movement

• Emphasizes reducing data movement while improving security controls around the AI workflow.