Build an AI agent without leaving VS Code
Microsoft Developer demonstrates building the “Sparkles” AI agent in VS Code using the Microsoft Foundry Toolkit, including deploying a model, wiring up an MCP server, and validating the agent with traces and evaluation runs for repeatable testing.
Overview
The video walks through creating a production-oriented agent project and iterating on it end-to-end from the development environment.
Build production-ready AI agents
- Goal: build an agent that can use the shop’s prompts and live ordering tools.
- Keep project artifacts (traces, evaluation datasets, and results) alongside the project to support repeatable testing.
Create a Foundry project
- Create a project in Azure AI Foundry.
- Use the Microsoft Foundry Toolkit to work with the project from VS Code.
Select and deploy a model (GPT-5.4)
- Select and deploy the GPT-5.4 model.
- Configure the deployed model for use by the agent.
Configure the prompt agent
- Configure the agent’s prompt setup (the “Prompt Agent”).
Connect the MCP server
- Configure an MCP connection.
- Connect the agent to an MCP server so it can access external tools.
Test the cupcake ordering flow
- Invoke the agent from the development environment.
- Verify it can execute the cupcake ordering flow using the shop’s prompts and live ordering tools.
Inspect observability traces
- Inspect stored traces to understand agent behavior.
- Keep traces with the project for later review and repeatability.
Set up agent evaluations
- Create evaluation datasets.
- Generate test cases with Copilot.
- Run evaluation metrics.
- Review evaluation results and wrap up.