One model deployment to route them all

Microsoft Developer uses a “Sparkle Cupcakes” agent scenario to show why mixed traffic (simple FAQs vs complex catering requests) benefits from routing rather than forcing every request through one model.

Overview

Meet the Azure AI Foundry Model Router

• The video introduces the Azure AI Foundry model router as a single deployment endpoint that can route requests to different underlying models.
• The goal is to send simpler requests to cheaper models and more complex reasoning requests to frontier models, in real time.

Supported models and deployments

• The router works across supported models and deployments available in Azure AI Foundry.
• The router is positioned as a way to avoid maintaining separate application logic for model selection.

Configure routing priorities

• The presenter configures routing priorities to influence how requests are assigned to models.
• The routing configuration is used to balance:
  • Cost (cheaper models for simpler tasks)
  • Quality (stronger models for harder tasks)

Test the “Cupcake Agent” with a single-line change

• The agent is pointed at the model router by changing a single line (swapping the target from a single model deployment to the router deployment).
• This keeps the agent’s integration surface small while enabling dynamic model selection.

Route complex reasoning tasks vs general knowledge questions

• The presenter tests:
  • Complex catering/order scenarios that require more reasoning
  • General knowledge / simpler questions
• The router selects different models depending on the request complexity.

Monitor usage and model selection

• The video shows logging/monitoring of which model served each request.
• The router is evaluated as a unit to confirm quality remains acceptable while overall cost drops.