How Project HydraFusion reduces the cost of frontier AI
Aashna Garg and Julia Kasper explain Project HydraFusion in GitHub Copilot, focusing on how orchestrating multiple models per request (instead of selecting a single model) can reduce frontier-model costs, and how the approach is evaluated with demos and benchmarking.
Overview
The video introduces Project HydraFusion, a GitHub Copilot approach that aims to reduce the cost of using frontier AI models by orchestrating existing models per request rather than always choosing a single model.
What HydraFusion is trying to solve
- Many teams focus on building or choosing a “better” single model.
- HydraFusion focuses on optimizing the path a request takes by combining multiple models in a workflow.
- The presenters describe a ceiling where selection (picking one model) stops improving outcomes/costs, while composition (combining models) can continue to help.
Where HydraFusion shows up
- The presenters point to HydraFusion in the GitHub Copilot CLI.
Why “model routing” alone isn’t enough
- The video frames routing as having “two jobs”:
- Selecting an option for a request.
- Handling the broader workflow needs around that selection.
- HydraFusion is positioned as going beyond routing by using composed workflows.
The three workflows (as described in the video chapters)
Cascade workflow
- Demonstrated on a failing test scenario.
Critic workflow
- Demonstrated on authentication changes.
- The presenters emphasize that the critic reviews but never edits.
Side-by-side cost comparison
- A demo compares costs directly.
- The presenters highlight that using two models can still cost less than a single approach in some cases.
Reported savings and evaluation
- The video calls out session savings figures of 42% and 47%.
- The presenters discuss how HydraFusion was evaluated.
- They also explain why they built Checkpoint Bench as part of the evaluation approach.
Resources
- Learn more about HydraFusion: https://gh.io/ghcpdayhydrafusion
- Get hands-on with the Copilot app: https://gh.io/ghcpdaycopilotapp
- Install the GitHub Copilot CLI: https://gh.io/ghcpdaycopilotcli
Video chapters
- 0:00 Intro: meet Julia and Aashna
- 0:52 Finding Hydra Fusion in the Copilot CLI
- 1:35 What is Hydra Fusion?
- 2:01 Why model routing isn't enough
- 2:52 The router's two jobs
- 3:19 Selection has a ceiling, composition doesn't
- 4:04 The three workflows explained
- 4:56 Why the critic reviews but never edits
- 5:50 How two models still cost less
- 6:15 Demo: side-by-side cost comparison
- 7:31 Cascade workflow on a failing test
- 8:19 Critic workflow on auth changes
- 8:46 Session savings: 42% and 47%
- 9:37 How Hydra Fusion was evaluated
- 10:53 Why we built Checkpoint Bench
- 12:48 Try it and send feedback
- 13:38 Final word: stop being the router