Beyond Vector Search: What RAG Actually Needs

Yohan Lasorsa explains why many RAG implementations fail when they rely on vector search as the default retrieval strategy, and demonstrates how to improve retrieval quality using agentic RAG with Azure Cosmos DB hybrid search.

Full summary based on transcript

Why “vector search by default” breaks RAG

• The session focuses on a common failure mode in RAG systems: the agent returns confident but wrong answers because retrieval quietly returns irrelevant or low-signal results.
• The core claim is that vector search alone is often not enough when users ask for specific, exact, or keyword-heavy queries.

What RAG systems actually need

• Better retrieval strategies than “always do vector search.”
• A way to combine semantic similarity with lexical matching so the system can retrieve:
  • semantically related content (vector)
  • exact terms, identifiers, and specific phrases (full-text)

Agentic RAG: letting the agent decide when to search

• The session describes an agentic approach where the agent decides whether retrieval is needed instead of blindly retrieving on every request.
• This is positioned as a practical pattern to reduce unnecessary retrieval calls and improve answer quality when retrieval is required.

Using Azure Cosmos DB hybrid search (vector + full-text)

• The demo extends an existing working AI agent (using MCP and a real database) with hybrid search in Azure Cosmos DB.
• Hybrid search combines:
  • vector search for semantic similarity
  • full-text search for precise matching
• The goal is to retrieve “what matters” for the question, especially when specificity is important.

Live demo and reusable patterns

• The session promises live code and a live demo.
• The takeaway is a set of patterns viewers can reuse to:
  • diagnose retrieval quality issues
  • choose hybrid retrieval over vector-only retrieval
  • implement agent-driven decisions about when to retrieve

Links from the description

• Cosmos DB Conf 2026 playlist: https://aka.ms/CosmosConf26Playlist
• Cosmos Conf Challenge (DP-420 voucher for first 500): https://aka.ms/CosmosDBConfChallenge
• Post-event survey: https://aka.ms/CosmosConf2026Survey
• Conference site: https://aka.ms/azurecosmosdbconf
• Speaker page: https://developer.azurecosmosdb.com/conf/#speaker/yohan-lasorsa
• Yohan Lasorsa (LinkedIn): https://www.linkedin.com/in/yohanlasorsa
• Yohan Lasorsa (X): https://x.com/sinedied