In this Microsoft Ignite 2025 session, Microsoft Events speakers explain how to architect agentic memory in generative AI apps with Azure Cosmos DB, highlighting security, governance, and advanced AI retrieval techniques.

From DEV to PROD: Building Agentic Memory with Azure Cosmos DB

Session Overview

This Microsoft Ignite 2025 breakout (BRK135) delivers an advanced look at architecting agentic memory for generative AI applications using Azure Cosmos DB. The session targets developers and architects seeking to unify data platforms and leverage advanced AI features securely in production environments.

Key Topics Covered

• Latest AI and Search Features in Cosmos DB: Announcements and practical walkthroughs of new capabilities to enhance AI-driven apps.
• Agentic Memory Architecture: Explanation of both short-term and long-term memory paradigms in agentic applications, including design requirements and strategies for each.
• Efficient Retrieval with Summarization and Semantic Search: Techniques to utilize summarization for storing long-term memory, and leveraging semantic hybrid queries within Cosmos DB for efficient AI data retrieval.
• Query Patterns:
  • SQL-like queries
  • Semantic queries
  • Hybrid approaches for flexible data access
• Latency and Architectural Considerations: Discussion of potential latency pitfalls, especially in on-premise environments, and functional requirements to address them.
• Compliance, Security, and Governance: Best practices to prevent data oversharing, manage sprawl, safeguard AI workloads from threats, and ensure regulatory compliance and responsible data governance throughout the application lifecycle.
• Real-World Implementation: Case study featuring Walmart Chile on customer-centric AI strategies leveraging these capabilities.

Session Structure

• 0:00 — New AI and Search Features Announced for Cosmos DB
• 6:40 — Core Concepts: Generative AI Applications & Prompts
• 7:26 — Defining Agentic Memory (Short-Term & Long-Term)
• 12:45 — Summarization for Memory & Retrieval/Semantic Search
• 14:12 — Retrieval: SQL-like, Semantic, Hybrid Queries
• 21:02 — Latency and Limitations (on-premise architecture)
• 22:26 — Designing for Short-Term Memory (functional needs)
• 32:01 — Walmart Chile Perspective: Customer-Centric AI

Speakers

• Derek Boudreau
• Kendall Brasch
• James Codella
• Felipe Morales Heerlein (Walmart Chile)

Further Resources

• Session resources on Azure SQL AI Development
• Microsoft Ignite 2025 On-demand Sessions

Takeaways

• Framework for building advanced AI memory features with Cosmos DB
• Approaches for securing, governing, and scaling enterprise AI workloads
• Patterns for reliable, efficient data retrieval in AI-powered apps
• Security and compliance must be foundational for production deployments

This session provides actionable guidance and advanced strategies for anyone working on AI-enabled enterprise data platforms with Microsoft technologies.