Teach AI agents to manage context with Elastic Agent Builder | DEMSP395

Mike Richter shows how Elastic Agent Builder (Elasticsearch 9.4) can help AI agents manage long-running context by using a conversation context store, selective compaction, and dynamically loaded skills, with an emphasis on deploying these patterns in the Microsoft ecosystem on Azure and with Azure AI Foundry models.

Overview

This Build 2026 demo focuses on practical ways to deal with LLM context limits during longer tasks:

• Avoiding bloated prompts and context drift
• Reducing input token usage while keeping responses accurate
• Keeping enterprise data governance and scalability in mind

The session positions Elastic running in Azure and integrating with Microsoft model endpoints (Azure AI Foundry models) as the environment for these patterns.

Key ideas covered

Why context management breaks down in long tasks

• Long-running agent workflows tend to accumulate conversation history and tool outputs.
• This can lead to:
  • Prompt bloat (too many tokens sent each turn)
  • Context drift (the model starts losing the thread or over-weighting irrelevant history)

Letting agents manage their own memory

The session describes an approach where the agent is responsible for managing what context is kept and what is compacted:

• Conversation context store to persist relevant state outside the prompt
• Selective compaction to summarize or compress older context
• Dynamically loaded skills so the agent can pull in capabilities only when needed

Elastic capabilities used for agentic and RAG scenarios

• Elastic’s vector support is highlighted for RAG and agentic applications.
• The goal is to bridge contextual gaps across disparate data sources by retrieving the right information at the right time, rather than stuffing everything into the prompt.

Secure dispatch and data efficiency

The session calls out:

• Dispatching responses securely
• Improving data/compute efficiency
• Using secure external data connectors to access enterprise data sources while maintaining governance controls

Demo walkthrough (high level)

Skills as building blocks

• Skills are presented as composable building blocks for agent behavior.
• The demo includes creating a custom financial exposure skill.

Tool creation with ES|QL

• ES|QL (ESQL) is introduced as a query language used in the workflow for tool creation.

Workflow integration example

• The workflow includes sending summaries to Slack channels, showing how an agent can produce outputs for downstream collaboration tools.

Practical next steps mentioned

The session points to these implementation building blocks as takeaways:

• Deploying dynamically loaded skills
• Setting up a conversation context store
• Implementing selective compaction to control token usage
• Using secure external data connectors for governed enterprise data access

Speakers

• Mike Richter (Microsoft Partner Solution Architect)
• Deepti Dheer