Build persistent and scalable AI agent memory with TiDB | ODSP918
Ravish Patel presents an approach to building durable “memory” for AI agents using TiDB as the backing data store, focusing on hybrid retrieval (vector + keyword) and transactional consistency for agent workflows.
Overview
AI agents often need different data infrastructure than human-facing apps, especially when they must:
- Handle bursty workloads
- Support massive concurrency
- Perform constant context recall across sessions (so the agent doesn’t “forget”)
This session shows how TiDB targets these agentic workload needs by combining multiple retrieval and storage capabilities in one system.
Why agent memory is hard
The talk frames three core challenges for agent memory systems:
- Bursty workloads: traffic patterns can spike unpredictably.
- Massive concurrency: many agent sessions and tool calls can run in parallel.
- Constant context recall: agents need to retrieve relevant prior context repeatedly and quickly.
TiDB capabilities highlighted for agentic workloads
TiDB is presented as a database that can unify several needs for agent memory:
- SQL for structured data access
- Vector search for semantic retrieval
- BM25 for keyword-based retrieval
- A unified table approach to store and query memory
- ACID transactions to keep multi-step agent updates consistent
Demo: inserting memories with automatic embedding generation
The demo walkthrough includes inserting user “memories” and generating embeddings automatically.
Key idea:
- When new memory entries are inserted, embeddings are generated (using Azure OpenAI embeddings) so the same data can be retrieved semantically later.
Demo: hybrid retrieval (vector + keyword) with RRF
The session demonstrates hybrid search that combines:
- Vector similarity results
- BM25 keyword results
These are merged using Reciprocal Rank Fusion (RRF) to produce a single ranked set of results suitable for agent context recall.
Transactions for consistent multi-table updates
A section of the talk focuses on using transactions across multiple tables to ensure consistent state changes.
The demo shows a transaction flow that performs consistent multi-table insert and update operations, emphasizing that agent systems often need to update multiple related records as a single unit of work.
Case study
The session references a case study of Manus AI deploying millions of agent databases on TiDB.
Azure deployment context
The description positions the approach as deployable in Azure environments, with Azure OpenAI embeddings used as part of the retrieval pipeline for agent memory.