Introduction
Hey everyone!
You can find all the C# code samples here: MicrosoftAgentFramework GitHub Repo
Before reading about chat history in Microsoft Agent Framework, it is worth to get familiar with this post, especially if MAF is a completely new topic for you:
When you start implementing an agentic solution, one of the biggest challenges at the beginning isn’t choosing the LLM, prompts or tools, because these are things you can always replace without a lot of effort. However, there is one decision which affects your further actions greatly: it is all about deciding what your chat history storage strategy is.
If you are just about to make such a decision, especially when thinking about chat history in Microsoft Agent Framework, this post is for you! I am sure that after reading it, you will have enough information to choose the best strategy for your individual use case.
Let’s get started.
The Blueprint: How to Choose Your Storage Strategy
Before we look at the code, I want to show you why this decision is so important. Choosing how to save Chat History in Microsoft Agent Framework is not just about picking a database. It changes how your whole application works.
3 Important Questions
Here are the 3 critical questions, you should always ask before writing any storage infrastructure code:
1. What features must the user experience support?
Are you building a simple conversation where messages just follow one after another? Or do you need more complex options? Think about how your database will handle it if a user clicks an “Undo” button to remove the last message, wants to start a new side conversation from an old prompt, or needs to copy a whole active chat into a new window.
2. What are the security rules about where data can go?
In large companies, this is the most important question: can we delegate our data to an external third party service, or must it stay strictly inside our company’s private network? When managing Chat History in Microsoft Agent Framework, you must check if security rules allow you to send conversation logs to outside vendors. If strict company policies say that data cannot leave your internal infrastructure, you cannot use these ready-made SaaS platforms. You will have to build, host, and manage the database yourself. Also, how will you automatically delete old chats if corporate rules say that data must disappear after 30 days?
3. Are you going to manage the chat history yourself or delegate it?
This is a choice between building your own data storage or letting a cloud platform do the heavy lifting for you. When planning Chat History in Microsoft Agent Framework, you need to decide if you want to write the code and control the databases yourself, or delegate the whole job to a managed cloud service like Microsoft Foundry.
This decision divides your options into two completely different approaches, which we will look at next.
Service Managed Storage
In this approach, the cloud provider stores the conversation data on its own servers. Your application only keeps a reference ID, and the external service automatically adds the past messages whenever the agent processes a request.
Pros:
- Your local code stays very simple because you do not have to build storage logic.
- The cloud platform automatically manages message limits and shortens long text for you.
- You send less data per request, which reduces your network traffic.
Cons:
- Your private company data lives completely on the provider’s servers.
- You have zero control over how the service decides to summarize or delete old messages.
- It creates provider lock-in because your conversation state is trapped in their system.
Client Managed Storage
Here, your application maintains the complete chat history and sends the relevant messages with every request. The AI service remains completely stateless, meaning it processes your prompt and immediately forgets the interaction.
Pros:
- You have full control over data location, privacy, and security boundaries.
- It is incredibly easy to switch AI vendors because no state is locked in an external system.
- You decide exactly how to clean, summarize, or shorten old conversation turns.
Cons:
- Your request payloads become much larger because you must send past messages every time.
- Your client-side logic is more complex since you must write all the database code.
- You have to build and maintain your own logic to handle conversation size limits as the chat grows.
In this post I am going to focus only on the Client Managed Storage.
The core abstraction: ChatHistoryProvider
When I create a new AIAgent, I can specify a ChatHistoryProvider. If I do not provide one, the framework automatically uses the InMemoryChatHistoryProvider. As the name suggests, this provider stores all the messages for Chat History in Microsoft Agent Framework in memory.
However, I want to build a custom provider to show you the 4 main methods you need to know. Let’s create a custom provider to see how it works:
public class SimpleInMemoryChatHistoryExample
{
private readonly AIAgent _mafAgent;
public SimpleInMemoryChatHistoryExample()
{
// trimmed for brevity
_mafAgent = chatClient
.AsAIAgent(new ChatClientAgentOptions()
{
Name = "Agent with a simple in memory history provider",
ChatOptions = new ChatOptions { Instructions = "You are a helpful assisant." },
ChatHistoryProvider = new SimpleInMemoryChatHistoryProvider()
});
}
}Here is how this class is implemented:
public class SimpleInMemoryChatHistoryProvider : ChatHistoryProvider
{
private readonly ProviderSessionState<State> _sessionState;
public SimpleInMemoryChatHistoryProvider(
Func<AgentSession?, State>? stateInitializer = null,
string? stateKey = null)
: base(
provideOutputMessageFilter: null,
storeInputRequestMessageFilter: null,
storeInputResponseMessageFilter: null)
{
_sessionState = new ProviderSessionState<State>(
stateInitializer ?? (_ => new State()),
stateKey ?? GetType().Name);
}
public override IReadOnlyList<string> StateKeys => [_sessionState.StateKey];
protected override ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(InvokingContext context, CancellationToken cancellationToken = default)
{
var messages = _sessionState.GetOrInitializeState(context.Session).Messages;
return new(messages);
}
protected override ValueTask StoreChatHistoryAsync(InvokedContext context, CancellationToken cancellationToken = default)
{
var state = _sessionState.GetOrInitializeState(context.Session);
var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []);
state.Messages.AddRange(allNewMessages);
_sessionState.SaveState(context.Session, state);
return default;
}
public class State
{
public List<ChatMessage> Messages { get; set; } = [];
}
}When building custom storage for Chat History in Microsoft Agent Framework, I want to focus on the two methods you will work with most often:
ProvideChatHistoryAsync: This method runs before every call to the LLM. It provides the past messages that need to be included in the prompt sent to the model.StoreChatHistoryAsync: This method runs after each call to the LLM. It is responsible for saving the new messages to your chosen database (in this example, in memory).
But now another question arises: where is this state actually stored?
It is saved inside context.Session, which is an AgentSession type. The ProviderSessionState class you saw at the top is just a wrapper around a StateBag property owned by the AgentSession. You can think of StateBag as a thread-safe dictionary because it uses ConcurrentDictionary behind the scenes to hold the Chat History in Microsoft Agent Framework.
Below, you can see its only two methods:
public TState GetOrInitializeState(AgentSession? session)
{
if (session?.StateBag.TryGetValue<TState>(this.StateKey, out var state, this._jsonSerializerOptions) is true && state is not null)
{
return state;
}
state = this._stateInitializer(session);
if (session is not null)
{
session.StateBag.SetValue(this.StateKey, state, this._jsonSerializerOptions);
}
return state;
}
public void SaveState(AgentSession? session, TState state)
{
if (session is not null)
{
session.StateBag.SetValue(this.StateKey, state, this._jsonSerializerOptions);
}
}So we have 2 methods covered, but there are 2 additional ones that you would use only when you want to have the full-control:
InvokingCoreAsync: This method loads your past messages by calling ProvideChatHistoryAsync. It applies your optional filter (_provideOutputMessageFilter) and marks them as history data by adding AgentRequestMessageSourceType.ChatHistory to the AdditionalProperties dictionary inside the ChatMessage class. Finally, it combines these old messages with the new user request.
protected virtual async ValueTask<IEnumerable<ChatMessage>> InvokingCoreAsync(InvokingContext context, CancellationToken cancellationToken = default)
{
var output = await this.ProvideChatHistoryAsync(context, cancellationToken).ConfigureAwait(false);
if (this._provideOutputMessageFilter is not null)
{
output = this._provideOutputMessageFilter(output);
}
return output
.Select(message => message.WithAgentRequestMessageSource(AgentRequestMessageSourceType.ChatHistory, this.GetType().FullName!))
.Concat(context.RequestMessages);
}InvokedCoreAsync: This method stops immediately if there is an error during the agent turn. If everything is successful, it uses separate filters for context.RequestMessages > _storeInputRequestMessageFilter and context.ResponseMessages > _storeInputResponseMessageFilter so you only save the new conversation turn. Finally, it calls StoreChatHistoryAsync.
protected virtual ValueTask InvokedCoreAsync(InvokedContext context, CancellationToken cancellationToken = default)
{
if (context.InvokeException is not null)
{
return default;
}
var subContext = new InvokedContext(context.Agent, context.Session, this._storeInputRequestMessageFilter(context.RequestMessages), this._storeInputResponseMessageFilter(context.ResponseMessages!));
return this.StoreChatHistoryAsync(subContext, cancellationToken);
}How does the framework stop old messages from being saved twice? If you do not provide a custom filter, it automatically uses DefaultExcludeChatHistoryFilter. This filter checks the AdditionalProperties dictionary and removes any message marked as ChatHistory from the input request list.
You can see this default setup inside the ChatHistoryProvider constructor below:
private static IEnumerable<ChatMessage> DefaultExcludeChatHistoryFilter(IEnumerable<ChatMessage> messages) => messages.Where(m => m.GetAgentRequestMessageSourceType() != AgentRequestMessageSourceType.ChatHistory);
private static IEnumerable<ChatMessage> DefaultNoopFilter(IEnumerable<ChatMessage> messages)=> messages;
protected ChatHistoryProvider(
Func<IEnumerable<ChatMessage>, IEnumerable<ChatMessage>>? provideOutputMessageFilter = null,
Func<IEnumerable<ChatMessage>, IEnumerable<ChatMessage>>? storeInputRequestMessageFilter = null,
Func<IEnumerable<ChatMessage>, IEnumerable<ChatMessage>>? storeInputResponseMessageFilter = null)
{
this._provideOutputMessageFilter = provideOutputMessageFilter;
this._storeInputRequestMessageFilter = storeInputRequestMessageFilter ?? DefaultExcludeChatHistoryFilter;
this._storeInputResponseMessageFilter = storeInputResponseMessageFilter ?? DefaultNoopFilter;
}Let’s visualize the flow now, so that you can remember that part better.
Phase 1: Before the LLM call.
Phase 2: After the LLM call.
As you can see, when writing a custom provider for chat history in Microsoft Agent Framework, you will most often override only ProvideChatHistoryAsync and StoreChatHistoryAsync, and maybe define one of the 3 filters we discussed. I believe overriding the other 2 core methods is only necessary if you need complete control over the entire pipeline.
This is all I need to cover about how ChatHistoryProvider works behind the scenes. Now, let’s start persisting our chat history in an external storage so that it is not only stored in memory.
Persisting Chat History in Azure
In the previous example, I stored all the messages in a simple list using ProviderSessionState<State> _sessionState. But what should we actually save when we want to move chat history to external storage?
We need to save context data that helps us uniquely identify a specific conversation. At a minimum, this is just a ConversationId. However, you can also store extra details like a UserId or a TenantId if you are building multi-tenant solutions.
In the next examples, I will be storing all three of these properties.
Azure Blob Storage
In this section, I will describe the BlobChatHistoryProvider (you can find the full source code in the GitHub repository), which saves our conversation data into Azure Blob Storage. For this provider, I only need to override two main methods: ProvideChatHistoryAsync and StoreChatHistoryAsync.
Let’s focus on the architectural setup instead of the raw file handling logic, as the read and write code is quite standard.
protected override async ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(
InvokingContext context,
CancellationToken cancellationToken = default)
{
var state = _sessionState.GetOrInitializeState(context.Session);
var blobName = BuildBlobPath(state);
// Save
} As you can see, the provider reads the conversation state from context.Session. Under the hood, this fetches data from Session.StateBag["BlobChatHistoryProvider"]. Once it has the state, it builds the storage path using the following helper method:
private static string BuildBlobPath(State state)
{
return $"{state.TenantId}/users/{state.UserId}/{state.ConversationId}.json";
}But how do we initialize this state and pass it along in the first place? Let’s look at how the main application loop handles the user interaction:
public async Task RunAsync()
{
while (true)
{
Console.Write("You: ");
var input = Console.ReadLine();
if (string.IsNullOrWhiteSpace(input))
{
break;
}
var agentSession = await _mafAgent.CreateSessionWithUserContextAsync(
tenantId: FakeUserData.TENANT_ID,
userId: FakeUserData.USER_ID,
conversationId: FakeUserData.CONVERSATION_ID);
var response = await _mafAgent.RunAsync(input, agentSession);
Console.WriteLine($"Agent: {response}");
}
}To make this clean, I created two extension methods that help store and retrieve the user context inside the session’s internal state bag:
using Microsoft.Agents.AI;
namespace _02_ChatHistory
{
public static class Extensions
{
private const string TenantIdKey = "App:UserContext:TenantId";
private const string UserIdKey = "App:UserContext:UserId";
private const string ConversationIdKey = "App:UserContext:ConversationId";
public static async ValueTask<AgentSession> CreateSessionWithUserContextAsync(
this AIAgent agent,
string tenantId,
string userId,
string conversationId)
{
var session = await agent.CreateSessionAsync();
session.StateBag.SetValue(TenantIdKey, tenantId);
session.StateBag.SetValue(UserIdKey, userId);
session.StateBag.SetValue(ConversationIdKey, conversationId);
return session;
}
public static (string? TenantId, string? UserId, string? ConversationId) GetUserContext(this AgentSession session)
{
if (session == null)
{
return (null, null, null);
}
session.StateBag.TryGetValue<string>(TenantIdKey, out var tenantId);
session.StateBag.TryGetValue<string>(UserIdKey, out var userId);
session.StateBag.TryGetValue<string>(ConversationIdKey, out var conversationId);
return (tenantId, userId, conversationId);
}
}
}
In a real enterprise application, you would likely extract this user context from an incoming JWT security token, but I used hardcoded values here to keep the example straightforward.
Now that we have a way to inject user data into the session, the next critical step is configuring the stateInitializer when registering the chat history in Microsoft Agent Framework:
_mafAgent = chatClient
.AsAIAgent(new ChatClientAgentOptions()
{
Name = "Agent which stores chat history in Blob storage",
ChatOptions = new ChatOptions { Instructions = "You are a helpful assisant." },
ChatHistoryProvider = new BlobChatHistoryProvider(
blobContainerClient,
stateInitializer: (agentSession) =>
{
var (tenantId, userId, conversationId) = agentSession!.GetUserContext();
return new BlobChatHistoryProvider.State(conversationId!, tenantId, userId);
})
});With this configuration in place, whenever _sessionState.GetOrInitializeState(context.Session) runs inside the ProvideChatHistoryAsync method, it can read the user context and build the exact blob path it needs.
This same approach applies when the framework automatically triggers the output pipeline to save new messages:
protected override async ValueTask StoreChatHistoryAsync(
InvokedContext context,
CancellationToken cancellationToken = default)
{
var state = _sessionState.GetOrInitializeState(context.Session);
var blobName = BuildBlobPath(state);
// Save
}Why Use Separate Key(s) for User Context?
I want to explain why I keep the user context in the StateBag using separate keys (you could store that data using a single key too) instead of wrapping it all directly inside the provider’s specific state.
The main reason is that I want this user context data to be easily shared and reused by various other components in the framework, such as AIContextProviders. Technically, we could save these values directly inside the provider’s key, like StateBag["BlobChatHistoryProvider"]. However, keeping them separate is a much cleaner architectural solution. It ensures that your core user identity data is not tightly coupled to that specific provider, making it fully available to any other component that needs it.
Demo
Let’s see how it works now!
You: Hi, I am Michal
Agent: Hi Michal! Nice to meet you. How can I help you today?
You:A new blob was created in Azure.
As you can see, it aligns with the hardcoded values I used. Let’s check the blob’s content now.
And now, thanks to that, when I run my simple console app again and ask the follow up question, it can provide the correct answer.
You: could you remind me my name?
Agent: Your name is Michal.
You:Azure Cosmos DB
Now, let’s focus on storing chat history in Microsoft Agent Framework using Azure Cosmos DB.
I started implementing my own logic for storing chat history in Cosmos DB, but then I realized that a native CosmosChatHistoryProvider already exists (however, at the time of writing, it is still in preview).
First, we need to install the Microsoft.Agents.AI.CosmosNoSql NuGet package. Then, we can register the CosmosChatHistoryProvider as our ChatHistoryProvider:
public class CosmosDbChatHistoryExample
{
private readonly AIAgent _mafAgent;
public CosmosDbChatHistoryExample()
{
// trimmed for brevity
_mafAgent = chatClient
.AsAIAgent(new ChatClientAgentOptions()
{
Name = "Agent which stores chat history in Azure Cosmos DB",
ChatOptions = new ChatOptions { Instructions = "You are a helpful assisant." },
ChatHistoryProvider = new CosmosChatHistoryProvider(
accountEndpoint: "https://deployed-in-azure-cosmosdb.documents.azure.com",
new DefaultAzureCredential(),
databaseId: "MyDatabase",
containerId: "ChatHistory",
stateInitializer: (agentSession) =>
{
var (tenantId, userId, conversationId) = agentSession.GetUserContext();
return new CosmosChatHistoryProvider.State(conversationId, tenantId, userId);
})
});
}
}Before creating a new container in your Azure Cosmos DB service, you have to decide how to partition your data. You have two options (when using this library) for logical partitions:
- Partition the data based on
ConversationId. - Use hierarchical partitioning:
TenantId->UserId->ConversationId.
⚠️ Important Note for Single Tenant Architectures:
In my code examples, I pass "Default" as a placeholder for TenantId. However, if you are building a single-tenant application, you should never use a static tenant string at the first level of a hierarchical partition key. Cosmos DB groups and routes data to physical partitions based on that first level key (of course, not only on the first level key). Using a single static value (the same Entra tenant ID) forces all traffic onto a single physical partition (until it reaches 50 GB and gets split into two partitions), completely breaking your horizontal scaling potential.
For single-tenant production deployments, I highly recommend either falling back to a single partition key using ConversationId, or writing a custom cosmos provider that structures a UserId -> ConversationId subpartition hierarchy because the native CosmosChatHistoryProvider library only supports the flat ConversationId or the full 3 level multi-tenant scheme out of the box.
If you are building a multi-tenant enterprise system, dumping all customer data into a single shared container is highly questionable for security and compliance reasons. A simple bug in your repository filtering logic could cause cross-tenant data leaks, so I am not sure why the creators of that library decided to expose such functionality. For me, it is encouragement to do things the wrong way.
There are two main reasons we want to design clean logical partitions (which applies to all use cases, not just AI or chat history):
- Performance: We want to fetch data as performantly as possible. Cosmos DB is excellent at point reads, but if you need to retrieve multiple items, you should aim to query them within a single logical partition.
- Transactions: When applying changes across multiple documents (whether using an ordinary document replacement or the Patch API), we want that operation to be transactional.
If you check the source code of the CosmosChatHistoryProvider, you will notice it uses the partition key for both pulling messages (for performance) and creating documents (for consistency).
Inside the CosmosChatHistoryProvider you can find this logic:
/// <summary>
/// Determines whether hierarchical partitioning should be used based on the state.
/// </summary>
private static bool UseHierarchicalPartitioning(State state) =>
state.TenantId is not null && state.UserId is not null;The state class for this provider is very similar to the custom one I created earlier, as it contains the same three properties. Below is the CosmosChatHistoryProvider.State constructor:
public State(string conversationId, string? tenantId = null, string? userId = null)
{
this.ConversationId = Throw.IfNullOrWhitespace(conversationId);
this.TenantId = tenantId;
this.UserId = userId;
}I will be using hierarchical partitioning, so my Cosmos DB container is configured to support that strategy.
Now, let’s look at the document schema that CosmosChatHistoryProvider uses under the hood:
private sealed class CosmosMessageDocument
{
public string Id { get; set; } = string.Empty;
public string ConversationId { get; set; } = string.Empty;
public long Timestamp { get; set; }
public string? MessageId { get; set; }
public string? Role { get; set; }
public string Message { get; set; } = string.Empty;
public string Type { get; set; } = string.Empty;
public int? Ttl { get; set; }
public string? TenantId { get; set; }
public string? UserId { get; set; }
public string? SessionId { get; set; }
}I am showing this schema deliberately because I highly recommend one more adjustment to your container settings: excluding the Message property from indexing.
Let’s run the app and see the results:
You: Hi, I am Michal
Agent: Hi Michal - nice to meet you! How can I help you today?
You:Let’s check Cosmos DB now.
Both messages were added successfully as separate documents, respecting our logical partitioning strategy. Here is the raw JSON data from the first document for better visibility:
{
"id": "f677ff05-3fba-471a-9331-2020f455d739",
"conversationId": "11925061-b53c-4fbd-8f35-7c7a673b25e6",
"timestamp": 1779892424,
"messageId": null,
"role": "user",
"message": "{\"AuthorName\":null,\"CreatedAt\":null,\"Role\":\"user\",\"Contents\":[{\"$type\":\"text\",\"Text\":\"Hi, I am Michal\",\"Annotations\":null,\"AdditionalProperties\":null}],\"MessageId\":null,\"AdditionalProperties\":null}",
"type": "ChatMessage",
"ttl": 86400,
"tenantId": "Default",
"userId": "123",
"sessionId": "11925061-b53c-4fbd-8f35-7c7a673b25e6",
"_rid": "9oRCALLeaYINAAAAAAAAAA==",
"_self": "dbs/9oRCAA==/colls/9oRCALLeaYI=/docs/9oRCALLeaYINAAAAAAAAAA==/",
"_etag": "\"0000bd00-0000-5600-0000-6a1700c90000\"",
"_attachments": "attachments/",
"_ts": 1779892425
}And here is the second document (the assistant response):
{
"id": "00403c34-bd53-44f7-863d-60821878dc7c",
"conversationId": "11925061-b53c-4fbd-8f35-7c7a673b25e6",
"timestamp": 1779892424,
"messageId": "chatcmpl-Dk9cGe0ggdtaw30c2Q0TkCCS39sje",
"role": "assistant",
"message": "{\"AuthorName\":\"Agent which stores chat history in Azure Cosmos DB\",\"CreatedAt\":\"2026-05-27T14:33:44+00:00\",\"Role\":\"assistant\",\"Contents\":[{\"$type\":\"text\",\"Text\":\"Hi Michal \\u2014 nice to meet you! How can I help you today?\",\"Annotations\":null,\"AdditionalProperties\":null}],\"MessageId\":\"chatcmpl-Dk9cGe0ggdtaw30c2Q0TkCCS39sje\",\"AdditionalProperties\":null}",
"type": "ChatMessage",
"ttl": 86400,
"tenantId": "Default",
"userId": "123",
"sessionId": "11925061-b53c-4fbd-8f35-7c7a673b25e6",
"_rid": "9oRCALLeaYIOAAAAAAAAAA==",
"_self": "dbs/9oRCAA==/colls/9oRCALLeaYI=/docs/9oRCALLeaYIOAAAAAAAAAA==/",
"_etag": "\"0000be00-0000-5600-0000-6a1700c90000\"",
"_attachments": "attachments/",
"_ts": 1779892425
}Let’s perform the final test now:
You: remind my name please
Agent: Your name is Michal.
You:Eveything works as expected!
To wrap things up, I want to remind you that you can customize the CosmosChatHistoryProvider by applying these additional settings. The first three filters are already familiar to you, so let’s focus on the remaining parameters:
ChatHistoryProvider = new CosmosChatHistoryProvider(
accountEndpoint: "https://deployed-in-azure-cosmosdb.documents.azure.com",
new DefaultAzureCredential(),
databaseId: "MyDatabase",
containerId: "ChatHistory",
stateInitializer: (agentSession) =>
{
var (tenantId, userId, conversationId) = agentSession.GetUserContext();
return new CosmosChatHistoryProvider.State(conversationId, tenantId, userId);
},
provideOutputMessageFilter: null,
storeInputRequestMessageFilter: null,
storeInputResponseMessageFilter: null)
{
MaxBatchSize = 100, // Gets or sets the maximum number of items per transactional batch operation
MaxItemCount = 100, // Gets or sets the maximum number of messages to return in a single query batch
MaxMessagesToRetrieve = 20, // Gets or sets the maximum number of messages to retrieve from the provider
MessageTtlSeconds = 86400, // Gets or sets the Time-To-Live (TTL) in seconds for messages
}MessageTtlSeconds: Configures the native Azure Cosmos DB Time-To-Live (TTL) in seconds (default=86400 > 24h) meaning old history records will automatically delete themselves from your container without requiring manual cleanup logic (if enabled, in my container settings I do not have that feature enabled).
MaxBatchSize: Controls the maximum number of history items processed in a single Cosmos DB transactional batch operation during saves (default=100).
MaxItemCount: Specifies the maximum number of messages returned in a single query batch roundtrip from the database (default=100).
MaxMessagesToRetrieve: Limits the total number of historical messages pulled from the provider into the current LLM context, which helps you manage your token budget (default=null).
There are 2 more Azure services I would like to describe here, even if I have not prepared C# example to show you how it works in practice.
Azure SQL Database
Another excellent service we can use to store chat history is Azure SQL Database (or any other relational database provider). If you already have a relational database in your infrastructure, leveraging it is a great choice to avoid introducing unnecessary infrastructure complexity.
While the data schema will look very similar to the Cosmos DB example, your indexing strategy becomes critical here. You might be tempted to use a non-clustered index and INCLUDE the Message payload to keep your lookups “covered.” However, forcing a heavy, serialized JSON string into a secondary index might not be the best idea.
Conversely, if you omit the Message column from a non-clustered index, retrieving a longer conversation history will trigger so many key lookup operations that the SQL engine may abandon the index entirely and switch to a full table scan which would be catastrophic for production performance.
Instead,I highly recommend letting the physical storage layout do the heavy lifting by configuring a composite clustered index with the following column order: (UserId, ConversationId, Timestamp ASC). Because SQL Server physically organizes the actual data rows on disk based on this key, it enables highly performant index range scans.
Azure Managed Redis
Storing chat history in Azure Managed Redis is a fantastic option if your conversation history is transient and doesn’t need to be persisted long term. By configuring a native Time-To-Live (TTL) on the Redis keys, old data cleans itself up automatically without putting deletion overhead on a persistent disk database.
Full Session Serialization in Blob Storage
I would also like to mention that there is yet another way to persist the state of the conversation with the LLM, and more…
We can save the entire session very easily by invoking agent.SerializeSessionAsync, saving that data to Blob Storage, and then later restoring it by using agent.DeserializeSessionAsync.
I created a helper class that does exactly this:
public class BlobSessionStore
{
// trimmed for brevity
public async Task<AgentSession> LoadSessionAsync(string tenantId, string userId, string conversationId, AIAgent agent)
{
var blobPath = BuildBlobPath(tenantId, userId, conversationId);
var blobClient = _containerClient.GetBlobClient(blobPath);
try
{
BlobDownloadResult downloadResult = await blobClient.DownloadContentAsync();
var jsonText = downloadResult.Content.ToString();
using var doc = JsonDocument.Parse(jsonText);
return await agent.DeserializeSessionAsync(doc.RootElement, _jsonOptions);
}
catch (RequestFailedException ex) when (ex.Status == 404)
{
return await agent.CreateSessionWithUserContextAsync(tenantId, userId, conversationId);
}
}
public async Task SaveSessionAsync(AgentSession session, AIAgent agent)
{
var (tenantId, userId, conversationId) = session.GetUserContext();
var blobPath = BuildBlobPath(tenantId!, userId!, conversationId);
var blobClient = _containerClient.GetBlobClient(blobPath);
var serializedSession = await agent.SerializeSessionAsync(session, _jsonOptions);
using var memoryStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(serializedSession.GetRawText()));
await blobClient.UploadAsync(memoryStream, overwrite: true);
}
private static string BuildBlobPath(string tenantId, string userId, string conversationId)
{
return $"{tenantId}/users/{userId}/{conversationId}.json";
}
}And this is how it is used in a simple example:
public async Task RunAsync()
{
var blobStore = new BlobSessionStore(_blobContainerClient);
while (true)
{
Console.Write("You: ");
var input = Console.ReadLine();
if (string.IsNullOrWhiteSpace(input)) break;
var agentSession = await blobStore.LoadSessionAsync(
tenantId: FakeUserData.TENANT_ID,
userId: FakeUserData.USER_ID,
conversationId: FakeUserData.CONVERSATION_ID,
_mafAgent);
var response = await _mafAgent.RunAsync(input, agentSession);
Console.WriteLine($"Agent: {response}");
await blobStore.SaveSessionAsync(agentSession, _mafAgent);
}
}Now, when I invoke this example and I provide the same text as before, such data is saved into a blob storage:
Please note that the session carries two main pieces of information:
ConversationId: Used for service-managed storage when populated.StateBag: The internal dictionary managed by theProviderSessionStateclass that we discussed earlier.
In this example, I did not specify a custom ChatHistoryProvider, so the framework automatically fell back to the default InMemoryChatHistoryProvider. Keep in mind that if you do not provide a custom stateKey, the framework simply defaults to using the type name of the active provider as the dictionary key.
This session serialization method offers a unique benefit that other approaches do not: it captures the entire session state atomically, including every item inside the StateBag dictionary. If you store extra operational data in a session—which is highly common when leveraging components like AIContextProviders—serializing the entire StateBag is exactly what you want.
When you decide to store your serialized sessions or raw chat messages in Azure Blob Storage, I highly recommend using the Premium tier (Block Blobs). Because this tier is backed by SSDs, it provides the ultra-low, consistent sub-millisecond latencies required to keep your AI conversation loops feeling snappy and responsive.
Additionally, leveraging Azure Blob Storage for your session states unlocks powerful native cloud features that are easily overlooked:
- Lifecycle Management: You can define native rules to automatically delete old sessions or transition them to cooler, cheaper storage tiers after a set period of inactivity, eliminating the need to write custom database cleanup scripts.
- Attribute-Based Access Control (ABAC): You can secure multi-tenant data with extreme precision. By tagging your session blobs with metadata tags (such as
UserId), you can configure Azure RBAC conditions to dynamically restrict data access based on the user’s security attributes. - Enterprise-Grade Data Protection: You gain immediate access to built-in resilience tools like soft delete, object replication, and versioning straight out of the box.
Summary
I hope this post about managing chat history in Microsoft Agent Framework helped you decide which storage option to use for your application.
I wanted to cover how ChatHistoryProvider works behind the scenes because once you understand that foundation, you can easily implement any custom provider or use and customize the existing providers like CosmosChatHistoryProvider or InMemoryChatHistoryProvider. I also hope the concept of the session StateBag is much clearer now, and that you know how to use it to store user context for identifying conversations in your external storage.
Thanks for reading, and see you in the next post!
