Agent Memory, Tools, and Knowledge

Three things an agent can draw on

An agent that only has a model and instructions forgets the user, knows nothing about your organization, and can't act. Foundry gives it three kinds of help, and choosing well starts with knowing which one solves which gap.

• Memory keeps what the agent learns about a user or a task across sessions.
• Knowledge grounds the agent in content your organization curates.
• Tools let the agent act: call APIs, search, run code, or reach other agents.

The memory guidance gives a three-way rule. Use memory to capture user-specific or agent context that is learned and persists through interactions over time. Use a Foundry IQ knowledge base to ground the agent on curated organizational content. Use the file search tool to search user-provided documents during an interaction.

Short-term and long-term memory

Agent memory falls into two categories.

• Short-term memory tracks the current session's conversation and keeps immediate context. Agent orchestration frameworks typically manage it as part of the session context.
• Long-term memory retains distilled knowledge across sessions, so the agent can recall and build on earlier interactions. It needs a persistent system that extracts, consolidates, and manages knowledge.

Memory in Foundry Agent Service is designed for long-term memory. It's a managed solution that gives an agent continuity across sessions, devices, and workflows, so it can retain user preferences, maintain conversation history, and personalize its responses.

Memory and the Memory Store API are both in preview, and pricing and billing can change during the preview. You're billed for the underlying chat and embedding models you configure.

How memory works

Behind the scenes, memories are stored as items in a managed memory store, and memory works in three phases.

1. Extraction: as a user interacts with the agent, the system extracts key information such as preferences, facts, and relevant context, for example "allergic to dairy".
2. Consolidation: the system uses LLMs to merge similar or duplicate topics, so the store doesn't hold redundant information, and it resolves conflicting facts, such as a new allergy.
3. Retrieval: when the agent needs to recall something, it searches the memory store for the most relevant memories.

For best results, retrieve stable user profile information early in the conversation, so the agent can personalize its responses from the start.

The three memory types

Foundry memory extracts and stores three types of long-term memory, and each has its own moment to be retrieved.

In words: user profile memory is retrieved near the start of each conversation to set stable personalization. Chat summary memory is retrieved per turn for continuity. Procedural memory is retrieved when the user asks for a task the agent has handled before. All three are enabled by default, and you can configure or disable them when you create a memory store.

Using and governing memory

There are two ways to use memory:

• The memory search tool: attach it to a prompt agent to read from and write to the memory store during conversations. This is ideal for most scenarios because it simplifies memory management.
• The memory store APIs: work directly with the store through low-level APIs, for more control, including direct control of individual memory records and retention.

Production controls include item-level create, read, update, list, and delete, a store-level default TTL in seconds for new memory entries, and direct remember-or-forget commands when a user explicitly asks for a memory change.

Limits shape the design too:

• Memory requires compatible Azure OpenAI chat and embedding model deployments.
• With the low-level APIs, you must set scope explicitly in each request. Automatic scope from the caller's identity works only through the memory search tool with scope set to the user ID placeholder.
• Virtual network (VNet) integration isn't supported for memory stores.
• A store holds up to 100 scopes, with up to 10,000 memories per scope.

Memory has its own risk: prompt injection and memory corruption, when harmful or wrong data gets stored and then shapes later answers. Validate prompts with Azure AI Content Safety and its prompt injection detection, and run adversarial tests regularly.

Toolbox: one endpoint for agent tools

Tools let an agent act, and they multiply fast. An onboarding agent might use a knowledge base, REST API, skills, and an MCP server, each with its own authentication and owner. Across many agents, teams re-implement the same tools, duplicate credentials, and govern inconsistently.

Toolbox fixes that. You define a curated set of tools once and expose them through a single MCP-compatible endpoint that agents consume across frameworks and runtimes. Toolboxes are created in Foundry but aren't limited to Foundry agents: any MCP-compatible runtime or client can use one, including Microsoft Agent Framework, LangGraph, or your own code.

Its key capabilities:

• Single endpoint: the agent discovers every tool at runtime, and you add, remove, or reconfigure tools without changing agent code.
• Centralized authentication: the toolbox handles credential injection, token refresh, and policy enforcement using Microsoft Entra ID and OAuth identity passthrough.
• Governance by default: apply guardrails to tool inputs and outputs at the toolbox level.
• Versioning: create and test a new toolbox version, then promote it to default. Every agent that points to the toolbox picks it up automatically, with no code changes or redeployment.

Tool search and skills

A toolbox can grow to hundreds of tools, and sending every definition with every request hurts in three ways: cost rises, because each definition adds input tokens; context shrinks, because definitions compete with conversation history; and tool selection gets less accurate.

Tool search (preview) hides tools by default and exposes only two meta-tools: tool_search, which returns the most relevant tools for a described need, and call_tool, which invokes a discovered tool by name. You can pin critical tools so they're always available, add context with your organization's terms, and auto-pin frequently used tools.

Skills (preview) separate what from how: tools define what an agent can do, and skills define how it performs a task. Skills package reusable, multistep workflows, are versioned and immutable, and agents discover and load them through MCP resources at startup.

What can go in a toolbox

Not every tool can live in a toolbox, and the split is examinable.

In words: most tools work both ways. Tool search, skills, and the reminder tool exist only in a toolbox. Function calling can't go in a toolbox, because it executes client-side, and the same direct-only rule applies to Grounding with Bing, computer use, image generation, SharePoint, the Fabric data agent, and Azure Functions.

Foundry IQ: knowledge for agents

The model behind an agent has a knowledge cutoff and can't reach your proprietary data on its own. Foundry IQ lets you create a configurable, multi-source knowledge base that gives agents permission-aware responses from your organization's data.

A knowledge base consists of knowledge sources, which are connections to internal and external data stores, and parameters that control retrieval behavior, including the retrieval reasoning effort: minimal, low, or medium. Multiple agents can share the same knowledge base. When an agent queries it, Foundry IQ uses agentic retrieval to process the query, retrieve information, enforce user permissions, and return grounded answers with citations. Azure AI Search provides the underlying indexing and retrieval.

What it can do:

• Connect one knowledge base to many agents, over sources such as Azure Blob Storage, SharePoint, OneLake, and public web data.
• Automate chunking, vector embedding generation, and metadata extraction, with scheduled indexer runs for incremental refresh.
• Issue keyword, vector, or hybrid queries across indexed and remote sources.
• Synchronize ACLs for supported sources, honor Microsoft Purview sensitivity labels, and run queries under the caller's Microsoft Entra identity, so agents return only authorized content.

You can use a knowledge base from Foundry Agent Service, Microsoft Agent Framework, or any app through the REST API or Azure SDKs. Some features are generally available and others are still in preview, depending on the Search Service REST API version. Foundry IQ sits next to Fabric IQ, for Microsoft Fabric business data, and Work IQ, for Microsoft 365 collaboration signals.

Key takeaways

• Use memory for context learned about a user over time, Foundry IQ for curated organizational content, and file search for documents shared during a chat.
• Foundry memory is long-term, in preview, and works through extraction, consolidation, and retrieval.
• User profile memory is retrieved near the start of a conversation, chat summary per turn, and procedural memory for recurring tasks.
• Memory stores don't support VNet integration, and low-level memory APIs need an explicit scope.
• A toolbox exposes curated tools through one MCP-compatible endpoint, and promoting a new default version updates every agent without code changes.
• Function calling stays a direct tool, and tool search exposes only tool_search and call_tool.
• A Foundry IQ knowledge base unifies knowledge sources, can be shared by many agents, and returns permission-aware answers with citations.