Foundry Agent Service at a Glance

What the service does for you

A model on its own answers a prompt. An agent goes further: it follows instructions, picks tools, and acts. Running that reliably means hosting, scaling, identity, and monitoring, and that is the work Foundry Agent Service takes off your hands.

Foundry Agent Service is a managed platform for building, deploying, and scaling AI agents. It works with any framework, any supported model from the Foundry model catalog, and a single entry point for model inference and tools.

Its Agent Runtime hosts and scales prompt agents and hosted agents, and manages conversations, tool calls, and the agent lifecycle. Around the runtime sit the other components:

The table in words: Toolboxes curate tools once and share them through one managed MCP endpoint. Models come from the catalog and can be swapped without code changes. Observability gives end-to-end tracing, metrics, evaluations, and Application Insights integration. The Agent optimizer, in preview, evaluates agent behavior and improves instructions, skills, tool descriptions, and model selections. Identity and security bring Microsoft Entra identity, RBAC, content filters, and virtual network isolation. Publishing versions agents, creates stable endpoints, and shares them through Microsoft Teams, Microsoft Copilot, and the Entra Agent Registry.

When a question names a job, map it to its component: hosting and scaling is the runtime, seeing every decision is observability, and sharing tools across agents is a toolbox.

Models, instructions, and tools

Every agent combines the same three ingredients. A prompt agent is defined entirely through configuration, including instructions, model selection, and tools. The instructions say what the agent should do, the model does the reasoning, and the tools let it act.

The model comes from the Foundry model catalog, such as GPT-4o, Llama, or DeepSeek, and you can swap models without changing your agent code. That makes the model a replaceable part: try a different one for cost or quality and keep the rest of the agent as it is.

Agents act on the world through tools. Foundry offers built-in tools such as web search, file search, code interpreter, and memory, and you can add custom tools through functions, OpenAPI specs, and MCP servers.

Sorting tools is a frequent exam move. Built-in tools cover general jobs that any agent might need:

So an agent that must call an internal inventory API described by an OpenAPI spec needs a custom tool built from that spec. Web search, file search, and the code interpreter do useful work, but none of them calls your private API.

Toolboxes and tool authentication

As agents multiply, wiring the same tools into each one becomes a maintenance job. A toolbox groups tools into a single, reusable unit behind one managed MCP-compatible endpoint that any agent or runtime can consume. Toolboxes centralize authentication, governance, and versioning, so you update tools in one place instead of rewiring every agent. You create a new version, test it, and promote it to default when you are ready.

Tools often call protected systems, so each connection needs a way to authenticate. Tool connections can authenticate with key-based access, Microsoft Entra using the agent's or the project's managed identity, OAuth identity passthrough (On-Behalf-Of), or unauthenticated access where appropriate.

Choose the option by asking whose identity the remote system should see.

When a remote MCP server must apply the signed-in user's own permissions, OAuth identity passthrough is the fit, because it carries the user's identity through to the server. A managed identity is right when the agent itself is trusted, and a key only proves possession of the key.

Three agent types

Agent Service offers three agent types, and the choice sets how much you build versus how much Foundry manages.

• Prompt agents are the fastest path: you define instructions, a model, and tools, and Foundry runs the agent with no code or infrastructure to manage.
• Voice-based prompt agents are managed real-time voice agents: you configure the model, instructions, audio, and tools, then connect through Voice Live over a real-time WebSocket.
• Hosted agents give the most control: you bring your own code and framework, and Foundry runs it as a container with a managed endpoint, scaling, and identity.

Prompt agents can be authored portal-first, in the Foundry portal and its playground, or code-first, through the SDK or REST API in a deployment pipeline. Either way, Foundry runs the agent, and there is no application code or container for you to maintain.

The quickest way to choose is a short sequence of questions.

In words: neither prompt agent type leaves runtime code or compute for you to manage, while a hosted agent runs your agent logic on Foundry-managed container compute. Prompt agents are best for a fast start and production agents without custom orchestration logic.

Hosted agents up close

Hosted agents can be built with Agent Framework, LangGraph, the OpenAI Agents SDK, the Anthropic Agent SDK, the GitHub Copilot SDK, or your own code. You ship a hosted agent as a container image or as a .zip file of your source code, and Foundry builds the image for you when you bring a .zip file.

Foundry then runs it with a managed endpoint, automatic scaling, a dedicated Microsoft Entra identity, session-level state persistence, and end-to-end observability. Under the hood, your code calls your Foundry project endpoint for model inference and tool orchestration, so catalog models and platform tools stay available.

Hosted agents are best for agents that call into your own custom code, and secondarily for custom orchestration logic, multi-agent systems, and custom protocols.

The trade-off shows up in the bill. For cost, prompt agents are billed per call for inference plus tool usage, while hosted agents add container compute on top.

Picture a team whose agent calls its own Python library and orchestrates steps with LangGraph, and that wants Foundry to handle the endpoint, scaling, and identity. A prompt agent with function tools is tempting, but prompt agents suit agents without custom orchestration logic. Running the code on its own servers against the Responses API would work too, but then the team hosts and scales it. A hosted agent is the fit.

Voice-based prompt agents

Voice-based prompt agents run real-time spoken conversations. Foundry Agent Service manages the agent lifecycle and uses Voice Live for speech recognition, turn taking, model interaction, and speech synthesis. Voice-based agents are in preview, so check support and availability before relying on them in production.

Two architectures are available:

• A native speech-to-speech architecture uses a realtime model that handles spoken input and output directly, and it is optimized for natural, low-latency conversation.
• A cascaded architecture chains speech recognition, a text model, and speech synthesis.

The cascaded one is the controllable one. It provides broader text-model and Azure voice choices and explicit control over transcription, phrase lists, turn detection, and interim responses, but each stage contributes to end-to-end latency. Native speech-to-speech is the low-latency choice, and its supported settings and voice families depend on the selected model.

Agents without an agent resource

Sometimes the agent code already lives in your own application. If you already run agent code elsewhere, you can call the Responses API directly to use Foundry models and tools, with no agent resource to manage.

This builds an ephemeral agent: its instructions, tools, and model live in your application code instead of as a persisted resource in Foundry. Each call assembles the agent in your process and runs it, so there is no agent to create, update, or delete in Foundry. The definition then versions alongside the rest of your code through source control and code review.

You still keep Foundry capabilities: catalog models, platform tools, project-scoped data, On-Behalf-Of authentication, and project-level observability and governance, all through your Foundry project endpoint.

The trap is the code-first prompt agent. It also lives in source control, but it creates a persisted agent resource that someone has to keep in sync with the app. Calling the Responses API directly keeps the agent definition in your app code without creating a Foundry agent resource.

From first build to a shared agent

Agent Service supports the full cycle: create, test, trace, evaluate, optimize, publish, and monitor. You test in the agents playground, inspect every model call and tool invocation with tracing, run evaluations to catch regressions, and track performance with service metrics and dashboards.

As you iterate, versions are captured automatically, so you can roll back to any previous version or compare versions. Publishing promotes an agent to a managed resource with a stable endpoint, and published agents can be shared through Microsoft Copilot and Teams and the Entra Agent Registry. For agent-to-agent communication the service supports the A2A protocol: A2A v1.0 is generally available, and v0.3 remains in preview.

On security, each agent can have a dedicated Microsoft Entra identity, and integrated content filters help mitigate prompt injection risks, including cross-prompt injection.