Choosing a Deployment Type

Two deployment options

A model in the catalog does nothing until it is deployed. The first choice is not the deployment type but the deployment option, because the types only exist inside one of them.

To make a model available for inference requests, you deploy it, and Foundry offers two deployment options. Serverless API serves Foundry Models, including Foundry Models sold by Azure and select models from partners and community, and it is the preferred and most capable deployment path. Managed compute (preview) serves open-source, partner, and custom models that run on dedicated GPU capacity that Foundry manages for you.

The deployment types in this lesson apply to the Serverless API option; open-source and custom models that use managed compute don't use them. If you only need to try a supported model, instant access (preview) calls models by name without creating a deployment, so it isn't a deployment type.

So there are three ways to reach a model, and only one of them has deployment types:

A deployment type decides three things: where your data is processed (global, data zone, or Azure geography), how you pay (pay-per-token or reserved capacity), and the performance characteristics (latency variance and throughput limits).

Categories and SKU codes

Inside Serverless API, every deployment type belongs to a family. Learn the families first and the nine names fall into place.

There are three main categories: standard (pay-per-token), provisioned (reserved capacity), and batch (discounted asynchronous processing). A Developer type is also available for fine-tuned model evaluation. Within the standard and provisioned categories, you choose global, data zone, or geography-based processing.

In words: each standard type is pay-per-token, each provisioned type is reserved PTU capacity, and the two batch types are the discounted asynchronous option. The SKU code is what you pass in code, and an Azure Policy rule on the deployment sku.name can disable a specific deployment type across your organization.

Two naming details catch people out. Every provisioned SKU ends in Managed, and the regional provisioned SKU is plain ProvisionedManaged, without the word Regional. The single-geography standard type is just Standard.

Where the data is processed

The processing scope is usually the first hard constraint, because it comes from compliance rather than preference.

Data stored at rest remains in the designated Azure geography for all deployment types. Inferencing data differs: Global types may be processed in any Azure region, Data Zone types are processed only within the Microsoft-specified data zone (US, EU, or Asia Pacific), and Standard and Regional Provisioned types process prompts and responses within the customer-specified Azure geography.

When nothing forces your hand, the answer is simple. For most workloads, start with Global Standard. It launches first when a new model releases, has the lowest price, and offers the broadest region coverage. Move to another deployment type only when you have a specific reason, such as data residency, reserved throughput, or asynchronous batch processing.

New deployment types become available in a set order: Global, then Data Zone, then geography-based. That order matters for planning: a brand-new model reaches geography-based types last.

Standard or provisioned

Most decisions come down to one question: pay per token and share capacity, or reserve capacity and pay by the hour?

Standard types are pay-per-token and offer best-effort service, which suits variable, bursty traffic. Customers with high consistent volume might experience greater latency variability on them.

Provisioned throughput holds a fixed amount of processing capacity exclusively for your deployment, whether or not requests are being made. You buy provisioned throughput units (PTUs), which guarantee a level of capacity and give lower and more consistent latency. Provisioned types fit predictable traffic, latency-sensitive requirements, and production-scale volume.

The three provisioned types differ in where traffic is processed. Global Provisioned routes across Azure regions globally and gives the highest availability. Data Zone Provisioned offers zone-level data residency with higher availability than regional. Regional Provisioned stays in the deployment's specific Azure region, for strict single-region data residency requirements.

Quota, capacity, and reservations

Provisioned deployments involve three things that sound alike and are not: quota, capacity and reservations.

PTUs are model-independent, but PTU quota is granted per subscription, per region, and per deployment type. The tokens per minute that a given number of PTUs delivers depends on the model, and each model has a minimum PTU count required to create a deployment.

Quota and capacity are different things. Having PTU quota doesn't guarantee that capacity is available: if capacity in the region is insufficient for the requested PTU count, the deployment fails.

Billing has its own rules. All provisioned types are billed hourly per PTU, regardless of the number of tokens consumed. Azure Reservations give a discounted rate for a one-month or one-year commitment, but reservations don't guarantee capacity, so you first create deployments to confirm capacity and then purchase the reservation.

Batch for large asynchronous jobs

Some work has no user waiting: product descriptions for a whole catalog, or summaries of a document archive. Batch is built for that.

Global Batch processes asynchronous groups of requests with separate quota and a 24-hour target turnaround, at 50% less cost than Global Standard. Instead of sending one request at a time, you send a large number of requests in a single JSON Lines file. Batch requests have a separate enqueued token quota, which avoids any disruption of your online workloads. Data Zone Batch provides the same functionality, with traffic kept inside the data zone. Batch requests don't have a real-time SLA.

The input file has strict rules. In the batch file, the same batch model deployment name must be present on each line; to target a different deployment, you use a separate batch file and job. Each line also carries a custom_id, because responses don't come back in the order of the file. The batch API doesn't currently support fine-tuned models.

In Python, the flow uses the openai package: client.files.create uploads the file with purpose set to batch, and client.batches.create starts the job with the uploaded file ID and a completion_window of 24h.

Developer for fine-tuned evaluation

One type has a single narrow job. The Developer type is designed for fine-tuned model evaluation only. It doesn't include data residency guarantees or an SLA, and Developer deployments have a fixed 24-hour lifetime and are automatically deleted after expiration.

Do not confuse its 24 hours with batch. For Developer, 24 hours is how long the deployment lives. For batch, 24 hours is the target turnaround of a job on a deployment that keeps running.

Managed compute in practice

Managed compute is the path for models that Serverless API does not serve, and it plays by different rules.

Managed compute is in public preview, without a service-level agreement, and isn't recommended for production workloads. It hosts models such as Qwen and NVIDIA Nemotron from the Hugging Face Collection. Billing is hourly per accelerator SKU, and quota is granted per accelerator SKU per region through the Foundry quota process, separate from Azure VM quota. Azure virtual machines are an IaaS offering with regional SKUs, while managed compute is a PaaS offering, so existing Azure VM quota can't be applied to a managed compute deployment.

Managed compute is currently available for global deployment only, and content filtering isn't available for it in public preview, while Serverless API offers built-in and customizable content filtering.