Foundry Tools at a Glance

What Foundry Tools are

Not every AI feature needs a large model and a prompt. Reading a receipt, transcribing a call, or translating a support ticket are jobs that ready-made services already do well, and Foundry groups those services as Foundry Tools.

They offer out-of-the-box, prebuilt, and customizable APIs and models for applications such as natural language processing for conversations, search, monitoring, translation, speech, vision, and decision-making.

Most Foundry Tools are available through REST APIs and client library SDKs in popular development languages, so a Python developer calls them like any other web API.

Seven tools to recognize

Each tool covers one kind of input or task. Learn them by the question each one answers.

The same seven, as you would say them:

• Speech provides speech to text, text to speech, translation, and speaker recognition. Its input is audio.
• Translator translates more than 100 in-use, at-risk, and endangered languages and dialects.
• Language builds apps with natural language understanding capabilities.
• Vision analyzes content in images and videos.
• Document Intelligence turns documents into intelligent data-driven solutions.
• Content Understanding analyzes and comprehends various media content types.
• Content Safety detects unwanted content.

Two pairs get mixed up. Document Intelligence focuses on documents, while Content Understanding is the tool for a mix of media content types. Speech can translate too, but it starts from audio: you can send an audio file to Speech and get a transcription and a translation at the same time. Translator is the one built around its long list of languages.

Retiring tools and the rest of the family

The current family is wider than the seven above. The current family also includes Azure AI Search for AI-powered cloud search, Custom Vision for customized image recognition, and Immersive Reader to help users read and comprehend text.

Some older tools are scheduled for retirement: Anomaly Detector, Content Moderator, Language Understanding, Metrics Advisor, Personalizer, and QnA Maker. They remain available for existing applications, but you should not use them for new AI applications.

The pair to remember is Content Moderator and Content Safety. Both detect unwanted content, so either looks fine at first. The difference is status: Content Moderator is retiring, so a new application uses Content Safety.

Out of the box or custom

The key decision with a Foundry Tool is how much you change it. Out of the box, most Foundry Tools let you send data and receive insights without any customization. For example, you can send an image to Vision to detect words and phrases or to count the number of people in the frame. You can also send an audio file to Speech and get a transcription and a translation at the same time.

When the baseline model is not enough, some services let you bring your own data and train a custom model. When you bring your own data, you might need to tag it in a way specific to the service, such as marking where a flower appears in each image of a flower catalog. Custom Vision exists for exactly that kind of customized image recognition.

So the order of questions is: does the baseline do the job? If yes, call the prebuilt tool. If not, check whether that service supports custom training with your tagged data. Separately, if processing must stay near your data, check whether the chosen tool supports an on-premises container.

Use a prebuilt tool when its baseline insight is enough; when it is not, consider custom training with tagged data if that service supports it. A team that only needs to count people in lobby images, with no data science skills and a deadline, sends the images to Vision. A retailer whose products the baseline does not recognize tags product photos and trains a custom model.

Ways of working with the tools

The tools that you use to customize and configure models are different from the tools that you use to call the Foundry Tools. When it comes to calling them, there is a trade-off between ease and control.

Designer-driven tools are the easiest to use and quick to set up, but they might have limitations when it comes to customization. REST APIs and client libraries give the greatest flexibility to call the services from any language and environment, but they require more effort, time, and expertise. That route expects you to be comfortable with a language such as Python.

Two more options change how the tools are delivered rather than how they are called.

Many Foundry Tools can be deployed in containers for on-premises use, which brings them closer to your data for compliance, security, or other operational reasons. For example, a hospital using a tool that supports containers can process sensitive records on its premises instead of sending them to a cloud API hosted in Microsoft-managed data centers.

For continuous integration and deployment, Azure DevOps and GitHub Actions can manage deployments and automate training and release of custom models, for example for Custom Speech. That is pipeline automation for custom models, not a different way to call a prebuilt tool.

Billing and security

Foundry Tools are billed by use. Pricing tiers, and the amount you are billed, are based on the number of transactions you send using your authentication information. Each pricing tier specifies the maximum number of allowed transactions per second, the service features enabled, and the cost for a predefined number of transactions. Going above that predefined number adds an extra charge.

Many tools have a free tier, which you use by choosing F0 as the SKU for your resource. It is the way to try a service before paying.

Security is layered, with authentication through Microsoft Entra credentials or a valid resource key, plus Azure Virtual Networks. Language and regional availability are published per API, so check both before you pick a region for a tool.