Information Extraction with Content Understanding

What Content Understanding is

Azure Content Understanding in Foundry Tools is generally available. Its one-line pitch is simple: one service turns documents, images, audio, and video into structured data you define.

That last phrase matters. You define the data shape. Content Understanding is not just OCR, and it is not just a raw generative prompt. It processes content through an analyzer and returns structured JSON that matches the schema you specified. The response can also include confidence scores, grounding, Markdown for retrieval-augmented generation, and content-safety flags.

Use it when the content is unstructured, mixed-format, or multimodal. A contract, product image, call recording, and training video can all be handled through the same service family. The app does not need separate code paths for every content type if the goal is to turn content into downstream fields.

Analyzers are the core unit

An analyzer is a reusable, configurable processing unit. It sets the content type through baseAnalyzerId, what to pull out through a field schema, the output shape, and which models to use. You create it once and call it repeatedly from your app.

A field schema is a JSON definition of the named fields you want. Fields have types, descriptions, and methods. The description is important because custom analyzers can work zero-shot: you describe the field in plain language, and the analyzer uses that description without requiring labeled training data.

{
  "fields": {
    "vendorName": {
      "type": "string",
      "method": "extract",
      "description": "The supplier or company name printed on the invoice."
    },
    "documentType": {
      "type": "string",
      "method": "classify",
      "enum": ["invoice", "receipt", "contract"]
    },
    "executiveSummary": {
      "type": "string",
      "method": "generate",
      "description": "A concise summary of the content for a business reviewer."
    }
  }
}

The three field methods

Prebuilt and custom analyzers

Prebuilt analyzers are ready-made. Know the families rather than trying to memorize every variant. Content extraction analyzers include prebuilt-read and prebuilt-layout. The four base analyzers are prebuilt-document, prebuilt-image, prebuilt-audio, and prebuilt-video. The four RAG analyzers are prebuilt-documentSearch, prebuilt-imageSearch, prebuilt-audioSearch, and prebuilt-videoSearch. Domain-specific analyzers include prebuilt-invoice, prebuilt-receipt, prebuilt-idDocument, and prebuilt-contract.

Custom analyzers are the exam surprise. They work zero-shot. You can describe the fields in plain language and use the analyzer without labeled examples. Labeled examples are optional and can improve accuracy, but they are not required to start.

Confidence tells you how certain the service is about a field. Grounding tells you where the value came from. In documents and images that can mean bounding boxes. In audio and video it can mean timestamps. This matters because a real extraction app often needs a reviewer experience, not just a final JSON object.

The four content types

Content Understanding is multimodal. The exam may describe a document, image, recording, or video and ask which analyzer family fits. Choose the analyzer that matches the content type and whether the goal is field extraction or RAG indexing.

Choosing Content Understanding or Document Intelligence

This is the highest-value exam distinction in objective 2.4. Azure Content Understanding in Foundry Tools and Azure Document Intelligence in Foundry Tools are both active. They are complementary, not replacements for every scenario.

If a question says document, do not jump too quickly. Ask what kind of document and what constraints matter. A standard receipt at scale points toward Document Intelligence. A long contract with custom business fields and no labeled examples points toward Content Understanding. If the input is not a document, Document Intelligence is not the answer.

Build a lightweight extraction app

AI-901 is a fundamentals exam, but it expects builder thinking. You should recognize the flow of an extraction app even if you are not asked to write production code.

1. Create the resource. Work in Microsoft Foundry and use the appropriate Foundry resource and project context.
2. Pick or create an analyzer. Choose a prebuilt analyzer when it matches, or create a custom analyzer for your app fields.
3. Define the field schema. Name the fields, choose types, write descriptions, and choose extract, classify, or generate.
4. Call the analyze operation. Send the document, image, audio, or video to the analyzer from your app.
5. Read the structured JSON. Inspect field values, confidence scores, grounding, and safety flags.
6. Route it downstream. Store fields, trigger approvals, index Markdown for RAG, show low-confidence values for review, or call another workflow.

You can author analyzers in the portal without code. Code still matters for calling the analyzer, handling results, and integrating the output into the rest of the app.

{
  "analyzerId": "contract-risk-analyzer",
  "status": "succeeded",
  "result": {
    "fields": {
      "counterparty": {
        "value": "Contoso Ltd.",
        "confidence": 0.94,
        "grounding": [{ "page": 1, "box": [1.2, 2.4, 4.8, 2.8] }]
      },
      "renewalRisk": {
        "value": "auto-renewal requires 90 days notice",
        "confidence": 0.82,
        "grounding": [{ "page": 6, "box": [0.9, 5.1, 7.2, 5.8] }]
      }
    },
    "contentSafety": {
      "flagged": false
    }
  }
}

In a real app, you should not treat every field the same. High-confidence grounded fields can flow through automatically. Low-confidence or high-risk fields can be routed to a human reviewer. Generated summaries should be shown with source grounding when the decision matters.

Traps candidates make

• Confusing OCR with extraction. OCR returns text. Content Understanding returns fields you defined.
• Confusing a prompt with an analyzer. A raw prompt can produce prose. An analyzer is reusable and configured with a field schema.
• Assuming custom means trained. Content Understanding custom analyzers can work zero-shot. Labeled examples are optional.
• Ignoring modality. Document Intelligence is documents-only. Content Understanding also handles images, audio, and video.
• Missing grounding. Bounding boxes and timestamps help prove where a field came from.
• Choosing Content Understanding for air-gapped deployment. Content Understanding has no container option; Document Intelligence containers are the answer.
• Using Video Indexer for custom fields. Video Indexer gives prebuilt insights. Content Understanding gives schema-driven extraction.

The night-before cheat sheet