Computer Vision on Azure
This covers skill area 3, “Describe features of computer vision workloads on Azure” (15–20% of the exam).
The four solution types
Computer vision lets software interpret visual input. For AI-900, do not memorize every API operation. Learn the shape of the problem: what goes in, what comes out, and which Azure service fits the business sentence.
The exam loves the difference between classifying an image and detecting objects in it. Classification labels the whole image. Object detection returns a class plus a bounding box for each instance. Semantic segmentation goes further and labels every pixel. Optical character recognition reads printed or handwritten text.
Image classification
Input: one image. Output: one or more labels for the entire image, usually with confidence scores.
A retailer can classify uploaded product photos as shoes, jackets, bags, or accessories. The model is not saying where the shoe is. It is saying what the overall image is about.
Object detection
Input: one image. Output: each detected object, its class, confidence, and rectangular bounding box.
A warehouse camera can find every pallet in an aisle and return coordinates for each one. If there are six pallets, expect six detections, not one image-level label.
Semantic segmentation
Input: one image. Output: a pixel-level mask that assigns a category to each pixel or region.
A city planning team can label road, sidewalk, building, vehicle, and tree pixels in street imagery. This is more detailed than a bounding box.
Optical character recognition
Input: an image, scanned page, sign, receipt, or document. Output: extracted text, lines, words, and location data.
A mobile app can read a serial number from a photo of equipment. A back-office process can extract typed and handwritten text from scanned forms.
| Solution type | Output | Example question wording | Azure service |
|---|---|---|---|
| Image classification | A label for the whole image, such as product category or scene type | “Determine whether an image is a receipt, invoice, or contract.” | Azure AI Vision for prebuilt tags, or Azure AI Custom Vision for your own categories |
| Object detection | A class and bounding box for each object instance | “Find every defective part on a conveyor belt and show where each one is.” | Azure AI Custom Vision object detection, or Azure AI Vision for supported prebuilt detection scenarios |
| Semantic segmentation | A pixel-by-pixel label map | “Label each pixel in satellite imagery as water, road, building, or vegetation.” | Azure Machine Learning with a custom computer vision model |
| Optical character recognition | Text, lines, words, and text locations | “Read the words on a road sign from a photo.” | Azure AI Vision Read; use Azure AI Document Intelligence for forms and structured documents |
If the question says “is this an X?” or “which category best describes this image?”, think classification. If it says “locate each X”, “count the objects”, or “draw a rectangle around each one”, think object detection.
Faces: detection, analysis, recognition
Face workloads are still computer vision, but AI-900 expects you to separate ordinary face detection from identity-related recognition. A camera can detect that a face exists and return a face rectangle without identifying the person. Recognition requires enrollment, matching, and stricter responsible AI controls.
Face detection answers “is there a face, and where is it?” It returns face locations and can support simple scenarios such as cropping a profile photo. Azure AI Vision can also do basic face detection, while Azure AI Face is the service focused on face analysis and recognition scenarios.
Face analysis describes visible properties of a detected face. Current Face capabilities can include attributes such as head pose, blur, occlusion, exposure, noise, glasses, facial hair, mask, and landmarks depending on the API and access level. Microsoft retired broad inferred attributes such as emotion and gender because they created high risk and unreliable social inference. On the exam, do not pick emotion analysis as a modern Face feature.
Face verification compares two faces and answers whether they are likely the same person. Face identification, sometimes called recognition, compares a detected face against an enrolled person group and returns who it most likely is. Identification is a one-to-many search. Verification is a one-to-one comparison.
Azure AI Face has a Limited Access policy for sensitive capabilities such as face recognition. Microsoft uses this policy because biometric identification can affect privacy, consent, fairness, and safety. Customers must apply for access and describe an appropriate use case. That does not mean face detection is banned; it means identity recognition is controlled.
| Operation | Question it answers | What must exist first? | Typical output |
|---|---|---|---|
| Detection | Is there a face, and where? | An image | Face rectangle, landmarks, and supported analysis attributes |
| Verification | Are these two faces the same person? | Two face images or face IDs | Same or not same, with a confidence score |
| Identification | Who is this person? | An enrolled person group with stored face data | Candidate person identity and confidence |
Detection finds faces. Verification compares two faces. Identification finds a person from an enrolled group. If the scenario says “who is this employee?”, it is identification and it points to Azure AI Face, subject to Limited Access.
Reading text from images and documents
OCR appears in two different Azure services, and the exam checks whether you can choose between them. Azure AI Vision has a Read capability for extracting text from images and scanned pages. Azure AI Document Intelligence extracts structured information from documents such as forms, invoices, receipts, and ID documents.
Use Azure AI Vision Read when the goal is the text itself. Good clues are photos, signs, labels, license plates, screenshots, scanned pages, and handwritten notes. The output is text lines and words, often with bounding polygons. The service is not trying to understand an invoice total or map a receipt merchant into a field. It is reading.
Use Azure AI Document Intelligence when the goal is document understanding. The clue is usually a business form, a semi-structured document, or a requirement to extract key-value pairs, selection marks, tables, fields, and normalized values. Document Intelligence can use prebuilt models for common document types, including invoice, receipt, ID document, business card, W-2, health insurance card, and layout. The layout model extracts text, tables, selection marks, and document structure. Custom models let you train extraction for your own forms and templates.
The core decision is simple: plain text out of a photo or sign is Azure AI Vision Read. Key-value pairs and tables out of a form is Azure AI Document Intelligence.
| Scenario | Best choice | Why |
|---|---|---|
| Read a street sign from a phone photo | Azure AI Vision Read | The output is unstructured text from an image |
| Extract invoice number, vendor, due date, line items, and total | Azure AI Document Intelligence prebuilt invoice model | The output is structured fields and tables |
| Extract text and checkboxes from a blank form layout | Azure AI Document Intelligence layout model | The output includes tables, selection marks, and document structure |
| Read handwriting from a scanned page | Azure AI Vision Read | The task is OCR, not form understanding |
| Train extraction for a company-specific claim form | Azure AI Document Intelligence custom model | The schema is specific to your organization |
Document Intelligence includes OCR, but it is chosen for structure. If the question only asks to read words from an image, do not overbuild the answer. Choose Azure AI Vision Read.
The Azure services, and how to choose
Most AI-900 computer vision questions are service selection questions. Start with whether the model is prebuilt, custom-trained, face-specific, document-specific, or video-specific.
| Service | What it does | Use it when | Do not use it for |
|---|---|---|---|
| Azure AI Vision | General image analysis, OCR Read, image tagging, captions, object-related analysis, smart crops, and basic face detection | You need prebuilt computer vision without training your own model | Extracting structured invoice fields, training custom categories, or identifying known people |
| Azure AI Custom Vision | Trains custom image classification and object detection models from your labeled images | Your business has its own visual categories or objects that a prebuilt model will not know | General OCR, document key-value extraction, face recognition, or video transcript indexing |
| Azure AI Face | Detects and analyzes faces, verifies faces, and identifies people from enrolled groups when approved | The scenario is face-specific, especially verification or identification | General image tagging, product defect detection, document extraction, or emotion and gender inference |
| Azure AI Document Intelligence | Extracts text, tables, selection marks, and fields from documents using prebuilt or custom models | You need invoice, receipt, ID, business card, W-2, health insurance card, layout, or custom form extraction | Classifying ordinary photos, detecting objects in images, or recognizing faces |
| Azure AI Video Indexer | Analyzes video and audio to create searchable insights such as transcripts, speakers, faces, labels, scenes, topics, and keywords | You need to search, summarize, caption, or index video content | Single-image classification, form extraction, or training an object detector from still images |
Azure AI Vision versus Azure AI Custom Vision
Azure AI Vision is the prebuilt general-purpose option. You send an image and ask for features such as tags, captions, image analysis, or Read OCR. You do not train it for your company first.
Azure AI Custom Vision is the train-your-own option. You upload labeled images, train a model, evaluate performance, and publish an iteration. Use it when the categories or objects are specific to your domain: types of machine damage, acceptable versus defective packaging, species in local wildlife images, or product variants that a generic model will not distinguish.
If the question says “no training data”, “prebuilt model”, or “analyze general images”, lean toward Azure AI Vision. If it says “train a model with labeled images”, “custom categories”, or “detect our products”, lean toward Azure AI Custom Vision.
Custom Vision project types and resources
Custom Vision has two project types. A classification project predicts image-level tags. An object detection project predicts object tags plus bounding boxes. This is the same distinction as the workload types above, but now it is in the custom training service.
Custom Vision also separates training and prediction resources. The training resource is used to upload images, label them, train iterations, and manage the project. The prediction resource hosts the published model so applications can send images and receive predictions. You can create the resources separately, which is useful when different teams, regions, pricing, or access controls are needed.
More broadly, Azure AI services can be provisioned as a multi-service resource or as a single-service resource. A multi-service resource gives one key and endpoint for multiple supported Azure AI services. A single-service resource gives one key and endpoint for one service. On AI-900, this is usually a provisioning detail, not a modeling detail.
Prebuilt image analysis is Azure AI Vision. Your own labeled images are Azure AI Custom Vision. Faces are Azure AI Face. Forms and fields are Azure AI Document Intelligence. Video search and insights are Azure AI Video Indexer.
Where people lose points here
They choose detection when the question says classification
If the required output is one label for the image, it is classification. Detection only matters when the output includes object locations or bounding boxes.
They choose Document Intelligence for every OCR question
Read text from a sign, photo, or scanned page with Azure AI Vision Read. Choose Document Intelligence when the output is structured fields, tables, or form layout.
They forget Limited Access for face recognition
Identity-related Face capabilities are sensitive and controlled. Detection is not the same as identifying a known person from an enrolled group.
They pick Custom Vision without training data
Custom Vision is for custom classification and object detection from labeled images. If the scenario needs general image tags with no custom training, use Azure AI Vision.
Semantic segmentation labels every pixel. Object detection draws rectangles. If the answer options include a custom machine learning model for pixel masks, that is usually the segmentation answer.
The night-before cheat sheet
Four solution types
- Classification: labels the whole image.
- Object detection: returns class plus bounding box for each instance.
- Semantic segmentation: labels every pixel.
- OCR: reads text from images or scanned pages.
Three face operations
- Detection: finds whether and where faces exist.
- Verification: compares two faces and asks whether they match.
- Identification: matches one face against an enrolled group to answer who it is.
Read versus Document Intelligence
Text from a photo, sign, screenshot, or scanned page: Azure AI Vision Read. Key-value pairs, tables, line items, and fields from forms: Azure AI Document Intelligence.
Services in one line
- Azure AI Vision: prebuilt image analysis and Read OCR.
- Azure AI Custom Vision: train custom image classifiers and object detectors.
- Azure AI Face: face detection, analysis, verification, and identification with access controls.
- Azure AI Document Intelligence: structured extraction from documents and forms.
- Azure AI Video Indexer: searchable insights from video and audio.