Models, Deployment, and Configuration
Objective 1.2 asks whether you understand generative model components, how to choose a model from Foundry Models, how deployment works, and which inference setting changes behavior.
How generative models work
A generative model produces new output from a prompt. In Foundry, you usually send input to a deployed model or an agent, receive a completion, and decide how your app should use it. Keep the concepts plain: the model does not look up a single fixed answer. It predicts useful next pieces of output based on patterns learned during training and the context you provide.
Tokens and tokenization
Models read and write tokens, not whole pages. A token can be a word, part of a word, punctuation, or spacing. Tokenization is the process of splitting text into those pieces.
Sentence: Foundry helps you build agents.
Tokens: Foundry | helps | you | build | agents | .Real tokenizers may split words differently, especially names, code, numbers, or punctuation. For the exam, remember the purpose: tokens are the units the model consumes and generates.
Prompt and completion
The prompt is the input: user text, system instructions, retrieved grounding content, images, or other supported modalities. The completion is the generated response. In chat-style APIs, messages have roles such as system and user. The system message is especially important because it sets behavior, constraints, and boundaries.
Next-token prediction
A language model generates output one token at a time. At each step, it estimates likely next tokens given the prompt and everything already generated. Settings such as temperature and top_p affect how deterministic or varied that choice is. They do not turn the model into a database.
Context window
The context window is the amount of input and generated output the model can consider at one time. It includes system instructions, conversation history, grounding passages, user messages, tool results, and the answer being generated. When you exceed it, the app must shorten, omit, summarize, retrieve fewer passages, or otherwise manage context. If important facts fall out of context, the model cannot reliably use them.
Why models hallucinate
A hallucination is a confident, fluent, wrong answer. It happens because the model is generating plausible text, not verifying every claim against a source. You reduce hallucinations with grounding, citations, clear instructions, lower randomness, evaluation, and a rule that the model should say when evidence is missing.
Multimodal models
Multimodal means the model can work with more than one kind of input or output. A multimodal chat model might accept text plus an image and answer questions about a chart. Audio and realtime models can support speech input, speech output, or transcription. Image-generation and video-generation models produce visual media from prompts.
Embeddings
An embedding is a vector representation of content. Similar meanings end up close together in vector space. You use embeddings for semantic search and retrieval-augmented generation. A common RAG flow is: create embeddings for your documents, store them in a search index such as Azure AI Search, embed the user question, retrieve relevant passages, and place those passages in the prompt so the model can answer from current sources.
Choosing a model
Foundry Models is the model catalog. The exam is more likely to test model families and selection criteria than exact version trivia. Choose based on task, modality, capability, cost, latency, context window, region availability, data residency, and whether an SLA is required.
| Family | What it is for | Typical use |
|---|---|---|
| GPT-5.x general chat | General-purpose chat, coding, analysis, and multimodal text plus image input. | Customer assistants, knowledge chat, content drafting, app copilots. |
| o-series reasoning models | Deep reasoning for multi-step logic, math, and complex coding. | Planning, difficult troubleshooting, tasks that need careful step-by-step reasoning. |
| Phi small language models | Microsoft small language models optimized for lower cost, lower latency, and smaller environments. | Edge or local scenarios, simple extraction, routing, and compact assistants. |
| Open-weight models | Llama, Mistral, DeepSeek, and similar families when openness, customization, or ecosystem fit matters. | Workloads needing open weights, managed compute, custom deployment choices, or partner model behavior. |
| Embedding models | Convert text into vectors for semantic similarity. | Azure AI Search, RAG, duplicate detection, finding related passages. |
| gpt-image-1 | Current image-generation family in Foundry Models. | Create or edit images from text prompts and application context. |
| FLUX | Image-generation models from Black Forest Labs. | Alternative text-to-image generation choices in Foundry Models. |
| sora | Video generation, currently a preview area. | Create short video from prompts when the scenario asks for generated video. |
| Audio and realtime models plus whisper | Speech input, speech output, realtime conversation, and transcription. | Voice agents, live audio interaction, call transcription, speech-to-text with whisper. |
| Selection criteria | What to ask | Why it matters |
|---|---|---|
| Task | What must the model do? | Writing, reasoning, image generation, transcription, and embeddings use different families. |
| Modality | What input and output types are required? | Text-only, image input, generated image, video, and audio are not interchangeable. |
| Capability | How hard is the task? | More capable models handle harder reasoning and richer instructions. |
| Cost | How much work will run? | Smaller variants can make high-volume workloads practical. |
| Latency | How quickly must users get a response? | Realtime and user-facing apps often favor faster models. |
| Context window | How much information must fit at once? | Long prompts, conversation history, and RAG passages consume context. |
| Region availability | Is the model available where you need it? | Availability can determine whether a design can be deployed. |
| Data residency | Where may data be processed? | Regulated workloads may require data-zone or regional choices. |
| SLA | Is a service-level agreement required? | Production systems may need deployment types with stronger operational commitments. |
The mini and nano tradeoff is simple: smaller variants usually cost less and respond faster, but may have lower capability on hard reasoning, long instructions, or nuanced content. Use them when speed and scale matter more than maximum reasoning depth.
DALL-E was taught on the old AI-900 path. For current AI-901 content, image generation in Foundry Models points to the gpt-image-1 family, with FLUX as another image-generation family. If an answer choice asks for current image generation, do not default to DALL-E.
Getting a model running
In Microsoft Foundry, choosing a model is not always the same as making it callable. A deployed model exposes an endpoint through a deployment. Your application usually passes the deployment name in code, not the model family name. This matters because you might deploy a model with a friendly name such as support-chat-prod and then use that deployment name in the request.
Three ways to get a model
Serverless API
The preferred path. You do not size infrastructure. It covers pay-per-token, provisioned throughput, and batch styles.
Managed compute
Preview option with dedicated GPU capacity you size. Use it for open-source, community, or custom-weight models when serverless is not the right fit.
Instant access
Preview option that lets you call supported models by name without creating a deployment first. It is useful for quick evaluation where supported.
| Type | Data processed in | Billing | Use when |
|---|---|---|---|
| Global Standard | Any Azure region | Pay-per-token | You want the default starting point with broad model access and no reserved throughput. |
| Global Provisioned (PTU) | Any Azure region | Reserved capacity | You need predictable high-throughput production capacity. |
| Global Batch | Any Azure region | Batch billing for async work | You have large asynchronous jobs that do not require immediate responses. |
| Data Zone variants | Within a data zone such as US, EU, or APAC | Standard, provisioned, or batch pattern | You need data processed within a broader residency zone. |
| Regional Standard | A selected Azure region | Pay-per-token | You have a single-region compliance or architecture requirement. |
| Developer tier | Any supported region | Pay-per-token | You are evaluating fine-tuned models and do not need a production SLA. |
Microsoft guidance is to start with Global Standard and move only for a specific reason: data residency, reserved throughput, or asynchronous batch. Do not choose a more specialized deployment type unless the scenario gives you that reason.
Inference settings
Inference parameters change how the model responds at runtime. They are not model training. They are request or deployment configuration choices that shape behavior, length, variability, stopping, and safety.
| Parameter | What it changes | When to touch it |
|---|---|---|
| System message | Sets role, persona, constraints, allowed behavior, and response rules. | Always define it for app behavior, safety boundaries, and output format. |
| temperature | Controls randomness. Lower is more focused and repeatable; higher is more varied. | Lower it for consistency. Raise it for brainstorming or varied drafts. |
| top_p | Controls nucleus sampling, another way to limit token choice variety. | Use as an alternative to temperature, not as a second knob to tune at the same time. |
| max tokens | Caps generated response length. | Use when you need concise answers, cost control, or predictable output size. |
| frequency penalty | Penalizes tokens based on how often they already appeared. | Use to reduce repetition of the same words or phrases. |
| presence penalty | Penalizes tokens that have appeared at all, nudging the model toward new topics. | Use when output gets stuck on the same ideas and you want broader coverage. |
| stop sequences | Strings that halt generation when encountered. | Use to stop at a delimiter, prevent extra sections, or separate records. |
| Content filters | Apply safety checks to input and output with configurable severity thresholds and blocklists. | Use to reduce harmful content and enforce safety requirements. |
Raising temperature does not make answers more accurate. It makes answers more varied. For factual or grounded responses, prefer clear instructions, grounding, citations, and lower randomness. Tune temperature or top_p, not both at once.
Traps candidates make on objective 1.2
The model family is what you choose from Foundry Models. The deployment name is the name your application passes when calling a deployed model.
Embedding models create vectors for similarity search and RAG. They do not write the final natural-language answer. A chat or reasoning model uses retrieved passages to respond.
Serverless API is the preferred path. Managed compute is a preview option for open-source, community, and custom-weight scenarios where you need dedicated capacity choices.
The context window is what the model can consider now. If old messages or source passages are omitted, the model cannot reliably use them unless your app retrieves or summarizes them back into context.
AI-901 focuses on generative AI concepts, Foundry Models, deployments, and configuration. Retired objective areas from older prep guides are not valid shortcuts for objective 1.2.
The night-before cheat sheet
Model families
GPT-5.x: general chat, coding, analysis, multimodal text plus image input.
o-series: deeper reasoning and multi-step logic.
Phi: small Microsoft language models for speed and lower cost.
Open-weight: Llama, Mistral, DeepSeek, and similar choices.
Embeddings: vectors for search and RAG.
Image: gpt-image-1 and FLUX.
Video: sora preview.
Audio: realtime, audio, live transcription, and whisper.
Selection criteria
Task, modality, capability, cost, latency, context window, region availability, data residency, and SLA. Mini and nano usually mean faster and cheaper, with less headroom for difficult tasks.
Deployment recall
Serverless API: preferred. Managed compute: preview for open-source or custom weights. Instant access: preview, no deployment. Serverless types: Global Standard, Global Provisioned, Global Batch, Data Zone variants, Regional Standard, Developer tier.
Parameter meanings
System message: behavior. Temperature: randomness. top_p: token-choice variety. Max tokens: length cap. Frequency penalty: reduce repeated words. Presence penalty: encourage new topics. Stop sequences: halt at strings. Content filters: safety checks.