Generative AI on Azure

Generative AI produces new content: text, code, images, summaries, search answers, or chat responses. Traditional AI patterns usually return a label, number, group, or prediction. For AI-900, keep it simple: generative AI creates an artifact from learned patterns and prompt instructions.

If the requirement says “generate,” “draft,” “rewrite,” “summarize,” “create,” or “answer in natural language,” think generative AI.

Language generation relies on the Transformer architecture from skill area 2. Transformers represent relationships between tokens, so they can generate text, answer questions, translate language, write code, and use conversation context.

A large language model is a foundation model trained on enormous text collections to predict useful continuations. You send a prompt; the model returns a completion. Chat apps package prompts as messages: system instructions, the user request, prior turns, and grounding data.

A token is a chunk of text: a word, word part, punctuation mark, or whitespace. For example, the sentence Azure AI helps students prepare. might split roughly as Azure AI helps students prepare .. The exact split depends on the model.

The context window is the maximum text the model can consider: instructions, prompt, history, retrieved content, tool outputs, and completion. Facts outside the window cannot be used reliably.

Multi-modal models accept more than one input type. A common example is text plus images, such as asking about a screenshot or diagram. Choose the model family that matches the input and output.

Inference settings affect the current response; they do not train the model.

Prompt engineering means writing instructions that steer the model toward the desired output. It is the cheapest lever because it changes the request, not the model or infrastructure. Good prompts reduce ambiguity.

The system message is the highest-level instruction in many chat apps. It defines role, boundaries, tone, allowed sources, format, safety rules, and what to do when information is missing. Put durable behavior there, not one-time user details.

Specific prompts name the audience, task, format, constraints, and source material. Replace “make this better” with the exact output you want.

Examples help enforce style or structure. Few-shot prompts can often fix format problems without fine-tuning. Chain-of-thought style prompts ask the model to explain reasoning or show steps. In practice, ask for a concise rationale, evidence, assumptions, or calculation steps so users can review the answer.

You can include source material directly in short prompts. Paste the policy, email, ticket, or passage and tell the model to answer only from that text. For large content libraries, use retrieval augmented generation instead of stuffing everything into the prompt.

Grounding means giving the model trusted information to use when it answers. The model does not automatically know your current policies, contracts, tickets, inventory, or private documents. Grounding connects responses to supplied facts.

Retrieval augmented generation, or RAG, is the key grounding pattern. It combines search with generation by retrieving relevant content at request time and placing it in the prompt.

RAG reduces hallucination because authoritative context is in the prompt. It also keeps answers current without retraining: update the index, and the next retrieval can use the new passage. That is very different from fine-tuning, which changes behavior but is not a knowledge-refresh strategy.

Fine-tuning trains a model further on curated examples. It can teach tone, format, domain phrasing, or a narrow task when prompting and grounding cannot produce a consistent pattern. It requires prepared data, evaluation, and more cost.

Fine-tuning is bad for changing knowledge. If the assistant must answer from a handbook, catalog, policy, contract library, or knowledge base, choose RAG. Private or domain-specific data does not automatically mean fine-tuning.

Hallucinations are plausible but false, unsupported, or invented statements. They happen because the model predicts likely text; confident tone is not proof. Mitigate them with trusted grounding, lower temperature, and instructions to say it does not know when context is insufficient. For high-risk decisions, add human review and citations.

Azure OpenAI Service gives Azure customers access to OpenAI models through Azure security, networking, regional availability, monitoring, quota, and governance. You create a model deployment before calling a model so the app has a named endpoint tied to a model version and capacity configuration.

Quota is commonly described in tokens per minute and requests per minute. Long prompts with retrieved content can hit token quota quickly. The Azure OpenAI “on your data” capability wires an Azure AI Search index into a chat deployment for RAG. The model generates the answer; Azure AI Search supplies passages.

Azure AI Foundry is the portal and development environment for generative AI apps on Azure. It provides hubs, projects, model playgrounds, prompt flow, and evaluations for quality, groundedness, relevance, risk, and safety. The Azure AI Foundry model catalog contains OpenAI models and open models from other providers. Serverless API deployment is pay per token with no infrastructure. Managed compute means you provision and pay for the compute that hosts the model.

Azure AI Content Safety detects and filters risky content. It covers hate, sexual, violence, and self-harm categories with severity levels, plus prompt shields for jailbreak and indirect prompt injection attempts, groundedness detection, and protected material detection.

Microsoft describes a four-stage process for responsible generative AI: identify potential harms, measure them, mitigate them, and operate responsibly. Keep doing it after release because model and user behavior can change.

Mitigation is layered: choose an appropriate model, use Azure AI Content Safety and filters, define refusal rules in the system message, ground with trusted data and citations, and design the user experience with warnings and confirmations.

Content filters can block or annotate harmful prompts and responses. Thresholds are configurable by severity, but production apps still need safety controls that match risk. Use human oversight when outputs affect people, money, health, legal obligations, employment, or safety. Be transparent that content is AI-generated and may need source verification.

Responsible AI process: identify harms → measure harms → mitigate harms → operate responsibly. Map every generative AI system to fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.

Final memory hook: Prompt for behavior, RAG for facts, fine-tune for a specialized pattern, and content safety for guardrails.