Machine Learning on Azure
This covers skill area 2, “Describe fundamental principles of machine learning on Azure” (15–20% of the exam).
Regression, classification, clustering
The exam expects you to recognize the three basic machine learning techniques from a scenario. Start with one question: does the training data include the answer the model should learn?
Supervised learning uses labeled examples. Each training row includes input features and the correct answer, called the label. Regression and classification are supervised. Unsupervised learning looks for structure in data without labeled answers. Clustering is unsupervised.
Regression predicts a number. The number can be a price, duration, quantity, temperature, demand level, or risk score. Classification predicts a category. Binary classification has two classes, such as approve or reject. Multiclass classification has more than two classes, such as low, medium, or high priority. Clustering groups unlabeled records by similarity, such as customer segments discovered from behavior.
| Technique | Predicts | Labeled data needed? | Example scenario | Typical metric |
|---|---|---|---|---|
| Regression | A numeric value | Yes | Predict next month’s sales revenue for each store. | RMSE, MAE, R-squared |
| Binary classification | One of two categories | Yes | Predict whether a claim is fraudulent or legitimate. | Accuracy, precision, recall, F1, AUC |
| Multiclass classification | One of three or more categories | Yes | Classify a support ticket as billing, technical, account, or shipping. | Accuracy, precision, recall, F1 |
| Clustering | A group assignment based on similarity | No | Group customers into segments based on purchase patterns. | Cluster quality measures such as separation and cohesion |
If the answer is a number, choose regression. If the answer is a named category, choose classification. If the question says there are no labels and you need natural groups, choose clustering.
Features, labels, and the data splits
A feature is an input used by the model. A label is the known answer the model learns to predict. In a house-price model, size, bedrooms, location, and age are features. The sale price is the label.
| Size | Bedrooms | ZIP code | Age | Sale price | Role |
|---|---|---|---|---|---|
| 1,450 sq ft | 3 | 98103 | 18 years | $640,000 | Features: size, bedrooms, ZIP code, age · Label: sale price |
| 2,200 sq ft | 4 | 98052 | 6 years | $915,000 | Features: size, bedrooms, ZIP code, age · Label: sale price |
| 900 sq ft | 2 | 98118 | 42 years | $485,000 | Features: size, bedrooms, ZIP code, age · Label: sale price |
Training data is split because a model can appear good when it is only memorizing examples. The training set teaches the model. The validation set helps tune choices while you are developing, such as algorithm, features, or hyperparameters. The test set is held back until the end to estimate how the final model performs on unseen data.
Overfitting
The model learns the training data too specifically, including noise. The tell-tale symptom is high training performance but poor validation or test performance. It works on examples it saw and fails on new ones.
Underfitting
The model is too simple or lacks useful features. The tell-tale symptom is poor performance on both training data and validation data. It has not learned the pattern well enough.
Validation data guides model selection during development. Test data is the final unbiased check. If you keep changing the model based on the test set, the test set is no longer a clean test.
Deep learning and the Transformer architecture
A neural network is a model made of connected processing units that transform inputs into outputs. In plain language, each layer learns a representation that helps the next layer make a better prediction. For an image, early layers might detect edges, later layers might detect shapes, and later layers might recognize objects.
Deep learning is “deep” because the network has multiple layers. More layers can learn richer patterns, but they also require more data, more compute, and careful evaluation. AI-900 does not expect you to calculate neural network weights. It expects you to know that deep learning is useful for complex unstructured data such as images, speech, and language.
The Transformer architecture is a neural network architecture designed to process sequences, especially language. Its key idea is attention: the model can weigh which words or tokens are most relevant to each other when interpreting or generating text. Attention lets the model connect information across a prompt, such as linking a pronoun to the noun it refers to or using earlier instructions when creating a later answer.
Transformers matter for AI-900 because they are the basis for many large language models used in generative AI. When the exam later discusses prompts, completions, summarization, grounding, and Azure OpenAI Service, the conceptual foundation is that Transformer-based models use attention over tokens to generate likely next content.
For AI-900, you do not need the math behind attention. Remember the purpose: Transformers handle language sequences well and power modern generative AI systems.
Evaluating a model
Evaluation metrics tell you how a trained model behaves. The best metric depends on the business cost of mistakes. A disease screening model, fraud model, and product recommendation model can all be classification models, but they may need different metrics.
For a disease screening test, assume “positive” means the model predicts the person has the disease.
| Actual disease | Actual no disease | |
|---|---|---|
| Predicted disease | True positive: sick person correctly flagged. | False positive: healthy person incorrectly flagged. |
| Predicted no disease | False negative: sick person missed by the test. | True negative: healthy person correctly cleared. |
| Metric | What it measures | When it misleads |
|---|---|---|
| Accuracy | The share of all predictions that were correct. | It can look excellent when one class is very common. |
| Precision | Of the items predicted positive, how many were truly positive. | It can ignore many missed positives if recall is low. |
| Recall | Of the actual positives, how many the model found. | It can be high while producing too many false alarms. |
| F1 | A balanced score that combines precision and recall. | It hides whether the problem is mostly false positives or false negatives. |
| AUC/ROC | How well the model separates classes across thresholds. | It may be less intuitive than precision or recall for a specific operating threshold. |
Use precision when false positives are expensive. If a model flags transactions for manual review, low precision wastes analyst time. Use recall when false negatives are expensive. In a disease screen, missing a sick patient may be worse than doing extra follow-up tests. Use F1 when you need a balance between precision and recall.
| Regression metric | Meaning | Exam clue |
|---|---|---|
| MAE | Mean absolute error; average absolute difference between predicted and actual values. | Easy-to-explain average error in original units. |
| RMSE | Root mean squared error; penalizes larger errors more heavily than MAE. | Choose when large mistakes should hurt more. |
| R-squared | How much variation in the label is explained by the model. | A higher value usually means a better fit, but it is not an error amount. |
When one class is rare, accuracy is often the wrong answer. A fraud model could be 99% accurate by predicting “not fraud” for every transaction. Use precision, recall, F1, or AUC depending on whether false positives, false negatives, or threshold behavior matters most.
Azure Machine Learning
Azure Machine Learning is Microsoft’s cloud platform for building, training, evaluating, managing, and deploying machine learning models. Azure Machine Learning studio is the web experience where you create workspaces, run experiments, manage data and compute, train models, register models, deploy endpoints, and review responsible AI insights.
Automated machine learning tries multiple algorithms and settings for a supervised task such as classification, regression, or forecasting. It still produces trained models, metrics, explanations, and deployable assets; it does not mean no model exists. The designer is a drag-and-drop canvas for building pipelines without writing much code. Notebooks are code-first environments for data scientists who want full control with Python and SDKs.
Datastores are connections to storage locations. Data assets are versioned references to data used by jobs. Jobs run training, evaluation, or processing work. Pipelines chain jobs together so steps can be reused and automated. The model registry stores trained models with names, versions, metadata, and lineage.
Compute matters because Azure separates authoring, scalable training, and serving. A compute instance is a personal development workstation in the cloud, often used for notebooks. A compute cluster scales out training jobs and can add or remove nodes. An inference cluster or managed online endpoint hosts a model for predictions.
Real-time endpoints return predictions immediately for applications and APIs. Batch endpoints process many records asynchronously, such as scoring a file of customers overnight. The responsible AI dashboard helps examine error analysis, model interpretability, counterfactuals, causal analysis, and fairness-related views when available for the scenario.
| Tool | Who uses it | What it needs | When to pick it |
|---|---|---|---|
| Automated ML | Analysts and data scientists who want fast model selection. | Labeled data, target column, task type, compute. | Pick it when you need the service to try algorithms and compare metrics. |
| Designer | Users who prefer a visual pipeline. | Data assets, components, compute, pipeline configuration. | Pick it when the question says drag-and-drop or no-code pipeline. |
| Notebooks | Data scientists and ML engineers. | Code, packages, compute instance, SDK access. | Pick it when the scenario needs custom Python or full control. |
| Comparison | First option | Second option |
|---|---|---|
| Compute instance vs compute cluster | Compute instance: personal cloud VM for development and notebooks. | Compute cluster: scalable pool for training or batch jobs. |
| Real-time vs batch endpoint | Real-time endpoint: low-latency prediction for one or small numbers of requests. | Batch endpoint: asynchronous scoring for large datasets. |
Where people lose points here
Calling clustering supervised
Clustering does not need labeled answers. If the scenario says discover natural groups, it is unsupervised learning.
Choosing accuracy on imbalanced data
If the positive class is rare, accuracy can reward a useless model. Look for precision, recall, F1, or AUC.
Confusing compute instance and compute cluster
A compute instance is mainly for interactive development. A compute cluster is for scalable training or jobs.
Thinking AutoML writes no model
Automated ML creates trained models and compares them. The automation is in model selection and tuning, not in avoiding models.
Confusing validation data with test data
Validation data helps choose and tune a model. Test data estimates final performance after choices are made.
Forgetting deployment mode
Immediate app predictions need a real-time endpoint. Scheduled scoring for many records usually needs a batch endpoint.
The night-before cheat sheet
Techniques
- Regression: supervised; predicts a number.
- Classification: supervised; predicts a category.
- Clustering: unsupervised; groups unlabeled data.
Confusion matrix cells
- True positive: predicted positive and actually positive.
- False positive: predicted positive but actually negative.
- False negative: predicted negative but actually positive.
- True negative: predicted negative and actually negative.
Classification metrics
- Accuracy: overall percent correct.
- Precision: correctness of positive predictions.
- Recall: coverage of actual positives.
- F1: balance of precision and recall.
- AUC/ROC: class separation across thresholds.
Regression metrics
- MAE: average absolute error.
- RMSE: error metric that penalizes large misses.
- R-squared: variation explained by the model.
Azure Machine Learning components
- Automated ML: tries algorithms and settings for you.
- Designer: visual drag-and-drop pipelines.
- Notebooks: code-first development.
- Datastores: storage connections.
- Data assets: versioned data references.
- Compute instance: personal dev VM.
- Compute cluster: scalable job compute.
- Model registry: versioned trained models.
- Real-time endpoint: immediate predictions.
- Batch endpoint: large asynchronous scoring.
- Responsible AI dashboard: error, fairness, and explainability views.