The Complete Azure Machine Learning Guide

This is part one of three. It covers everything you need to do real work with Azure Machine Learning: create a workspace, give yourself compute that costs nothing while idle, run a training script in the cloud, read the results, register the model, and put it behind a web endpoint that returns predictions. Mid-level and Senior take the same topics further. Nothing here gets thrown away.

Each section ends with a Try it task. Do them as you go. Cloud tooling is the kind of subject you only understand after you have watched a job fail with a real error message and fixed it yourself.

A note on the shape of the product before we start. Azure Machine Learning (Azure ML from now on) is a managed service. There is no server for you to install and no "Azure ML 4.2" to download. What you install is a client: the Azure CLI with its ml extension, or the Python package azure-ai-ml. The service itself lives in Microsoft's cloud. This guide was checked against the Python SDK azure-ai-ml 1.35.0 (released September 2026) and the current CLI v2, which is what the catalogue entry means by its version.

What Azure ML is, and the problem it solves

A machine learning model starts life as a script on a laptop. It reads a CSV, trains something, prints an accuracy, and the person who wrote it is happy. Then the real questions arrive. The dataset is now forty gigabytes and does not fit. Training takes six hours and the laptop is needed for other things. A colleague wants to reproduce the result and cannot, because nobody wrote down which library versions were used. The model has to be called by a web application, which means somebody has to wrap it in a server, package it, host it, secure it, and keep it running.

Azure ML exists to take those problems off your hands. It gives you rented computers on demand, a place to keep versioned data and models, a record of every training run, and a way to publish a trained model as an HTTPS endpoint. It does this by organising everything around a single top-level resource called a workspace, and by letting you describe your work in small YAML files or short Python programs.

That diagram is the spine of the whole guide. Every section below is one arrow in it, or the plumbing that makes an arrow possible.

What came before? Teams used to rent a virtual machine, install Python on it by hand, copy files over with scp, and leave the machine running over the weekend because nobody remembered to switch it off. Some wrote their own scheduling scripts. Azure ML replaces that with managed compute that scales up when you submit work and back down when it finishes, and with a record of what ran, with which code, on which data, producing which result.

There is also an older generation of Azure ML itself that you will meet in search results. Azure ML has two client generations, called v1 and v2. The v1 command-line extension (azure-cli-ml) lost support on 30 September 2025, and the v1 Python SDK (azureml-core and its relatives) lost support on 30 June 2026. Microsoft says existing v1 workflows continue to operate, but they may be exposed to security risks or breaking changes in architecture. Everything in this guide uses v2. If a tutorial you find imports from azureml.core import Workspace, it is v1: close the tab. The v1 and v2 labels describe the client, not the service, so there is no workspace upgrade to perform. One workspace can serve both.

Who uses Azure ML? Anyone whose employer already lives in the Microsoft cloud: banks, telecoms, government bodies and large enterprises across the Gulf and Egypt often standardise on Azure, and for them the practical questions are where the data may live and who may see it. Azure ML workspaces are created in a specific Azure region, and you choose that region when you create the workspace. Pick a region that satisfies your employer's data-residency rules before you create anything, because the workspace and its storage live there.

The mental model: six nouns

Azure ML has a long list of features, but a beginner needs only a handful of nouns. Learn these and the documentation becomes readable.

Subscription and resource group. These are Azure's general scoping tools, not Azure ML ideas. A subscription is the billing container. A resource group is a folder inside it that holds related resources so you can see them together and delete them together. Every command you run needs to know the subscription, the resource group, and the workspace.

Workspace. The top-level Azure ML resource. It holds your jobs, models, data references, environments, compute and endpoints. When you create one, Azure also creates or links four supporting resources: a storage account (where your files and logs go), a Key Vault (where secrets are kept), a container registry (where Docker images are stored), and Application Insights (monitoring). You rarely touch those directly at first, but you will see them in your resource group and wonder what they are. Now you know.

Compute. The machines that run your work. There are four kinds you should recognise. A compute instance is one virtual machine used as a personal development workstation. A compute cluster (the type is called AmlCompute) is a pool that scales between a minimum and a maximum number of machines. Serverless compute means you do not create anything: you submit a job without naming any compute and Azure allocates it for you. Kubernetes compute lets you attach an existing Kubernetes cluster, which is an advanced topic.

Data asset. A named, versioned reference to data. It does not copy the data; it records where the data is. The three types are uri_file (one file), uri_folder (a folder) and mltable (a table described by a small definition file). A datastore is the related idea of a saved connection to a storage service such as Blob storage.

Environment. A Docker image plus a conda file describing which Python packages to install. It answers "what software does this job need?". Microsoft maintains curated environments whose names start with AzureML-, and you can build your own.

Job. One unit of execution: "run this command, with this code, on this environment, on this compute, with these inputs." The common kind is a command job. Other kinds exist (sweep for hyperparameter search, pipeline for chained steps, Spark, and AutoML), but they are all variations on the same idea. Jobs are grouped under an experiment name, which is just a label that keeps related runs together in the studio.

After training, two more nouns appear. A model is a registered, versioned artifact: the thing your job produced. An endpoint is a stable URL and an authentication setting, and a deployment is the model, code and environment actually running behind it. One endpoint can hold several deployments with traffic split between them. We return to this in the deployment section.

There is one more habit of mind worth adopting now. Azure ML is YAML-first. Almost every entity (a job, a compute cluster, an endpoint) can be written as a YAML file, checked into git, and created with one command. The Python SDK does the same things in code. The CLI and the SDK are two front doors to the same house, so you can learn the concepts once and switch between them. This guide shows both, because you will meet both in the wild.

Installing the tools and signing in

There is nothing to install for the service. You install a client. This guide uses both clients, because the CLI is best for creating infrastructure and running YAML jobs, while the Python SDK is best when your logic needs loops or conditions.

You also need an Azure account with a subscription where you are allowed to create resources. If you work for a company, ask your administrator which subscription to use. If you are studying alone, Microsoft offers a free account for new users; check the current terms on the Azure website, since offers change.

The Azure CLI and its ml extension. The Azure CLI is the az command. Azure ML adds its commands through an extension named ml. The official page says you need Azure CLI version 2.38.0 or newer. On Debian or Ubuntu, Microsoft's documented install is:

curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az extension add -n ml -y

On macOS, Homebrew is the usual route, and on Windows the winget package manager or Microsoft's MSI installer works. Check the Azure CLI install page for the exact current instructions for your platform, then add the extension:

brew update && brew install azure-cli
az extension add -n ml

Check that everything is in place:

az version
az ml -h

az version prints the CLI version and the extensions installed; confirm that ml appears. az ml -h prints the list of Azure ML command groups, and seeing job, model, compute and online-endpoint among them tells you the extension loaded. Upgrade it later with az extension update -n ml.

If you ever used the old v1 extension, remove it first, because it uses the same az ml prefix and the two conflict:

az extension list
az extension remove -n azure-cli-ml
az extension remove -n ml
az extension add -n ml

The Python SDK. Install the package that provides the azure.ai.ml module, plus the identity library that handles sign-in:

pip install azure-ai-ml azure-identity

Do this inside a virtual environment (python -m venv .venv, then activate it), so that these packages do not mingle with other projects. Microsoft links its install instructions at https://aka.ms/sdk-v2-install; check them for the Python versions currently supported.

Signing in. Both clients use your Azure identity. Sign in once from a terminal:

az login
az account set -s "<SUBSCRIPTION_NAME_OR_ID>"

az login opens a browser window. az account set chooses which subscription later commands target, which matters if your account can see more than one. The Python SDK then reuses that sign-in through a class called DefaultAzureCredential, which tries several sources of credentials in turn, including your az login session. In automated systems the same class picks up a service principal from environment variables, which is how a CI pipeline signs in without a person.

Creating a workspace and setting defaults

The workspace is the home for everything else, so we create it first. Two environment variables keep the commands short. Set them in your shell (the docs use Bash syntax; PowerShell users must adapt the variable lines):

GROUP=mlops-mena-rg
LOCATION=uaenorth
WORKSPACE=mlops-mena-ws

The location is the Azure region. uaenorth is a Middle East region, used here as an example; list the regions your subscription can use with az account list-locations -o table and choose the one that matches your data-residency requirements. Not every service or machine size is available in every region, so if a later step complains about capacity, a different region is a legitimate remedy.

Create the resource group and the workspace:

az group create -n $GROUP -l $LOCATION
az ml workspace create -n $WORKSPACE -g $GROUP -l $LOCATION

The workspace command takes a minute or two, because it creates the storage account, Key Vault, container registry and Application Insights alongside it. When it finishes it prints a JSON description. Now save defaults so you stop typing the group and workspace on every command:

az configure --defaults group=$GROUP workspace=$WORKSPACE location=$LOCATION
az configure -l -o table

Every az ml command needs a workspace and a resource group, and these defaults supply them. The second command shows what is currently set, which is the first thing to check when a command complains it does not know your workspace.

Open the studio, the web interface, at https://ml.azure.com, sign in, and choose your workspace. You will use it to watch jobs, view metrics and browse assets. Everything it shows is the same data the CLI sees; the studio is a window, not a separate system.

From Python, connect with a client object. Every SDK call goes through it:

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential

subscription_id = "<SUBSCRIPTION_ID>"
resource_group = "mlops-mena-rg"
workspace = "mlops-mena-ws"

ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)
print(ml_client.workspace_name)

If this prints your workspace name, authentication, subscription and workspace are all correct. If it raises DefaultAzureCredential failed, you have not run az login (or have not set service principal variables). If it raises ImportError: No module named 'azure.identity', run pip install azure-identity.

Compute: renting machines and not paying for idle ones

Your laptop is not where training should happen. In Azure ML you rent machines in two shapes, and the difference matters both for convenience and for your bill.

A compute cluster is the workhorse for jobs. You define a name, a virtual machine size, a minimum and a maximum number of nodes. When you submit a job, the cluster scales up from the minimum to run it, and when the work is done, it scales back down after an idle timeout. Set the minimum to 0 and you hold no running machines between jobs. Create one with the CLI:

az ml compute create -n cpu-cluster --type amlcompute --min-instances 0 --max-instances 4

The same cluster from Python:

from azure.ai.ml.entities import AmlCompute

compute = AmlCompute(
    name="cpu-cluster",
    size="STANDARD_D2_V2",
    min_instances=0,
    max_instances=4,
)
ml_client.compute.begin_create_or_update(compute).result()

Two details deserve explanation. First, the begin_ prefix on SDK methods marks a long-running operation: the call returns immediately with a poller object, and .result() waits for it to finish. You will see this pattern everywhere in the SDK. Second, size names an Azure virtual machine type. The right choice depends on your workload and on what your subscription and region have available. If you pick a size you have no quota for, you will meet QuotaExceeded, covered in the errors section.

Serverless compute is the zero-setup alternative. If you omit the compute setting from a job, Azure allocates compute for that job automatically. For a beginner's first jobs this is attractive, because there is nothing to create and nothing to forget about. The cluster is better when you want control over the machine size, want to reuse the same pool repeatedly, or want to cap the number of nodes.

A compute instance is a single VM meant to be your cloud workstation, with notebooks and a terminal available through the studio. It is convenient for exploration, but it has a property that catches beginners: it is a machine that stays on until you stop it, and a running machine is billed. Treat it like a laptop you must remember to close.

Inspect and remove compute with the same grammar as everything else. The CLI is always az ml <noun> <verb> <options>:

az ml compute list -o table
az ml compute show --name cpu-cluster
az ml compute delete -n cpu-cluster --yes

Your first training job

We now run a script in the cloud. A command job has five ingredients: the code to upload, the command to run, the environment, the compute, and the inputs. Start with a project folder:

azureml-first/
  src/
    main.py
  job.yml

Here is a small training script. It reads a CSV of the classic iris flower dataset, trains a LightGBM classifier, and records what it did with MLflow, the open-source tracking library that Azure ML recommends for logging:

import argparse

import lightgbm as lgb
import mlflow
import mlflow.lightgbm
import pandas as pd
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--iris-csv", type=str, required=True)
    parser.add_argument("--learning-rate", type=float, default=0.1)
    args = parser.parse_args()

    df = pd.read_csv(args.iris_csv)
    target = df.columns[-1]
    X = df.drop(columns=[target])
    y = df[target].astype("category").cat.codes

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

    model = lgb.LGBMClassifier(learning_rate=args.learning_rate, n_estimators=50)
    model.fit(X_train, y_train)

    accuracy = accuracy_score(y_test, model.predict(X_test))
    print(f"accuracy={accuracy:.3f}")

    mlflow.log_param("learning_rate", args.learning_rate)
    mlflow.log_metric("accuracy", accuracy)
    mlflow.lightgbm.log_model(model, "model")


if __name__ == "__main__":
    main()

Read the script as an ordinary Python program, because that is what it is. Azure ML does not require a special framework; your script only has to be runnable from a command line and read its settings from arguments. The only Azure-aware parts are the mlflow calls, and even those work on your laptop.

Now the job definition. This is the YAML that tells Azure ML what to do:

$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
code: src
command: >-
  python main.py
  --iris-csv ${{inputs.iris_csv}}
  --learning-rate ${{inputs.learning_rate}}
inputs:
  iris_csv:
    type: uri_file
    path: https://azuremlexamples.blob.core.windows.net/datasets/iris.csv
  learning_rate: 0.9
environment: azureml:AzureML-lightgbm-3.3@latest
compute: azureml:cpu-cluster
display_name: lightgbm-iris-first-run
experiment_name: lightgbm-iris-first-run

Go through it line by line. $schema points to a published definition of the file format, which lets editors such as VS Code validate and autocomplete it. code: src says which local folder to upload. command is what runs inside the machine, and the ${{inputs.iris_csv}} placeholders are replaced with the actual input values at run time. inputs declares two inputs: a data file and a plain number. Note how the file input is given a type (uri_file) and a location, while the number is just written down. environment names a curated environment. The @latest suffix means "the newest version of that environment". compute names the cluster from the previous section; delete that line to use serverless compute instead.

Submit the job:

run_id=$(az ml job create -f job.yml --query name -o tsv)
az ml job show -n $run_id --web
az ml job stream -n $run_id

The first line submits the job and captures its generated name in a shell variable, using --query name -o tsv to extract just that field. The second opens the job's page in the studio. The third streams the job's log to your terminal until it finishes. The job's status moves through Starting, Preparing, Running and finally Completed. The first run is slow, since Azure may need to build the environment image and the cluster has to scale from zero. Later runs reuse the cached image and are much faster. Check the status at any time with:

az ml job show -n $run_id --query status -o tsv

The same job through the Python SDK reads like this:

from azure.ai.ml import command, Input

job = command(
    code="./src",
    command="python main.py --iris-csv ${{inputs.iris_csv}} --learning-rate ${{inputs.learning_rate}}",
    environment="AzureML-lightgbm-3.3@latest",
    inputs={
        "iris_csv": Input(
            type="uri_file",
            path="https://azuremlexamples.blob.core.windows.net/datasets/iris.csv",
        ),
        "learning_rate": 0.9,
    },
    compute="cpu-cluster",
    display_name="lightgbm-iris-first-run",
    experiment_name="lightgbm-iris-first-run",
)

returned_job = ml_client.jobs.create_or_update(job)
print(returned_job.studio_url)
ml_client.jobs.stream(returned_job.name)

create_or_update submits the job and returns right away; studio_url is a link to its page; and stream follows the logs. Notice that the YAML and the Python are the same job: code, command, environment, inputs and compute appear in both.

Reading results: logs, metrics and outputs

A job that ran is only useful if you can see what it did. Azure ML records three kinds of evidence, and knowing where each lives will save you hours.

The studio job page is the main view. The Overview tab shows status, timing, inputs, the compute used and the command. The Metrics tab shows the values you logged with mlflow.log_metric, and for repeated values over time it draws a chart. The Outputs + logs tab holds the files: the standard output of your script (what print wrote) is in a log file there, alongside system logs from the environment preparation. The Code tab shows the exact snapshot of your src folder that was uploaded, which is how you answer "what code produced this result?" months later.

When a job fails, read logs in this order. First the user log, the one that contains your script's output, because most failures are your own code: a missing import, a wrong file path, a typo in an argument. Only if your script never started do you look at the system logs, which describe the environment build and container start. A job that dies in Preparing almost always has an environment problem; one that dies in Running almost always has a script problem. That single distinction solves a large share of beginner failures.

This is also the place to understand why logging with MLflow matters. A printed accuracy is gone once you close the terminal. A logged metric is attached to the job, stored in the workspace, and comparable with other runs. Run the job a second time with a different learning_rate, and the studio lets you tick both runs and compare their metrics side by side. That comparison is the daily work of model development.

From the command line, the same information is available without opening a browser:

az ml job list -o table
az ml job show -n $run_id
az ml job stream -n $run_id

job list shows recent jobs with their status, job show prints the full definition and state, and stream replays the log. You can also download everything a job produced with az ml job download -n $run_id, which is useful when you want the model files locally.

Data: assets, datastores and inputs

So far the data was a public URL, which is convenient for a tutorial and unrealistic for work. Real data lives in your own storage, and Azure ML lets you refer to it by name and version rather than by a path someone might change.

A data asset is that named reference. Recall the three types: uri_file for a single file, uri_folder for a directory, mltable for tabular data described by a definition file. The word URI tells you that the asset stores a location, not a copy. Creating a data asset from a local file also uploads that file to the workspace's default storage, so the asset has somewhere stable to point.

A YAML file describes an asset:

$schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
name: iris-data
version: 1
type: uri_file
path: ./data/iris.csv
description: Iris flower measurements used in the first-job tutorial.

az ml data create -f data.yml
az ml data list -o table

The Python equivalent uses the Data entity and AssetTypes constants:

from azure.ai.ml.entities import Data
from azure.ai.ml.constants import AssetTypes

iris = Data(
    name="iris-data",
    version="1",
    type=AssetTypes.URI_FILE,
    path="./data/iris.csv",
    description="Iris flower measurements used in the first-job tutorial.",
)
ml_client.data.create_or_update(iris)

Once registered, a job refers to it as azureml:iris-data:1 in place of the URL. The path in the job YAML becomes:

inputs:
  iris_csv:
    type: uri_file
    path: azureml:iris-data:1

Why bother? Because versions make experiments reproducible. If somebody replaces the CSV next week, version 1 still points at what you trained on, and a new file becomes version 2. When a model's accuracy drops, you can ask which data version it was trained on and get an answer. A bare path cannot tell you that. If you already know DVC, the idea will feel familiar; see DVC for versioning data with git.

A datastore is the saved connection that makes this possible. The workspace creates a default one for its storage account; you can register more to point at other Blob containers or file shares, and the credentials stay in the workspace rather than in your scripts. One limitation to know about: database datastores are not supported in v2, so data in a database has to be exported to object storage such as Blob first.

Registering the model

A finished job leaves behind a trained model, but it is buried in that job's outputs. Registering a model gives it a name and a version in the workspace, which is what later steps (deployment, sharing, comparison) refer to.

Because our script saved the model with mlflow.lightgbm.log_model(model, "model"), the job's output contains an MLflow-format model at the path model. Azure ML supports three model types: mlflow_model, custom_model and triton_model. MLflow models are the easy path, since MLflow records how to load and call the model. Register from the job on the command line:

az ml model create -n sklearn-iris-example -v 1 -p runs:/$run_id/model --type mlflow_model

Here -n is the model name, -v the version, -p the path (runs:/<job name>/model means "the folder named model inside that job's output"), and --type says it is an MLflow model. In Python the same step is:

from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes

model = Model(
    path=f"azureml://jobs/{returned_job.name}/outputs/artifacts/paths/model/",
    name="run-model-example",
    type=AssetTypes.MLFLOW_MODEL,
)
ml_client.models.create_or_update(model)

Check that it registered:

az ml model list -o table

The mental picture is a library shelf. Each registered model has a name; each registration adds a version; and the studio's Models page lists them with the job that produced each. When you deploy, you say "name and version", never "that folder somewhere".

Deploying a model as an online endpoint

A model that nobody can call is a file. An online endpoint turns it into an HTTPS service that takes a request and returns a prediction in real time. A batch endpoint, a different feature, scores large datasets asynchronously; here we cover the real-time kind.

Two words must be clear. The endpoint is the stable front: a name, a URL and an authentication mode. The deployment is what actually runs behind it: a model, the code that loads and calls it, an environment, and a machine size with an instance count. An endpoint can hold more than one deployment, conventionally called blue and green, with traffic split between them by percentage. That is how you ship a new model version safely: add green, send it ten percent of traffic, and shift more as confidence grows.

A managed online endpoint means Azure runs the underlying infrastructure for you. You choose the instance type and count and do not manage servers.

The endpoint is a short YAML file:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: iris-endpoint-demo
auth_mode: key

auth_mode is key (a secret key you send with each request) or aml_token (a short-lived Microsoft Entra token). Keys are simplest for a first test.

The deployment YAML ties the pieces together. For a non-MLflow model you also supply a scoring script; for an MLflow model Azure ML can generate one, but you should still know what a scoring script looks like, because you will meet it in other people's projects. It has two functions:

import json
import os

import mlflow


def init():
    """Runs once when the container starts. Load the model here."""
    global model
    model_dir = os.getenv("AZUREML_MODEL_DIR")
    model = mlflow.pyfunc.load_model(os.path.join(model_dir, "model"))


def run(raw_data):
    """Runs for every request. raw_data is the request body as a string."""
    data = json.loads(raw_data)["data"]
    predictions = model.predict(data)
    return predictions.tolist()

init() runs once when the container starts, which is where you load the model so that you do not reload it for every request. run() runs for every call. If either raises an exception, callers see HTTP 502, as the errors section explains.

The deployment file references a registered model, the scoring script, an environment, and a machine size:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: iris-endpoint-demo
model: azureml:sklearn-iris-example:1
code_configuration:
  code: ./onlinescoring
  scoring_script: score.py
environment: azureml:AzureML-lightgbm-3.3@latest
instance_type: Standard_DS3_v2
instance_count: 1

The environment must contain everything score.py imports, including the azureml-inference-server-http package that Azure ML uses to wrap your script in a web server. A curated environment aimed at training may not include it, so a deployment-specific environment is common. Check the deployment documentation for the environment you intend to use.

Test locally first. Deploying to the cloud can take many minutes, and a typo costs you the whole wait. Local deployment runs the same container on your machine and needs Docker installed (see Docker):

az ml online-endpoint create --local -n iris-endpoint-demo -f endpoint.yml
az ml online-deployment create --local -n blue --endpoint iris-endpoint-demo -f blue-deployment.yml
az ml online-endpoint invoke --local --name iris-endpoint-demo --request-file sample-request.json
az ml online-deployment get-logs --local -n blue --endpoint iris-endpoint-demo

Here sample-request.json is a file you write containing the input your run() function expects, for example {"data": [[5.1, 3.5, 1.4, 0.2]]}. The --local flag on each command keeps everything on your machine, and get-logs is the first place to look when something is wrong.

When local testing works, drop --local and deploy for real:

az ml online-endpoint create --name iris-endpoint-demo -f endpoint.yml
az ml online-deployment create --name blue --endpoint iris-endpoint-demo -f blue-deployment.yml --all-traffic
az ml online-endpoint show -n iris-endpoint-demo
az ml online-endpoint invoke --name iris-endpoint-demo --request-file sample-request.json

--all-traffic sends one hundred percent of requests to the new deployment; without it the deployment exists but receives none. To call the endpoint from outside, fetch its address and key:

ENDPOINT_KEY=$(az ml online-endpoint get-credentials -n iris-endpoint-demo -o tsv --query primaryKey)
SCORING_URI=$(az ml online-endpoint show -n iris-endpoint-demo -o tsv --query scoring_uri)
curl -X POST "$SCORING_URI" \
  -H "Authorization: Bearer $ENDPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d @sample-request.json

The SDK version of the endpoint and deployment looks like this:

from azure.ai.ml.entities import ManagedOnlineEndpoint, ManagedOnlineDeployment, CodeConfiguration

endpoint = ManagedOnlineEndpoint(name="iris-endpoint-demo", auth_mode="key")

blue = ManagedOnlineDeployment(
    name="blue",
    endpoint_name="iris-endpoint-demo",
    model="azureml:sklearn-iris-example:1",
    code_configuration=CodeConfiguration(code="./onlinescoring/", scoring_script="score.py"),
    environment="AzureML-lightgbm-3.3@latest",
    instance_type="Standard_DS3_v2",
    instance_count=1,
)

ml_client.begin_create_or_update(endpoint).result()
ml_client.begin_create_or_update(blue).result()

endpoint.traffic = {"blue": 100}
ml_client.begin_create_or_update(endpoint).result()

The everyday commands, grouped by what you are trying to do

The CLI has one regular grammar: az ml <noun> <verb> <options>. Learn the verbs once and every noun behaves the same way. Here is a working reference grouped by intent.

Run something. az ml job create -f job.yml submits a job. az ml job stream -n <name> follows its log. az ml job show -n <name> --web opens it in the studio. az ml job cancel -n <name> stops a running job.

Create or change infrastructure. az ml compute create, az ml data create, az ml environment create, az ml model create and az ml online-endpoint create all take -f file.yml, so the file in git is the source of truth. Changing a thing is usually update with a modified file, as in az ml online-deployment update -n blue --endpoint <name> -f blue-deployment.yml.

Look at what exists. list and show exist for every noun. Add -o table to list for a readable summary, and --query with a JMESPath expression to pull one field out of the JSON, as in --query name -o tsv.

Export an existing thing as YAML. az ml <noun> show --output yaml prints the definition of something already created, which is a handy way to learn the schema. Delete the system-generated properties (ids, timestamps) before reusing it as a file.

Clean up. az ml compute delete -n <name> --yes, az ml online-endpoint delete --name <name> --yes --no-wait. The --yes flag skips the confirmation prompt, and --no-wait returns immediately rather than waiting for deletion to finish.

In the SDK, the same nouns appear as attributes of ml_client: ml_client.jobs, ml_client.compute, ml_client.data, ml_client.environments, ml_client.models. Each offers list, get and create_or_update, and the ones that take time use the begin_ prefix with .result(). For example:

for job in ml_client.jobs.list():
    print(job.name, job.display_name, job.status)

for env in ml_client.environments.list():
    print(env.name)

The second loop is the one to run when EnvironmentNotFound appears. A good habit is to explore with list and show, because they cost nothing and teach the vocabulary.

When should you choose the CLI and when the SDK? Microsoft's own guidance is that the CLI is well suited to automation and repeatable pipelines, and the SDK suits programmatic work with logic in it. In practice, many teams write YAML, run it with the CLI from a CI workflow (see GitHub Actions), and reach for Python when they need loops, conditions or tight integration with notebooks.

Configuration and the errors you will actually meet

Most of Azure ML's configuration is the YAML you already wrote, but a few settings live elsewhere. Defaults (group, workspace, location) live in the Azure CLI configuration and are managed with az configure. Credentials come from az login or from service principal environment variables. Project settings such as the environment and compute belong in the job file, and that is where you should keep them.

Errors fall into predictable families. Here are the ones beginners meet, with how to read each.

Sign-in and setup errors.

• ImportError: No module named 'azure.identity': the identity package is missing. Run pip install azure-identity.
• DefaultAzureCredential failed: the SDK found no credentials. Run az login, or set service principal variables in an automated environment.
• ComputeNotFound: the compute name in your job does not match a cluster that exists, or the cluster was deleted. Run az ml compute list -o table and compare the name.
• EnvironmentNotFound: the curated environment was retired or misspelled. List with ml_client.environments.list().
• QuotaExceeded: your subscription does not have enough virtual CPU quota for the machine size and node count. Request a quota increase or choose a smaller size.
• No such file or directory: endpoint.yml: you ran the command from the wrong folder. Change into the folder that contains the YAML.

Deployment errors. These come from the online endpoint troubleshooting page and appear as ERROR: <Code>.

• ImageBuildFailure: Azure could not build the container image. The build log is in the workspace's default storage. A message mentioning container registry authorization failure is fixed with az ml workspace sync-keys.
• OutOfQuota: you ran out of something: CPU quota, disk, memory, endpoint count, or the region simply lacks capacity for that VM size. Choose another size or region, or request quota.
• BadArgument: a family of causes, including a role assignment problem at startup or a model that cannot be downloaded. Read the full message; it names the cause.
• ResourceNotReady: the container crashed while running your score.py. Typical causes are a missing import, a syntax error, or an exception in init(). Use az ml online-deployment get-logs, or reproduce it with a local deployment.
• ResourceNotFound, OperationCanceled and InternalServerError: look at the detail in the message. A canceled operation usually means another operation took priority.

HTTP status codes when you call the endpoint.

• 409: another operation is in progress on the endpoint, for example you tried to delete while it was still updating. Wait and retry.
• 502: an exception or crash inside run() or init() of your scoring script. Read the logs.
• 503: a spike in requests that the instances could not absorb. Add instances.
• 504: the request timed out.
• 500: a failure on Azure's side.

A mistake worth naming: when a job fails, read the first error in the log, not the last. Later messages are usually consequences. The line where the traceback begins is the real cause.

Putting it all together

Here is one small end-to-end project that uses every piece. Treat it as a script for a rehearsal, and do it once from an empty directory.

1. Log in and pick the subscription. az login, then az account set -s "<SUBSCRIPTION>".
2. Create the group and workspace. az group create, then az ml workspace create, then az configure --defaults.
3. Create a cluster that scales to zero. az ml compute create -n cpu-cluster --type amlcompute --min-instances 0 --max-instances 2.
4. Register the data. Write data.yml and run az ml data create -f data.yml.
5. Write the training script and job file. Put main.py in src, write job.yml that uses azureml:iris-data:1, and set display_name and experiment_name.
6. Run it. az ml job create -f job.yml, stream the logs, and read the metrics in the studio.
7. Compare. Change the learning rate, run again, and compare the two runs.
8. Register the best model. az ml model create -n iris-model -v 1 -p runs:/<run>/model --type mlflow_model.
9. Deploy locally, then to the cloud. Test with --local first, then deploy with --all-traffic, and call the endpoint with curl.
10. Clean up. Delete the endpoint, delete the cluster, and delete the resource group if this was only practice.

That last step deserves a command of its own, because deleting the resource group removes everything inside it, including the workspace and its supporting storage, in one go:

az group delete -n $GROUP --yes --no-wait

Do this only when you are sure. It is irreversible, and it is the cleanest way to be certain nothing is left running and billing.

What did you just build? A full loop: data in, training on rented machines, results recorded, a model with a name and version, and a live endpoint. Every production machine learning platform is a larger version of that loop with more safeguards: tests, approvals, scheduled retraining, monitoring and access control. The next level of this guide covers those.

To keep the project reproducible, commit the YAML files, the scripts and a requirements file to git, and write the commands in a short README. A teammate with an Azure subscription can then recreate your result from scratch, which is the real test of whether you have understood the system.

What you can now do, and what comes next

By now you can explain what a workspace, a compute cluster, an environment, a data asset, a job, a model and an endpoint are, and how they connect. You can install the CLI and SDK, sign in, and create a workspace with sensible defaults. You can run a training script in the cloud, read its logs and metrics, register the resulting model, test a deployment locally and publish it, and you know how to read the common errors and how to turn everything off to stop the meter.

Some things were deliberately left for later. Pipelines chain several jobs into a graph, and components are reusable steps within them. Sweep jobs search over hyperparameters. Registries share models and environments across workspaces, so a model proven in a development workspace can be promoted to production without being rebuilt. Autoscaling, network isolation, private endpoints, managed identity, role-based access, monitoring, cost controls and infrastructure as code are the topics that turn a working demo into something a team can rely on. The Mid-level guide starts there.

Neighbouring guides in the catalogue fit naturally around this one. MLflow explains the tracking and model-format layer that Azure ML builds on. Docker makes environments and local deployment testing much clearer. Terraform is one way to create workspaces and compute as code, and Kubernetes is where Kubernetes compute and endpoints lead. If you are working with large language models rather than classical ones, look at Azure OpenAI, which is a separate Azure service with its own model.

Finally, keep the migration lesson in mind. Cloud products change names, retire environments and replace whole client generations, as v1 to v2 showed. Pin the version of azure-ai-ml in a requirements file, re-test when you upgrade, and always check the date on a tutorial before copying commands from it.

Sources

• Azure Machine Learning documentation
• Upgrade to v2
• What is v2
• Install and set up the CLI (v2)
• Train models with the CLI, SDK and REST API
• Deploy a model to an online endpoint
• Troubleshoot online endpoint deployment and scoring
• Azure CLI installation
• azure-ai-ml changelog
• Azure ML examples repository

This file prepares you for the questions an interviewer asks someone who says "I have used Azure Machine Learning". It assumes you read the detailed guide, or at least know the six nouns. The aim is not to recite documentation. It is to sound like a person who has watched a job fail and fixed it.

The thirty-second answer

Azure Machine Learning is a managed cloud service for the machine learning lifecycle. You create a workspace, rent compute that scales up for a job and back down afterwards, run training scripts as recorded jobs, register the resulting models with names and versions, and publish them as HTTPS endpoints. You drive it from the Azure CLI with the ml extension, from the Python SDK azure-ai-ml, or from the web studio, and almost everything can be written as a YAML file kept in git.

If you get thirty more seconds, add the thing that shows you are current: there are two client generations, v1 and v2, v1 is out of support, and everything new is v2.

The model in one picture

An interviewer often asks you to draw it. Draw a line of four boxes and then add the supporting cast.

Then say the nouns out loud. The workspace is the top-level resource that holds everything, created inside a resource group in a subscription. Four supporting resources come with it: a storage account, a Key Vault, a container registry and Application Insights. Compute is a compute instance (one workstation VM), a compute cluster (an autoscaling pool, minimum zero), serverless compute (omit the compute setting) or attached Kubernetes. An environment is a Docker image plus a conda file. A job is one unit of execution, and the common type is a command job. An endpoint is the stable URL with an authentication mode; a deployment is the model, code, environment and machine size running behind it.

Two sentences make the difference between memorising and understanding. First: "A data asset is a reference to data, not a copy, which is why versioning it is cheap and why it makes experiments reproducible." Second: "An endpoint is stable and deployments are replaceable, which is how you roll out a new model without changing the URL the application calls."

Commands to have ready

You will not be asked to recite flags, but you should be able to write the shape of the important commands on a whiteboard without hesitating.

az login
az account set -s "<SUBSCRIPTION>"
az extension add -n ml
az group create -n $GROUP -l $LOCATION
az ml workspace create -n $WORKSPACE -g $GROUP -l $LOCATION
az configure --defaults group=$GROUP workspace=$WORKSPACE location=$LOCATION

That block is setup: sign in, choose the subscription, add the extension, create the resource group and the workspace, then save defaults so later commands are short.

az ml compute create -n cpu-cluster --type amlcompute --min-instances 0 --max-instances 4
run_id=$(az ml job create -f job.yml --query name -o tsv)
az ml job stream -n $run_id
az ml model create -n my-model -v 1 -p runs:/$run_id/model --type mlflow_model

That one is the training loop: create compute, submit a job from YAML, follow its log, register the model from the run's output.

az ml online-endpoint create --local -n $ENDPOINT_NAME -f endpoint.yml
az ml online-deployment create --local -n blue --endpoint $ENDPOINT_NAME -f blue-deployment.yml
az ml online-endpoint invoke --local --name $ENDPOINT_NAME --request-file sample-request.json
az ml online-deployment get-logs --local -n blue --endpoint $ENDPOINT_NAME

And that is the deployment loop, tested locally. Mention that the same commands without --local hit the cloud, and that --all-traffic on online-deployment create gives the new deployment all of the traffic.

The SDK shape is also worth knowing:

from azure.ai.ml import MLClient, command
from azure.identity import DefaultAzureCredential

ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)
job = command(code="./src", command="python main.py", environment="azureml:my-env@latest", compute="cpu-cluster")
ml_client.jobs.create_or_update(job)

If asked what begin_create_or_update(...).result() is about, answer that begin_ marks a long-running operation: the call returns a poller immediately and .result() blocks until Azure finishes.

The classic traps and why they happen

Interviewers love traps because they reveal whether you have run the thing. Here are the ones that come up, each with its mechanism.

Trap one: the forgotten machine. Compute instances stay on until stopped, and endpoint deployments hold virtual machines whether or not they receive traffic. The mechanism is that these are provisioned resources, not on-demand calls. The mitigation is a minimum of zero on clusters, serverless compute for ad hoc jobs, stopping instances, and deleting endpoints when finished. Do not quote prices in an interview; say that you check the pricing page, because prices change.

Trap two: v1 versus v2. A candidate who writes from azureml.core import Workspace has told the interviewer they learned from an old tutorial. The v1 CLI extension (azure-cli-ml) lost support on 30 September 2025, and the v1 SDK on 30 June 2026. The mechanism is a rewrite: v2 reorganised the vocabulary (datasets became data assets, experiments and runs became jobs, ACI and AKS web services became endpoints). Also mention that the label refers to the client, not the service, so there is no workspace upgrade.

Trap three: a stale curated environment. Curated environments, those named AzureML-..., rotate. A tutorial's environment name may not exist now, producing EnvironmentNotFound. The mechanism is simply that Microsoft retires old images. The fix is to list what exists.

Trap four: dependencies missing in the deployment. The model trained fine, but the deployment crashes with ResourceNotReady. The mechanism is that the deployment runs in its own container, built from the deployment's environment, not from the one used for training. If score.py imports something that environment lacks, init() fails and the container never becomes ready. Local deployment with --local exposes this in minutes instead of a long cloud wait.

Trap five: quota. QuotaExceeded on a cluster and OutOfQuota on a deployment mean your subscription has no allowance left for that VM family in that region, or the region has no capacity. The mechanism is that quota is per subscription, per region and per VM family. Fixes are a smaller or different size, a different region, or a quota request.

Trap six: mixing the two SDK generations. Class names like Workspace and Model exist in both and clash. Never put v1 and v2 imports in one script.

Trap seven: reading the last error. In a failed job log, the last lines are consequences. The real cause is where the traceback begins. Say this and the interviewer will nod, because it comes from experience.

Common interview questions

What is a workspace and what is created with it? The workspace is the top-level Azure ML resource that organises jobs, models, data, environments, compute and endpoints. When it is created it comes with a storage account for files and logs, a Key Vault for secrets, a container registry for Docker images and Application Insights for monitoring. Those resources can sit in another resource group but must share the subscription and tenant. Companies typically use separate workspaces for development, testing and production.

What is the difference between a compute instance and a compute cluster? A compute instance is one virtual machine used as a personal development workstation, with notebooks and a terminal. It stays on until stopped. A compute cluster is a pool that scales between a minimum and maximum node count and scales down after an idle timeout, so it suits training jobs and can sit at zero nodes between runs. Serverless compute goes further: you omit the compute setting and Azure allocates machines for the job.

What is a job, and what are its parts? A job is a unit of execution. A command job needs code, a command, an environment, compute (or serverless) and inputs. Other job types include sweep for hyperparameter tuning, pipeline for a graph of child jobs, Spark and AutoML. Jobs are grouped under an experiment name. The job records the code snapshot, inputs, logs and metrics, so a result can be traced back to what produced it.

Why use data assets instead of file paths? A data asset is a named, versioned reference with a type: uri_file, uri_folder or mltable. Because it is versioned, the training data of a given model can be identified later, even if the files change. It also keeps storage credentials in the workspace instead of in scripts.

What is an environment? A Docker image plus a conda file. It fixes the operating system and the Python packages the job or deployment runs with. There are curated environments from Microsoft and custom ones you build. Pinning versions in an environment is what makes a job reproducible.

How do you track experiments? With MLflow. In the script, you log parameters with mlflow.log_param and metrics with mlflow.log_metric, and you log the model with a flavour such as mlflow.lightgbm.log_model. Azure ML stores these against the job, and the studio compares runs side by side. Microsoft recommends MLflow for tracking.

What is the difference between an endpoint and a deployment? An endpoint is the stable address plus authentication. A deployment is the model, scoring code, environment and instance type actually running behind it. One endpoint can hold several deployments, and traffic is split among them by percentage, which enables blue-green rollouts: deploy green, send it a small share, then shift more.

Online endpoint or batch endpoint? Online endpoints serve real-time requests with low latency. Batch endpoints score large datasets asynchronously and are the right choice when no one is waiting for the answer. An overnight scoring run of a whole customer table is a batch problem; a fraud check during a payment is an online one.

What is a scoring script? A Python file with two functions. init() runs once when the container starts and loads the model. run(raw_data) runs on each request and returns the prediction. A crash in either function produces an HTTP 502 for the caller. MLflow models can be deployed without writing one, but the pattern is worth knowing.

How do you debug a failed deployment? Reproduce it locally with --local, which needs Docker, and read az ml online-deployment get-logs. Match the error code. ImageBuildFailure points at the image build log, and a container registry authorization failure is fixed with az ml workspace sync-keys. ResourceNotReady means the container crashed in score.py: missing imports, a syntax error or a failing init(). OutOfQuota is a capacity or quota problem.

What do the HTTP codes 502, 503 and 504 mean for an endpoint? A 502 means an exception or crash in run() or init() of the scoring script. A 503 means a spike in traffic the instances could not absorb, so add instances or minimum capacity. A 504 means the request timed out. A 409 means another operation is running on the endpoint, such as a delete during an update.

CLI or SDK? Both drive the same service. The CLI with YAML is excellent for repeatable automation and CI, because the YAML lives in git and the command is one line. The SDK is better when logic is needed: loops, conditions, building jobs dynamically. I would choose YAML and the CLI by default and reach for Python when the YAML gets awkward.

What changed between v1 and v2? The client generation. v1 CLI support ended 30 September 2025 and v1 SDK support ended 30 June 2026. Datasets became data assets (FileDataset to uri_folder, TabularDataset to mltable), experiments and runs became jobs, HyperDrive became sweep jobs, ParallelRunStep became batch endpoints, and ACI and AKS web services became managed online endpoints or Kubernetes endpoints. The service did not change, so one workspace can serve both.

How do you keep costs under control? Minimum zero on clusters, serverless for ad hoc work, stop compute instances, delete idle endpoints, choose the smallest machine that does the job, and tag or group resources so the bill can be understood. I also check the pricing page rather than assuming.

How would you share a model with another team? In a beginner's answer: register it with a name and version and give them access to the workspace. A more complete answer mentions registries, which share models, environments and components across workspaces.

Where would secrets go? In Key Vault, not in code or YAML. The SDK v1 helpers for secrets were removed, so use the Key Vault client libraries, or reference secrets in a deployment so they are injected as environment variables.

What is the first thing you do when a job fails? Open the logs and find the first error, not the last. Decide whether it failed in Preparing (environment) or Running (my script). Then check the obvious: compute name, environment name, input paths, quota.

How to talk about a project

Interviewers often ask, "Tell me about something you built with it." Structure the answer in five beats: the problem, the data, the training job, the deployment and what you would change. A concrete example: "I trained a classifier on a public dataset, registered the data as a versioned asset, ran it as a command job on a cluster that scales to zero, compared learning rates in the studio using MLflow metrics, registered the best model, tested a managed online endpoint locally with Docker and then in the cloud, and deleted everything at the end." Then add a limitation and a next step: "A real version would use a pipeline, a schedule for retraining and monitoring for drift."

Notice what that answer does. It names real features correctly, it shows you know the cost discipline, and it ends honestly. Do not claim production experience you do not have; the follow-up questions will find the edge of your knowledge within two minutes.

Be ready for one regional question as well. Many Gulf and Egyptian employers care about where data is stored. The right answer is that a workspace and its associated storage live in the Azure region you choose at creation time, so the region decision is made first and is hard to change, and that you would confirm the regulatory requirement before creating anything.

Questions to ask them

Good candidates ask questions that show they understand the operational side. Consider: Do you use one workspace per environment? Are the workspaces behind private endpoints? Is infrastructure created with code, and do jobs run from CI? Is v1 code still in the estate, and is there a migration plan? How are quotas managed across teams? The answers tell you how mature the team is, and asking about v1 shows you know about the deadline.

Sixty-second self-test

Answer each aloud, without looking back. Then check against the guide.

1. Name the six nouns in the order a training request meets them.
2. What four resources are created alongside a workspace?
3. Which compute type can scale to zero, and which stays on until stopped?
4. What are the five ingredients of a command job?
5. Which CLI grammar do all commands follow?
6. Why register data as an asset instead of using a path?
7. What does begin_create_or_update(...).result() do?
8. What is the difference between an endpoint and a deployment?
9. What do init() and run() do in a scoring script?
10. What does a 502 from an endpoint usually mean?
11. When did v1 SDK support end, and what replaced Experiment and Run?
12. Name three habits that keep the bill small.

Model answers in brief: subscription, resource group, workspace, compute, environment, data. Storage account, Key Vault, container registry, Application Insights. A cluster (minimum 0) scales down; an instance stays on. Code, command, environment, compute, inputs. az ml <noun> <verb>. Versions make runs reproducible and keep credentials out of scripts. It starts a long-running operation and waits for it. The endpoint is the stable URL and auth, the deployment is what runs behind it. init() loads the model once; run() scores each request. A crash or exception in the scoring script. 30 June 2026, and jobs. Minimum zero, serverless for ad hoc work, delete idle endpoints and instances.

If you got ten or more, you are ready for a beginner-level conversation. If you missed the endpoint and deployment distinction or the v1 and v2 dates, reread those sections, because those two come up most.

For further reading in this catalogue, connect Azure ML to MLflow, which supplies its tracking and model format, and to Docker, which underlies environments and local deployment. Then try the Mid-level interview file.

This file collects the mistakes beginners actually make with Azure Machine Learning, each with the symptom you will see, the cause underneath, and the fix. After that come habits worth adopting on day one, and a short checklist to run before you walk away from a session.

None of this replaces the detailed guide. It is the file to keep open in a second window while you work.

Mistakes beginners actually make

Following a v1 tutorial

Symptom. You copy a snippet that starts from azureml.core import Workspace, Experiment, and either it errors or it does something the current documentation never mentions.

Cause. Azure ML has two client generations. The v1 CLI extension azure-cli-ml lost support on 30 September 2025, and the v1 SDK (azureml-core and friends) lost support on 30 June 2026. A huge amount of older content still ranks well in search results.

Fix. Check the import. The current SDK is azure-ai-ml, imported as azure.ai.ml, and the client is MLClient. The current CLI is the ml extension. If a page mentions Experiment, ScriptRunConfig, Dataset.get_by_name or HyperDriveConfig, it is v1. The v2 equivalents are command() jobs, data assets and sweep jobs. Do not mix the two in one script, because names like Workspace and Model exist in both and clash. Run az extension list and remove azure-cli-ml if it appears.

Leaving something running

Symptom. A bill that is larger than the one job you ran justifies.

Cause. Several things bill while they exist. A compute instance runs until you stop it. An online endpoint deployment holds virtual machines whether or not anyone calls it. The documentation's clean-up section also notes that a compute cluster continues to bill while it exists.

Fix. Set --min-instances 0 on clusters, prefer serverless compute for one-off jobs, stop compute instances at the end of every session, and delete endpoints with az ml online-endpoint delete --name <name> --yes --no-wait. For a practice project, delete the whole resource group at the end with az group delete -n $GROUP --yes --no-wait. Check the Azure ML pricing page for the details that apply to you, because this guide does not quote prices.

Creating the workspace in the wrong region

Symptom. Everything works, but your employer tells you that data must stay in a particular country, or the VM size you need is not offered where the workspace lives.

Cause. The workspace and its associated storage live in the region chosen at creation. The same region also decides which machine sizes and how much quota are available to you.

Fix. Decide the region before running az ml workspace create. List what your subscription offers with az account list-locations -o table. A workspace cannot be casually moved afterwards; the practical fix is to create a new one in the right region and register your assets there.

Forgetting the defaults

Symptom. A command fails complaining that the workspace or resource group is not specified, or it runs against a different workspace than you intended.

Cause. Every az ml command needs a workspace and a resource group. Without az configure --defaults, you must pass -w and -g each time, and a previous session's defaults may point somewhere else.

Fix. Run az configure --defaults group=$GROUP workspace=$WORKSPACE location=$LOCATION and check with az configure -l -o table. Make that listing the first command of every session, in the same way you would check which git branch you are on.

Signed in to the wrong subscription

Symptom. The workspace you created last week cannot be found, or creation fails with a permission error, even though you are signed in.

Cause. An Azure account can see several subscriptions. az login picks a default, and it may not be the one you meant.

Fix. Run az account show to see the active one, and az account set -s "<SUBSCRIPTION_NAME_OR_ID>" to change it. If the SDK reports DefaultAzureCredential failed, you are simply not signed in; run az login again.

Curated environment names that no longer exist

Symptom. EnvironmentNotFound, right after you copied an environment name from a blog post.

Cause. Curated environments, the ones named AzureML-..., are maintained and retired by Microsoft. Even the documentation's own examples use different names in different places.

Fix. Never trust a copied name. List what exists:

for env in ml_client.environments.list():
    print(env.name)

Pick a current one, or build your own environment from a conda file so that you control the package versions. For anything you will run for months, a custom environment with pinned versions is more stable than a curated one with @latest.

Using @latest everywhere

Symptom. A job that worked last month fails today, or produces slightly different numbers, with no change to your code.

Cause. azureml:AzureML-lightgbm-3.3@latest means "the newest version", and the newest version changes. The same goes for unpinned packages in a conda file.

Fix. Pin versions that matter: environment versions, package versions, data asset versions. Use @latest while exploring and a fixed version once a result matters. Pin azure-ai-ml in requirements.txt too, and re-test when you upgrade.

Blaming the code when the environment failed (and the other way around)

Symptom. A job fails and you spend an hour editing main.py, but the error never mentions your script.

Cause. A job passes through phases: Starting, Preparing, Running, Completed. If it dies in Preparing, the environment or image failed to set up. Your script never ran.

Fix. Look at the status first, then the logs. Failure in Preparing points at the environment, the image build or the container registry. Failure in Running points at your script. In the studio, open Outputs + logs: the user log holds your script's output, and the system logs describe image preparation.

Reading the last line of the log

Symptom. You see a final message like "process exited with code 1" and have no idea what went wrong.

Cause. The last lines are consequences. The root cause is earlier.

Fix. Scroll up to where the traceback starts. Read the first exception, not the final summary. Nine times in ten, it is a missing import, a wrong path or a mistyped argument name.

A path that works locally but not in the job

Symptom. FileNotFoundError for a file you can see on your laptop.

Cause. The job runs on a different machine. Only the folder named in code: is uploaded, and data arrives through inputs, not through whatever happens to be on your disk.

Fix. Pass every file the script reads as an input, and refer to it with the ${{inputs.name}} placeholder in the command. Make sure the script is inside the code folder, and that the command is written relative to that folder, so python main.py and not python src/main.py when code: src.

Running a command from the wrong folder

Symptom. No such file or directory: endpoint.yml or job.yml.

Cause. The CLI resolves relative file paths from where you run the command, and the code: path inside the YAML is resolved relative to the YAML file's location.

Fix. Change into the folder that holds the YAML before you run it, and keep each project in its own folder with the YAML next to its code.

Hitting quota

Symptom. QuotaExceeded creating a cluster or OutOfQuota creating a deployment.

Cause. Quota is a limit on virtual CPUs per subscription, per region and per VM family. A brand-new subscription can have small allowances, and a region can run out of capacity for a given size.

Fix. Choose a smaller size, lower max_instances, try another region, or request a quota increase through the Azure portal. Do not retry the same request repeatedly; it will fail the same way.

Skipping the local test for endpoints

Symptom. A cloud deployment takes ten minutes and then fails with ResourceNotReady, and you repeat the cycle five times.

Cause. The container crashes while loading your scoring script. Typical reasons are a missing package in the deployment environment, a syntax error, or an exception in init(). Each cloud attempt pays for image build and provisioning.

Fix. Deploy locally first. It needs Docker running:

az ml online-endpoint create --local -n $ENDPOINT_NAME -f endpoint.yml
az ml online-deployment create --local -n blue --endpoint $ENDPOINT_NAME -f blue-deployment.yml
az ml online-deployment get-logs --local -n blue --endpoint $ENDPOINT_NAME

The logs show the real traceback in seconds. Only deploy to the cloud when the local run answers a request correctly.

A deployment that gets no traffic

Symptom. The deployment shows as healthy, but invoking the endpoint returns an error, or nothing reaches the new version.

Cause. A deployment exists separately from the traffic rule. If you did not pass --all-traffic when creating it, the endpoint routes zero percent to it.

Fix. Add --all-traffic to az ml online-deployment create, or set traffic afterwards. In Python: endpoint.traffic = {"blue": 100} and then ml_client.begin_create_or_update(endpoint).result(). Check the current split in az ml online-endpoint show -n $ENDPOINT_NAME.

Training environment used for scoring

Symptom. The model trained perfectly and the endpoint crashes at startup with ModuleNotFoundError, or a TypeError about register() taking three positional arguments but four being given.

Cause. The deployment builds its own container from the environment you give the deployment. If that environment lacks a package your score.py imports, init() fails. The register() message the documentation lists is a version mismatch between Flask and azureml-inference-server-http.

Fix. Give the deployment an environment that has everything score.py needs, including azureml-inference-server-http, with compatible versions. Test locally.

Hard-coding secrets and keys

Symptom. An endpoint key, storage key or password sits in a script or YAML file, and then in git history.

Cause. It was the fastest way to make something work.

Fix. Keep secrets in Key Vault or fetch them at run time. Endpoint keys come from az ml online-endpoint get-credentials -n $ENDPOINT_NAME -o tsv --query primaryKey. Add .env files and local config to .gitignore. If a key was committed, rotating it matters more than rewriting history.

Disabling certificate checks to get past a proxy error

Symptom. CERTIFICATE_VERIFY_FAILED when az login contacts login.microsoftonline.com, only on your corporate network or while a security gateway such as Cloudflare WARP is on.

Cause. The gateway inspects encrypted traffic and re-signs it with its own certificate authority, which your tool does not trust. This is local knowledge rather than something in Microsoft's docs.

Fix. Ask IT for the organisation's root certificate and configure the tool to trust it. Do not switch verification off. It trades a one-time setup for a standing security hole.

Habits worth adopting on day one

Keep everything as YAML in git. A job, compute definition, data asset and endpoint written as files can be reviewed, diffed and re-run. Using the studio's buttons is fine for exploring, but a result you cannot recreate is a result you cannot trust. See GitHub Actions when you want those files to run from a pipeline.

Name things deliberately. Use a consistent pattern for experiment names, display names, data assets and models, such as iris-lr0.05-baseline. The studio is only as readable as your naming.

Log with MLflow from the first script. Even a toy script should call mlflow.log_param and mlflow.log_metric. It costs two lines and turns a pile of runs into a comparable table. For the concepts behind it, read MLflow.

Explore with list and show. Every noun has them, they are free, and they teach you the vocabulary and the YAML schema. az ml <noun> show --output yaml prints a real, working definition you can learn from.

Use --query and -o tsv to capture values. run_id=$(az ml job create -f job.yml --query name -o tsv) is the idiom for chaining commands in a script, and it avoids copy-pasting generated names.

Use serverless or zero-minimum compute by default. You can always give yourself a bigger cluster when a job proves it needs one.

Run one thing at a time at first. When a first project goes wrong it is easier to diagnose a single failing step than a pipeline of five. Make the single job work, then register, then deploy, then automate.

Read the whole error message. Azure ML errors are unusually specific. Names such as ImageBuildFailure, ResourceNotReady and OutOfQuota tell you which phase failed, and the message beneath often tells you what to do.

Keep your Docker knowledge current. Environments and local deployment are Docker underneath. An hour with Docker pays back the first time an image build fails.

Pin and note versions. Write the azure-ai-ml version, the CLI version and the extension version in your README. When something breaks after an upgrade, you will know what changed. The service moves quickly, and tutorials age.

Treat the date on a tutorial as a warning label. Check the version of the page, whether it uses v2, and whether the environment names still exist.

A tiny debugging playbook

When something fails, run through these questions in order before you ask anyone else.

1. Am I signed in to the subscription I think I am? az account show.
2. Are my defaults right? az configure -l -o table.
3. Does the thing I named exist? az ml compute list -o table, az ml model list -o table, and the environment list.
4. Which phase failed: Preparing or Running?
5. What is the first error in the user log, not the last?
6. Is it quota? Look for QuotaExceeded or OutOfQuota.
7. For endpoints: does it work locally with --local?
8. What changed since it last worked: code, environment version, data version, package version, region?

Most beginner problems die by step four. The rest are quota and dependency mismatches.

Safe ways to practise

You learn faster with something at stake, but not money. Here are three practice projects that stay cheap and teach distinct skills.

Project one: compare two runs. Use the iris example. Run it with three learning rates, name each run clearly, and compare them in the studio. Skills: command jobs, inputs, MLflow metrics, the compare view.

Project two: version your data. Register iris.csv as version 1, edit a few rows, register version 2, and run the same job against each. Skills: data assets and reproducibility. Then answer the question "which data trained which model?" from the studio alone.

Project three: break a deployment on purpose. Deploy locally with a scoring script that imports a package the environment lacks, read the log, fix it, and deploy again. Skills: reading deployment errors, get-logs, environments. Do this before you need it at work.

In all three, the discipline is the same: run the cleanup commands at the end. If you want the full path from empty subscription to endpoint, follow the "Putting it all together" section of the detailed guide.

The beginner's checklist

Before you start a session:

• I know which subscription is active (az account show).
• My defaults point at the workspace I mean (az configure -l -o table).
• I know which region this workspace is in, and that it is allowed.
• I have written the cleanup commands at the bottom of my notes.

Before you submit a job:

• I am using v2: azure-ai-ml, MLClient, and the ml extension.
• The environment name exists, and I checked by listing.
• Every file the script reads arrives as an input.
• The command is relative to the code folder.
• The job has a display_name and an experiment_name.
• The compute is a cluster with minimum zero, or serverless.
• The script logs its metrics with MLflow.

Before you deploy:

• The model is registered with a name and version.
• The deployment environment contains everything score.py imports.
• It works locally with --local.
• I passed --all-traffic, or I set the traffic split on purpose.
• I know how to delete the endpoint.

Before you walk away:

• Compute instances are stopped.
• Endpoints I do not need are deleted.
• Clusters are at minimum zero or deleted.
• YAML and scripts are committed, with no secrets in them.
• I wrote down the versions I used.

Print the checklist or paste it into the top of your notes. The first time it saves you from a surprise bill or a mysterious failure, it will have paid for itself.

When you are comfortable with everything here, move on to the Mid-level guide, where the same ideas grow into pipelines, registries, networking and automation.

This is part two of three. It picks up where Beginner ended and takes every topic from there further, then adds the machinery you have not met yet. It is written against the v2 client surface: the Python SDK azure-ai-ml 1.35 (released 8 September 2026) and the ml extension of the Azure CLI. Where you already know the basics, we go straight to the depth.

Where this picks up

Each section starts with the problem it solves, and ends with a Try it you can do against a real workspace in a few minutes. Azure ML is a managed service, so there is no version of "the tool" to install. What you pin is the client, which is why the version at the top refers to the SDK. Commands here use Bash syntax; PowerShell users must adapt the shell variables.

What you are really talking to

Beginner treated the workspace as a place where things live. At work you need a sharper picture, because most confusing failures come from not knowing which system rejected your request.

Every az ml command and every MLClient call goes to Azure Resource Manager (ARM), the control plane that creates and updates Azure resources. When you run az ml job create, the CLI reads your YAML, uploads your code folder to the workspace's default storage account, and sends the job definition to ARM over HTTPS. From then on the service schedules the work: it allocates compute, pulls or builds the environment image, mounts or downloads inputs, runs your command, and records status. Your laptop is only a remote control. You can close the terminal and the job keeps going.

That split matters because the two halves are secured and networked differently. Creating, updating and deleting entities, and fetching endpoint keys, are control-plane operations. They go to ARM over the public network even when your workspace is private. Scoring requests to an online endpoint are data-plane traffic, and they go to the endpoint's own scoring URI. A workspace can therefore be fully locked down for scoring while an engineer's az ml online-deployment update still succeeds from a laptop. If you need the control plane private too, the documentation describes either enabling public access for workspace management or using a Private Link for ARM, which requires subscription-owner rights and a role at the root management group.

The workspace itself is a resource of type Microsoft.MachineLearningServices/workspaces with four associated resources: a Storage account, a Key Vault, a Container Registry and Application Insights. They may live in a different resource group from the workspace but must share its subscription and tenant. Each of the four is a dependency you will eventually debug. A job that cannot upload code is a storage problem. An image that will not build is often a registry problem. A deployment that cannot read a secret is a Key Vault problem. Knowing which neighbour is involved cuts the search space.

Finally, remember what "v2" refers to. It is the client API, the SDK and the CLI, and not the service. There is no workspace upgrade, and one workspace can serve both generations of client. New features ship only in v2, and the SDK v1 support window closed on 30 June 2026, so everything below is v2.

The life of a job

Beginner showed the happy path: Starting, Preparing, Running, Completed. Knowing what happens inside each phase lets you tell a slow job from a stuck one.

Starting means the service is allocating compute. On a cluster with min_instances of 0, the first job after an idle period waits for a VM to be provisioned, which is the price of not paying for idle nodes. On serverless compute, which you get by omitting the compute line from a command job, the service picks and allocates resources for you. Either way, this phase is where QuotaExceeded shows up: the subscription has a vCPU quota per VM family per region, and a request that does not fit is refused.

Preparing is the environment phase. The service needs a container image that contains your dependencies. For a curated environment, or one you built before, this is a pull. For a new environment defined from a Docker context or a conda file, it is a build in the workspace's Container Registry, and the first job pays for it. If the registry uses a custom domain-name label, jobs can fail at startup. The documented fix is to use <registry-name>.azurecr.io and set the registry's domain name label scope to Unsecure.

Running is your command, executed inside the container on the node, with your code folder snapshotted and available as the working directory. Inputs are resolved before your command starts, and ${{inputs.name}} in the command string is replaced with a path or a literal value. Outputs and logs stream back to the workspace storage.

A useful habit is to separate three kinds of time when a job feels slow: waiting for a node, preparing the image, and running your code. Studio shows these as the job's status transitions, and az ml job stream -n $run_id prints the logs as they arrive. If a job sits in Starting for minutes, look at quota and cluster scaling before you suspect your code.

Job status is also how automation should read results. This is a scriptable pattern for CI:

run_id=$(az ml job create -f job.yml --query name -o tsv)
az ml job stream -n "$run_id"
status=$(az ml job show -n "$run_id" --query status -o tsv)
test "$status" = "Completed" || { echo "job $run_id ended as $status"; exit 1; }

Note that stream returns when the job reaches a terminal state, but a terminal state is not always Completed. Checking the status afterwards is what turns a failed training job into a failed pipeline step.

Environments: the real unit of reproducibility

A job is reproducible only if its environment is. Beginner used a curated environment. At work you will define your own, and the choices you make here decide whether a rerun next month behaves like today's run.

An environment is a Docker image plus, optionally, a conda file layered on top. There are three common shapes. A curated environment is maintained by Microsoft, with names starting AzureML-. A conda-on-base environment takes a base image and a conda.yaml and lets the service build the result. A Docker-context environment builds from your own Dockerfile.

Curated names rotate. The fact that a tutorial uses AzureML-lightgbm-3.2-ubuntu18.04-py37-cpu@latest does not mean it will exist next quarter. When a job fails with EnvironmentNotFound, a retired curated environment is the usual cause. List what exists with ml_client.environments.list(), and treat curated environments as a convenience for experiments, not a dependency for production.

For anything you intend to keep, register your own environment and give it a version:

$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: train-env
version: 3
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04:latest
conda_file: conda.yaml
description: Training dependencies pinned for the churn model

az ml environment create -f environment.yml

Then refer to it by name and an explicit version, azureml:train-env:3, rather than @latest. The @latest form resolves at submit time, so two submissions an hour apart can run in different environments. An explicit version pins the thing you tested. One more trap is on the REST side: ARM has no @latest for environments, so automation that talks to REST directly must resolve a concrete version first.

Inside the conda file, pin your direct dependencies, and decide deliberately whether you need to pin transitive ones. The service caches image builds, so an unchanged definition does not rebuild, and a changed definition triggers a new build on the first job that uses it. That is the reason environment changes belong in their own pull request: the first job after the change will be slower, and a failure there should be attributable to the change.

Data assets, datastores and how inputs reach your code

Beginner passed a URL as an input. In a real project the data lives in storage you own, and you want it versioned and referable by name.

A datastore is a pointer to storage: Blob, Azure Data Lake or Azure Files. The credentials live with the workspace, so your code never carries a connection string. A data asset sits on top of a datastore (or a public URL) and gives a path a name and a version. There are three types, and choosing correctly saves you later:

The migration table from v1 gives the intuition. A v1 FileDataset maps to uri_folder, and a v1 TabularDataset maps to mltable. Database datastores are not supported in v2: export the data to object storage first.

Create a data asset from YAML and register a new version when the data changes, not when the code changes:

$schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
name: churn-training
version: 4
type: uri_folder
path: azureml://datastores/workspaceblobstore/paths/churn/2026-09/
description: September snapshot, deduplicated

az ml data create -f data.yml

Then a job consumes it by name and version:

$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
code: src
command: >-
  python train.py --data ${{inputs.training_data}} --n-estimators ${{inputs.n_estimators}}
inputs:
  training_data:
    type: uri_folder
    path: azureml:churn-training:4
  n_estimators: 300
environment: azureml:train-env:3
compute: azureml:cpu-cluster
experiment_name: churn-training

The command sees training_data as a path. How that path is provisioned, mounted or downloaded, depends on the input mode. If your code reads a small file once, either works. If it reads a large dataset, check the input mode options in the current documentation for your input type before you assume a default, because a download of a large folder can dominate job time. I have left the exact mode names out here because they were not part of the verified material for this guide.

The versioning discipline is the point. A model trained on churn-training:4 is reproducible because version 4 will keep meaning the same files. If you overwrite files under a path that an existing version points at, you have broken that promise silently. Write new snapshots to new paths and register new versions.

Components and pipelines

Once you have more than one step, scripts run by hand stop scaling. A component is a reusable, versioned pipeline step with declared inputs and outputs. A pipeline job is a graph of child jobs wired together through those inputs and outputs. Components exist only in v2, and they are the unit of reuse: the same preparation step can serve ten pipelines, and updating it is a versioned change.

A component YAML wraps a command with a typed interface:

$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: prep_data
version: 2
type: command
inputs:
  raw: { type: uri_folder }
outputs:
  prepared: { type: uri_folder }
code: ./src
environment: azureml:train-env:3
command: >-
  python prep.py --raw ${{inputs.raw}} --out ${{outputs.prepared}}

The interface is what makes it composable. Because inputs and outputs are typed and named, the service can draw the dependency graph, run independent branches in parallel, and skip work where the inputs have not changed and reuse is enabled for the step.

In the Python SDK you wire components together with a decorated function. The shape is below. Treat it as a sketch of the structure and check the class and decorator names against the SDK reference for your installed version before copying it, since I did not verify every import path against the current reference for this guide.

from azure.ai.ml import MLClient, load_component, Input
from azure.ai.ml.dsl import pipeline

prep = load_component(source="prep.yml")
train = load_component(source="train.yml")

@pipeline(default_compute="cpu-cluster")
def churn_pipeline(raw_data):
    prepared = prep(raw=raw_data)
    trained = train(data=prepared.outputs.prepared)
    return {"model": trained.outputs.model}

job = churn_pipeline(raw_data=Input(type="uri_folder", path="azureml:churn-training:4"))
returned = ml_client.jobs.create_or_update(job, experiment_name="churn-pipeline")

You can also write the whole pipeline as YAML and submit it with az ml job create -f pipeline.yml, which keeps the definition in git without any Python on the critical path. The trade-off: YAML is easier to review and diff, and Python is easier when the graph is generated, parameterised or has loops. Microsoft's own guidance maps to the same split: CLI for automation, SDK for complex programmatic pipelines.

Two design rules pay for themselves. First, keep components small and single-purpose: a component that prepares data, trains and evaluates cannot be reused or cached separately. Second, version components like code. A pipeline that refers to prep_data@latest changes behaviour when someone publishes a new version; a pipeline that pins prep_data:2 does not. When you share components across workspaces you will use a registry, covered below.

A pipeline can also run on a schedule. A schedule is a separate entity that points at a job or pipeline definition and a recurrence, so a nightly retrain is a file in git, not a cron entry on someone's laptop.

Hyperparameter tuning with sweep jobs

A sweep job runs many trials of one command job with different hyperparameter values and picks the best according to a metric you log. It replaces the v1 HyperDriveConfig. The structure has four parts: a command job as the trial template, a search space, a sampling algorithm and an objective with a limit on how much to spend.

The detail that trips people up is that the sweep reads a metric your code logged, so your training script must log it by the same name the sweep is configured to optimise. With MLflow, which Microsoft recommends for tracking, that is a single call:

import mlflow

mlflow.log_metric("val_auc", val_auc)

The objective then names val_auc and a goal of maximise or minimise. If the names do not match, the sweep has nothing to compare, so the first thing to check on a sweep that "finishes but picks nothing useful" is the exact metric name.

Plan the budget before you start. A sweep multiplies cost by the number of trials, and trials running in parallel multiply cluster size. The limits that matter are the maximum total trials, the maximum concurrent trials and a timeout. Set the concurrency to something your cluster's max_instances and your vCPU quota can actually serve, or trials queue and the sweep takes far longer than you expected. Quota is the usual wall.

Prefer a sweep over a loop in your own script when you want each trial to be an independent, logged job with its own compute lifecycle. Prefer a loop inside one job when each trial is seconds long and the overhead of spinning up a job per trial would dominate. I have kept the sweep API calls out of this section because the sampling and early-termination class names are best copied from the current SDK reference for your version.

Models, MLflow and registries

Registering a model from a job output was the beginner skill. The mid-level question is what you register and how you promote it.

A model asset has a name, a version and a type: mlflow_model, custom_model or triton_model. The choice affects deployment. An MLflow model carries its own signature and environment description, so it can be deployed with less configuration. A custom model is any folder of files, and you supply the scoring script and environment yourself, which gives you full control and full responsibility.

Register from a job's output rather than from a file on someone's disk, so the model carries its lineage:

az ml model create -n churn-model -v 7 -p runs:/$run_id/model --type mlflow_model

Treat versions as immutable. A version number means "these exact bytes". If you retrain, you register version 8, you do not overwrite 7. Deployments then refer to a specific version, and rollback is pointing a deployment at the previous one.

Most teams end up with separate workspaces for development, test and production. The documented pattern is to keep them in separate workspaces, or even separate subscriptions, and share assets between them through a registry. A registry is a cross-workspace store for models, environments and components. The flow is: train and validate in the development workspace, publish the approved model and the environment it needs to the registry, and have the production workspace deploy from the registry. Production never trains, and nothing is copied by hand.

Cross-tenant registry endpoint resolution was improved in SDK 1.34.0, which is a reminder that registries are an area where client versions matter. Pin azure-ai-ml, and re-test the promotion path on each minor release.

The model itself is the one part of this you cannot fix by deploying differently. If the registered artifact is wrong, every environment inherits the mistake. So record in the model's description or tags the data asset version, the code commit and the metrics, and make registration the last step of a pipeline that has already evaluated the model against a threshold.

Inside an online deployment

A managed online endpoint is a stable URL and an authentication setting. A deployment is the thing that actually serves: a model, code, an environment and an instance type. One endpoint can hold several deployments, and the endpoint splits traffic between them by percentage. Each deployment is isolated from the others, even inside one endpoint.

What runs inside the deployment container is the AzureML inference server, the azureml-inference-server-http package, which wraps your score.py. Your script has two functions. init() runs once at container start and loads the model. run(raw_data) runs per request and returns the result. This is where most deployment failures originate, because a crash in either function takes the container down. A minimal shape:

import json
import os
import joblib

model = None

def init():
    global model
    model_dir = os.getenv("AZUREML_MODEL_DIR")
    model = joblib.load(os.path.join(model_dir, "model.pkl"))

def run(raw_data):
    payload = json.loads(raw_data)
    preds = model.predict(payload["data"])
    return {"predictions": preds.tolist()}

The deployment definition ties the pieces together using the fields the schema names: endpoint_name, model, code_configuration with code and scoring_script, environment, instance_type and instance_count.

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: churn-endpoint
model: azureml:churn-model:7
code_configuration:
  code: ./onlinescoring/
  scoring_script: score.py
environment: azureml:serve-env:2
instance_type: Standard_DS3_v2
instance_count: 1

Notice the SDK equivalent uses CodeConfiguration(code=..., scoring_script=...). You may meet older examples with code_local_path. Prefer code, and do not copy the older form into new work.

The endpoint carries the authentication choice: auth_mode is key or aml_token. Key auth uses a pair of keys retrieved with az ml online-endpoint get-credentials. Token auth uses Microsoft Entra tokens and role-based access control. Prefer token auth where your callers can obtain tokens, because keys are shared secrets that you then have to rotate and keep out of logs.

The deployment loop that experienced engineers use has one rule: test locally before you touch the cloud. The same commands with --local run the container on your machine with Docker.

az ml online-endpoint create --local -n $ENDPOINT_NAME -f endpoint.yml
az ml online-deployment create --local -n blue --endpoint $ENDPOINT_NAME -f blue-deployment.yml
az ml online-endpoint invoke --local --name $ENDPOINT_NAME --request-file sample-request.json
az ml online-deployment get-logs --local -n blue --endpoint $ENDPOINT_NAME

A local deployment fails in seconds with the full traceback in the logs. A cloud deployment can take many minutes before it tells you the same thing, and the message arrives as a coded error. Every minute spent in the local loop is a saved cloud round trip.

Blue/green rollouts and traffic

The reason an endpoint holds several deployments is safe change. A model update is a new deployment, never an in-place edit of the serving one.

The pattern has four steps. Deploy green next to the existing blue without sending it traffic. Test green directly. Shift a small share of traffic to it. Then either move everything across or shift back. The CLI creates a deployment with --all-traffic to give it everything, which is right for a first deployment and wrong for an update. For an update, create it without that flag and assign traffic explicitly.

az ml online-deployment create --name green --endpoint $ENDPOINT_NAME -f green-deployment.yml
az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=90 green=10"
az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=0 green=100"
az ml online-deployment delete --name blue --endpoint $ENDPOINT_NAME --yes --no-wait

I have used the --traffic flag from the long-standing documented pattern, and traffic percentages must add up to 100, so check the exact syntax against az ml online-endpoint update -h for your installed extension. In Python the equivalent is to set endpoint.traffic = {"blue": 90, "green": 10} and call ml_client.begin_create_or_update(endpoint).

There is also a 409 trap: a conflict response means another operation is in progress on the endpoint, for example a delete during a create or update. Do not run two changes to the same endpoint in parallel from CI. Serialise them, or you will see intermittent 409 failures that disappear on retry.

Capacity, autoscale and the scoring edge

Online endpoints are where capacity problems become user-visible. The first tool is arithmetic. The troubleshooting documentation gives a formula for how many instances a target load needs:

from math import ceil

instance_count = ceil(
    target_rps * request_process_time / target_utilization / max_concurrent_requests_per_instance
)

With 20 requests per second, 10 seconds per request, 0.7 target utilisation and one concurrent request per instance, that is 286 instances. The number is meant to be a shock. It tells you that a slow model with low per-instance concurrency is a capacity problem you cannot solve by hoping, and it tells you which lever to pull: reduce the processing time, raise concurrency per instance if your code is safe for it, or accept the instance count and its quota.

Quota is the second constraint. Every VM family has a limit per region per subscription, and the hardest error to design around is OutOfQuota, which the documentation breaks into causes: CPU, cluster, disk, memory, role assignment, endpoint count, Kubernetes, and region-wide VM capacity. A deployment can fail because the whole region is short of the SKU you asked for, so ask for quota before the launch week, and have a second SKU you have already tested.

Managed online endpoints scale with Azure Monitor autoscale on top of instance_count. The exact autoscale commands were not part of the verified material for this guide, so configure rules through Studio or follow the current az monitor autoscale documentation. What matters is the policy: a minimum that keeps you above the cold-start cliff, a maximum that fits quota, and a scale-out trigger that fires before latency suffers, not after.

Now the edge. When clients complain, the HTTP status tells you where to look:

A 502 is your code, a 503 is capacity, and a 500 is the platform. That three-way split is the fastest triage you have. The documentation also lists bandwidth limits and a "Blocked by CORS policy" browser error as scoring-edge issues, and the CORS one is the usual surprise when a web page calls an endpoint directly.

Reading deployment errors

Deployment errors come with a code, and the code is the shortest path to the cause. Learn the common ones by heart and the rest by lookup.

ImageBuildFailure: the image could not be built. The build log is in the workspace's default storage. Sub-cases include container registry authorization failure (fixed with az ml workspace sync-keys), no image-build compute set in a private workspace, and a build timeout.

OutOfQuota: covered above. Read the message to see which resource ran out.

BadArgument: a family of causes, including a subscription that does not exist, an authorisation failure, a failure to download your model or user container image, and an MLflow model with an unsupported private-network configuration. On Kubernetes endpoints it also covers resource requests larger than limits.

ResourceNotReady: your container crashed while starting. This is the one you will see most, and the cause is in score.py or its dependencies: a missing import, a syntax error, an exception in init(), a readiness or liveness probe that fails, a slow start or a missing dependency. A well-known signature is TypeError: register() takes 3 positional arguments but 4 were given, which is a version mismatch between Flask 2 and the inference server package. The fix is a compatible dependency pin in your environment.

SecretsInjectionError: the endpoint's identity cannot read the secret you referenced, or the reference is malformed. Role assignments can take time to propagate, so a fresh assignment may need a few minutes.

Other codes in the same family include ResourceNotFound, WorkspaceManagedNetworkNotReady, OperationCanceled (a higher-priority operation took over, or a lock confirmation was pending) and InternalServerError. Kubernetes endpoints add a long list of their own, such as ImagePullLoopBackOff, DeploymentCrashLoopBackOff and PodUnschedulable.

The method behind all of them is the same, and it is worth stating as a procedure:

1. Read the error code first, then the message. The code names the class of problem.
2. Run az ml online-deployment get-logs -n blue --endpoint $ENDPOINT_NAME for the container's own output.
3. If it is a crash, reproduce it with --local, where the loop is seconds.
4. If it is identity or network, check role assignments and the network settings before you touch the code.
5. Change one thing, redeploy, and read the new error. A different error is progress.

Network isolation and private scoring

Many regulated employers in the Gulf and Egypt require that model endpoints are not reachable from the public internet and that data does not leave a defined boundary. Azure ML supports this, and the model has two directions to think about.

Inbound is who can call your endpoint. Set the endpoint's public_network_access to disabled, and scoring happens through the workspace's private endpoint. You can set this in the endpoint YAML, or override it at the command line with --set public_network_access=disabled. Managed endpoints share the workspace private endpoint, while deployments remain network-isolated from each other.

Outbound is what your deployment and jobs can reach. The workspace can use a managed virtual network with one of two modes: Allow internet outbound, or Allow only approved outbound, which blocks data exfiltration by default. In the strict mode you add rules for the private endpoints, fully qualified domain names and service tags your workloads genuinely need. The per-deployment egress_public_network_access setting is the legacy way to do this and is deprecated in favour of the workspace managed network.

Several documented failure patterns follow from this setup, and each has a specific fix:

• Endpoint creation fails because of v1 legacy mode. Workspaces with a private endpoint get v1_legacy_mode enabled automatically, which blocks v2 APIs and managed online endpoints. Set it to false, after talking to your network team.
• Key-based auth creation fails. The Key Vault network rule bypass must be AzureServices. Check it with az keyvault network-rule list -n <key-vault-name>.
• Deployments fail downloading the model when egress is disabled and the model sits outside the workspace storage account. Keep the model inside, or allow the route.
• The scoring endpoint cannot be resolved. A private DNS record is missing. The documentation calls for a wildcard A record in privatelink.api.azureml.ms of the form *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms.
• Private endpoints created before 24 May 2022 must be recreated before online endpoints can use them.

The DNS failure deserves attention because it looks like an outage. Everything is healthy and nothing can reach it. When a private endpoint "does not respond" from inside the network, resolve the scoring hostname from the same machine before you suspect the service.

The control-plane point from the start of this guide applies here too. Locking down scoring does not lock down az ml calls. If your CI runs on a hosted runner outside the network, decide explicitly how its management calls reach ARM.

Identity, secrets and access control

Network isolation answers where a request can come from. Identity answers who is allowed to do what.

At the workspace level, deployment needs Owner or Contributor. That is too broad for most people and most pipelines, so use custom roles for narrow jobs. A particularly important one is key retrieval: the permission Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action can be restricted through a custom role, so only the identities that must call an endpoint with keys can fetch them.

For training, use a managed identity on the compute, so jobs authenticate to storage and Key Vault without anyone's password in a file. For CI/CD, use a service principal that is granted exactly the workspace permissions the pipeline needs, and let pull requests gate what it deploys.

Secrets in deployments. SDK v1's set_secret and get_secret are gone. The v2 pattern is to reference a secret, from a workspace connection or from Key Vault, in the deployment YAML so that it is injected as an environment variable inside the container. The endpoint's identity must have read access, or deployment fails with SecretsInjectionError. This keeps the secret out of the image, out of the YAML in git and out of your logs. Your score.py reads it from os.environ like any other variable.

If you need secrets in a training job rather than a deployment, use the Key Vault client libraries directly from your code, authenticated by the compute's managed identity.

Running Azure ML from CI/CD

Everything above was done by hand. Production work needs it in a pipeline, and Azure ML is designed for that: entities are YAML files, so the repository is the source of truth.

Microsoft's recommended structure is to separate the ML code from the control-plane code. Infrastructure, meaning the workspace, its associated resources, the networking and the registries, is created with ARM or Bicep, or with Terraform if that is your team's tool (see the Terraform guide). The jobs, components, environments, data assets and endpoints are YAML plus CLI. The SDK fits where a pipeline's graph is generated programmatically, and REST is for platform integration. The MLOps v2 accelerator on GitHub collects reference implementations of this layout.

A pipeline that trains and deploys should look like this:

A GitHub Actions job (see the GitHub Actions guide) that submits a training job and fails on a bad outcome is short:

name: train
on:
  pull_request:
    paths: ["src/**", "jobs/**"]
jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install CLI extension
        run: az extension add -n ml -y
      - name: Sign in
        uses: azure/login@v2
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
      - name: Submit and wait
        run: |
          az configure --defaults group=$RG workspace=$WS
          run_id=$(az ml job create -f jobs/train.yml --query name -o tsv)
          az ml job stream -n "$run_id"
          test "$(az ml job show -n "$run_id" --query status -o tsv)" = "Completed"
        env:
          RG: ${{ vars.RESOURCE_GROUP }}
          WS: ${{ vars.WORKSPACE }}

Treat the action versions and the azure/login input shape as things to check against your organisation's standard, since they are not part of Azure ML itself. The Azure ML-specific lessons are these. Install or update the extension explicitly with az extension add -n ml -y or az extension update -n ml so the runner is not guessing. Authenticate with a service principal or federated identity, never with a person's credentials. Use az configure --defaults so every command has the right workspace. And pin the CLI extension and the SDK where you can, then re-test on each minor release, because curated environments and schemas do change.

If your own laptop sits behind a TLS-inspecting proxy, expect certificate errors from az against login.microsoftonline.com. That is local knowledge, not Azure ML documentation: trust the proxy's root certificate rather than disabling verification.

For secrets, never echo keys in CI logs. Prefer fetching with --query primaryKey -o tsv into a variable only where unavoidable, and prefer token auth so there is no key to leak.

Testing without paying for it

A change to an ML system can break in four places: the training code, the data, the scoring code and the deployment. Cover each as cheaply as you can.

Unit-test the logic that has no Azure in it. Feature engineering, metric computation and the body of run() are plain Python. Structure score.py so that run() delegates to a pure function you can call in a test with a fixture payload. The model-loading init() is thin enough to verify with a local deployment.

Contract-test the interfaces. Components declare typed inputs and outputs, so a test can load a component YAML and assert that the names and types your pipeline uses still exist. A renamed output is the commonest silent break in a pipeline.

Smoke-test the deployment locally. The --local commands from earlier are your integration test. In CI, a job that runs az ml online-endpoint create --local, invokes it with sample-request.json and asserts on the response catches most scoring failures before any cloud resource exists. It needs Docker on the runner.

Gate on the model, not on the job. A training job that completes is not a good model. Compute a metric on a held-out set, compare it to a threshold or to the current production model, and make registration conditional on that comparison.

Controlling cost

Azure ML cost is mostly compute that someone forgot about. Three habits control most of it.

Let training compute scale to zero. A cluster with min_instances of 0 releases its nodes when idle, after the idle timer, which the documentation illustrates with PT30M. Shorten the timer if your jobs are bursty. Note that the clean-up section of the training tutorial says a cluster continues to bill while it exists, so confirm against the pricing page how an idle zero-node cluster is charged in your configuration, and delete clusters you no longer use with az ml compute delete -n cpu-cluster --yes.

Prefer serverless for ad hoc work, and delete compute instances, the single-VM workstations, when you finish for the day. A forgotten compute instance is the classic surprise.

Endpoints bill for running instances whether or not they get traffic. That includes the old deployment you stopped sending traffic to. Delete idle deployments and whole endpoints that are no longer in use: az ml online-endpoint delete --name $ENDPOINT_NAME --yes --no-wait. Right-size by measurement, using the capacity formula, and review the instance type when the model changes.

Putting it all together

Here is one project that uses most of this guide. You will train a model from a versioned data asset in a pinned environment, register it, deploy it as a new deployment behind an endpoint, shift traffic and keep it cheap.

1. Set up the client. Run az extension update -n ml, az login, and az configure --defaults group=$GROUP workspace=$WORKSPACE location=$LOCATION. Pin azure-ai-ml in requirements.txt if you use the SDK.
2. Register the inputs. Create data.yml for a uri_folder asset and environment.yml for a pinned environment, and register both with az ml data create and az ml environment create.
3. Train with a job YAML that references azureml:churn-training:4 and azureml:train-env:3, logs a metric with MLflow, and runs on a cluster that scales to zero. Submit it, stream it and check the final status in a script.
4. Gate and register. Compare the metric to your threshold, and only then run az ml model create -n churn-model -v 8 -p runs:/$run_id/model --type mlflow_model, adding tags for the data version and the commit.
5. Test locally. Deploy with --local, invoke with a sample request and assert the response shape.
6. Deploy as a new deployment. Create green next to blue, send ten percent of traffic, watch error rate and latency, then move to a hundred percent.
7. Lock it down. Set public_network_access to disabled if policy requires it, inject secrets through the deployment and use token auth for your callers.
8. Control cost. Delete blue when you are confident, delete idle compute, and tag everything.
9. Automate. Put the YAML in git, run steps one to six from CI with a service principal, and gate deployment on the training job's status.

When this works end to end, you have what separates a notebook from a service: every input is versioned, every step is reproducible from git, and a bad release can be undone by moving a traffic number.

Where you are now

You can predict where a failure comes from, because you know that ARM, the workspace services, the compute, and the container each reject requests in different ways. You pin environments and data versions so runs repeat. You build pipelines from typed components, tune with sweeps, and promote models through a registry. You deploy as new deployments, shift traffic, size capacity with arithmetic and read error codes instead of guessing. You can describe private scoring, identity and secrets in the language a security reviewer expects, and you keep the bill under control.

The Senior level treats Azure ML as a platform you run for other teams: workspace topology, governance, multi-team quota, upgrade and migration policy, incident playbooks and when to choose something else. If you work with other clouds, the SageMaker and Vertex AI guides map the same ideas onto their platforms, and the Kubernetes guide helps if you attach clusters for Kubernetes endpoints.

Sources

• Azure Machine Learning documentation home
• Upgrade from v1 to v2
• What is v2
• Install and set up the CLI v2
• Train models with the CLI, SDK and REST API
• Deploy a model to an online endpoint
• Troubleshoot online endpoint deployment and scoring
• Network isolation with managed online endpoints
• azure-ai-ml changelog
• MLOps v2 accelerator
• Azure ML examples repository
• Install the Azure CLI

Part two of three. At this level an interviewer already assumes you can submit a job and create an endpoint. They want to hear how you reason when something is slow, broken, expensive or locked down, and whether you know where Azure ML's behaviour comes from. This file is organised as scenarios first, then trade-offs, then what separates strong answers from weak ones. It is cumulative: Beginner material still counts.

Where this picks up

The mental model in one screen

Azure Machine Learning is a managed service for training and serving models. You describe jobs, data, environments, models and endpoints as versioned entities, usually in YAML, and the service runs them on compute it manages. "v2" is the client API (the ml CLI extension and the azure-ai-ml SDK), not a service upgrade. Dev, test and prod are normally separate workspaces that share assets through a registry.

If you can say that in thirty seconds and then name the one thing you would check first when something fails, you are already ahead of most candidates.

Scenario: the job has been "Starting" for ten minutes

What a strong answer says. Starting means the service is allocating compute, not that your code is slow. I would check three things in order. First, the compute target: a cluster with a minimum of zero has to provision a VM, which takes a while, and serverless compute behaves the same way. Second, quota: QuotaExceeded means the subscription's vCPU quota for that VM family in that region cannot satisfy the request, and the fix is a quota increase or a smaller SKU. Third, whether the job is waiting behind others on a cluster whose max_instances is already in use. Only when the status moves to Preparing do I look at the environment, and only at Running do I look at my own code.

Follow-up an interviewer may ask. "How would you stop this hurting the team?" Keep a small warm capacity only where the wait costs more than the idle VM, which means a non-zero min_instances on a shared cluster, and keep quota requests ahead of launches, because the quota queue is slower than the VM queue.

What a weak answer does. It opens the training script. The status tells you the script has not started.

Scenario: the deployment fails with ResourceNotReady

ResourceNotReady means the container crashed while starting. The cause is in score.py or its dependencies. The usual suspects are a missing import, a syntax error, an exception in init(), a failing readiness or liveness probe, a slow start and a missing dependency. A telltale message is TypeError: register() takes 3 positional arguments but 4 were given, which indicates a mismatch between Flask 2 and the azureml-inference-server-http package.

The method matters more than the list:

1. Read the code and message together.
2. Pull the container's output with az ml online-deployment get-logs -n blue --endpoint $ENDPOINT_NAME.
3. Reproduce it locally with the same commands plus --local, which needs Docker and fails in seconds with the full traceback.
4. Fix one thing at a time and read the next error.

The point an interviewer is listening for is that you debug locally first. A candidate who iterates against the cloud, waiting for each coded error, has told you how they spend their day.

Scenario: clients see errors from the endpoint

Triage by status code, because Azure ML documents what each one means.

The one-line summary is that 502 is your code, 503 is capacity and 500 is the platform. A strong answer also mentions two non-HTTP causes: bandwidth limits and a "Blocked by CORS policy" error when a browser calls the endpoint directly.

If the interviewer asks what you would do about a 503, the best answer is not "add instances". It is: measure the real per-request time under concurrency, run the capacity formula, compare the answer to quota, then set the autoscale minimum and maximum accordingly.

Scenario: size an endpoint for a launch

The documentation gives the formula:

from math import ceil

instance_count = ceil(
    target_rps * request_process_time / target_utilization / max_concurrent_requests_per_instance
)

Walk through it with numbers. At 20 requests per second, 10 seconds per request, 0.7 target utilisation and one concurrent request per instance, the answer is 286 instances. Say plainly that this is a design smell, not a plan. The levers are the process time (a faster model, batching, a better SKU), the concurrency per instance (only if the scoring code is safe to run concurrently) and the utilisation target. After that, check quota for the VM family in the target region, because quota and regional capacity are the most common production blockers, and OutOfQuota has many sub-causes.

A strong answer adds that the formula depends on a measured process time. A weak one reuses a number from a single request on a developer laptop. Another strong point is a second tested SKU as a fallback, because a region can be short of the one you chose.

Scenario: a security team says no public endpoints

Split the problem in two directions, then name the traps in each.

Inbound. Set the endpoint's public_network_access to disabled, so scoring goes through the workspace private endpoint. It can be set in YAML or overridden with --set public_network_access=disabled. Callers inside the network then need to resolve the scoring name through private DNS. The documented requirement is a wildcard A record in privatelink.api.azureml.ms of the form *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms. Missing DNS is the classic "everything is healthy and nothing reaches it" symptom.

Outbound. Use the workspace managed virtual network. Allow internet outbound is permissive. Allow only approved outbound blocks data exfiltration and needs explicit rules for private endpoints, FQDNs and service tags. The per-deployment egress_public_network_access setting is the legacy approach.

The traps. Workspaces with a private endpoint get v1_legacy_mode enabled automatically, which blocks v2 APIs and managed online endpoints, and the fix is to set it to false after consulting the network team. Private endpoints created before 24 May 2022 need recreating. Key-based auth needs the Key Vault network rule bypass set to AzureServices. And the control plane is separate: create, update and key retrieval go to ARM over the public network, so a private workspace still needs a decision about how CI reaches ARM, through public access for management or an ARM Private Link.

A strong answer volunteers that last point. A weak one believes "private endpoint" means every call is private.

Scenario: a secret must reach the model at runtime

SDK v1's set_secret and get_secret no longer exist. The v2 answer is to reference a secret from a workspace connection or Key Vault in the deployment YAML so it is injected as an environment variable, and to give the endpoint's identity read access. If either is wrong you get SecretsInjectionError. Add that role assignments take time to propagate, so a just-created assignment may need a few minutes.

The reasoning an interviewer wants is about where the secret does not appear: not in the image, not in the conda file, not in git and not in logs. A candidate who suggests baking a key into the environment has failed the question, because an environment is an image others can pull.

For training jobs the answer shifts: use a managed identity on the compute and the Key Vault client libraries from the code.

Scenario: promote a model from dev to production

The strong answer is a registry-based flow. Train and evaluate in the dev workspace. If the evaluation clears a threshold, publish the model and its environment to a registry. The production workspace deploys from the registry. Nothing is copied by hand, and production never trains.

Then discuss rollout. Deploy the new version as a new deployment next to the current one, test it directly, shift a small share of traffic, watch error rate and latency, then complete the shift or roll back. Keep the old deployment until the new one has proven itself, and delete it afterwards because a zero-traffic deployment still bills for its instances.

Scenario: the cloud bill doubled

My checklist is by likelihood, and it starts with things nobody remembers creating.

1. Compute instances. Single-VM workstations left running.
2. Clusters. A non-zero minimum, or a long idle timer, or clusters nobody uses. Confirm how an idle zero-node cluster is billed on the pricing page, since the training documentation says a cluster continues to bill while it exists.
3. Endpoints and deployments. They bill for running instances whether or not they get traffic, including an old deployment that has had no traffic since a rollout.
4. Sweeps and pipelines. Trial count times concurrency times job length.
5. Autoscale maximum set above what quota-limited reality needs.

Then process fixes: tags for owner and project, serverless compute for ad hoc work, deletion of idle endpoints in the pipeline that retires a model, and a budget alert. A strong candidate does not quote prices from memory. Prices are on the pricing page, and they differ by region and change.

Trade-off: serverless compute versus a named cluster

Serverless means you omit compute and the service allocates resources. It removes pool management, is a good default for ad hoc and experimental jobs, and means nobody forgets to delete a cluster. A named cluster gives you a pool you control: a specific VM size reused by many jobs, predictable warm capacity if you set a minimum, and a single place to apply network and identity settings. The trade is operational effort and idle cost against control. A mature answer is "serverless by default, a named cluster where I can show I need control", with a reason attached to each exception.

Trade-off: CLI, SDK or REST

All three are first-class. The CLI with YAML is best for automation and reviewable changes, because the entity is a file in git. The SDK is best when the pipeline graph is generated, parameterised or tested in Python. REST is for integration into a platform that already speaks HTTP. Microsoft's own guidance separates ML code from control-plane code: infrastructure through ARM or Bicep, entities through YAML and the CLI. A strong answer picks per task and does not declare a winner.

Trade-off: managed online endpoint versus Kubernetes

A managed online endpoint means Azure runs the infrastructure: you choose an instance type and count, and the service handles provisioning, updates and the network integration. A Kubernetes online endpoint runs on a cluster you attach, such as AKS or Arc-enabled Kubernetes, so you own the cluster, its capacity and its failures, and get the error family that goes with it: ImagePullLoopBackOff, PodUnschedulable, KubernetesUnaccessible and so on. Choose managed unless a requirement forces your own cluster, such as running where only Kubernetes is allowed, or reusing an existing cluster's capacity. See the Kubernetes guide for the cluster side of that decision.

Scenario: your team still has v1 code

The facts an interviewer expects: CLI v1 (azure-cli-ml) support ended on 30 September 2025, and SDK v1 (azureml-core) support ended on 30 June 2026. Existing v1 workflows keep running, but may be exposed to security risks or breaking architecture changes. There is no workspace upgrade: the same workspace serves both client generations.

The plan is incremental. Inventory what uses v1. Migrate data readers first, because v1 datasets work in v2 but v2 data assets do not work in v1. Then convert training to command jobs, hyperparameter tuning to sweep jobs and ParallelRunStep to batch endpoints. Replace ACI and AKS web services with managed online endpoints. Replace secret methods with Key Vault. Do not mix v1 and v2 code in a single script, because class names such as Workspace and Model clash. Run old and new side by side where you can, compare outputs, then cut over.

Common interview questions

What is the difference between an endpoint and a deployment? An endpoint is the stable URL and authentication setting. A deployment is a model, code, environment and instance type running behind it. One endpoint holds several deployments, with traffic split by percentage.

Why deploy a new deployment instead of updating the live one? The old version keeps serving while the new one starts, rollback is a traffic change, and a crash in the new init() cannot take down the old one, since deployments are isolated.

What does --local do? It runs the endpoint and deployment on your machine in Docker, using the same commands. It exposes the full traceback of a bad score.py in seconds instead of a coded error from the cloud.

What are init() and run()? init() runs once at container start, typically loading the model. run(raw_data) runs per request. A crash in either produces a 502 at the edge, or ResourceNotReady at deployment.

When would you use a registry? When sharing models, environments or components across workspaces, most commonly to promote an approved model from development to production without copying.

How do you stop an idle cluster costing money? Set min_instances to 0 and a short idle timer, use serverless compute for ad hoc work and delete what nobody uses. Confirm how idle clusters are billed on the pricing page.

What does v1_legacy_mode do? It is enabled automatically on workspaces with a private endpoint, and it blocks v2 APIs and managed online endpoints. Setting it to false is the fix, after checking with the network team.

What strong answers cover that weak ones miss

Sixty-second self-test

Answer without looking back, then check yourself against the sections above.

1. Which two systems handle a deployment update and a scoring request, and which can be made private independently?
2. Name the three phases of a job's life and one failure that belongs to each.
3. Why is azureml:my-env@latest a reproducibility risk?
4. What does a 502 mean, and what does a 503 mean?
5. Write the capacity formula and say what is wrong with using a single laptop request time.
6. What are the two directions of network isolation, and what do the control-plane calls do?
7. How does a secret reach score.py without appearing in the image?
8. Why does a zero-traffic deployment still cost money?
9. What changed for v1 on 30 September 2025 and on 30 June 2026?
10. Which flag lets you debug a deployment on your own machine, and what does it need?

If you hesitated on more than three, reread the scenario for the topic and say the answer aloud. The Senior level covers system design, incidents and arguing a trade-off to leadership.

Part two of three. At this level your jobs run and your endpoints answer, so the problems change character. They are no longer "why will this not start" but "why is the first job always slow", "why did last week's run differ" and "why is the bill rising". Start with the error table, then the practices ordered by how much they save, then the practice cards and recipes you can paste into a runbook.

Common errors at this level

Cumulative: Beginner's errors still apply, and these are the ones that appear once things work.

The first column is what you see. The most useful habit is to read the code, such as ResourceNotReady, before the message, because the code names the class of problem and the message supplies detail.

Practices that pay off most, in order of leverage

These are ranked by how many problems each one removes, not by how interesting it is.

1. Pin everything a job depends on. An explicit environment version, an explicit data asset version, and an SDK and CLI extension you chose. This single habit removes the largest class of "it behaved differently" incidents. @latest is for experiments you intend to discard.

2. Debug locally before the cloud. A --local deployment fails in seconds with the full traceback. The same mistake in the cloud arrives minutes later as a coded error. Make the local deploy-and-invoke sequence part of your pull-request checks, which needs Docker on the runner.

3. Roll out as a new deployment. Create the new version next to the old, shift a small share of traffic, watch, then finish or roll back. Rollback becomes a traffic number instead of a redeploy.

4. Keep entities in git as YAML. The repository, not someone's memory of the portal, is the source of truth. Changes to compute, environments, data assets and endpoints go through pull requests, and a service principal applies them.

5. Let compute scale to zero and delete what is idle. Most cost is forgotten compute: instances, clusters nobody uses and old deployments still billing for instances.

6. Size from measurement. Measure the real per-request time under concurrency, apply the capacity formula and compare the result with quota before launch, not during it.

7. Use identity, not keys, where you can. Managed identity for training, Key Vault references injected into deployments, Entra token auth for callers you control.

8. Separate environments with workspaces and a registry. Develop, test and production live in different workspaces, and approved assets move through a registry.

Practice cards

Each card is a small exercise against a development workspace. Do them in order, since each reuses the previous one.

Operational recipes

Pre-flight for a CLI session

Before any session, confirm you are pointed at the workspace you think you are. Pointing at production by accident is the classic mistake.

az account show --query "{sub:name, user:user.name}" -o table
az configure -l -o table
az extension list --query "[?name=='ml'].version" -o tsv

If the extension is missing or old, install or update it:

az extension add -n ml -y
az extension update -n ml

Submit a job and fail the script if it fails

run_id=$(az ml job create -f job.yml --query name -o tsv)
az ml job stream -n "$run_id"
status=$(az ml job show -n "$run_id" --query status -o tsv)
test "$status" = "Completed" || { echo "job $run_id ended as $status"; exit 1; }

The stream command returns at a terminal state, which is not always Completed, so the status check is what turns a failed job into a failed step.

Local deploy-and-invoke smoke test

az ml online-endpoint create --local -n $ENDPOINT_NAME -f endpoint.yml
az ml online-deployment create --local -n blue --endpoint $ENDPOINT_NAME -f blue-deployment.yml
az ml online-endpoint invoke --local --name $ENDPOINT_NAME --request-file sample-request.json
az ml online-deployment get-logs --local -n blue --endpoint $ENDPOINT_NAME

Assert on the response shape in your test script, not just the exit code.

Cloud deployment with registered assets

az ml model create -n my-model -v 1 -f model.yml
az ml environment create -n my-env -v 1 -f environment.yml
az ml online-endpoint create --name $ENDPOINT_NAME -f endpoint.yml
az ml online-deployment create --name blue --endpoint $ENDPOINT_NAME -f blue-deployment-with-registered-assets.yml --all-traffic
az ml online-endpoint show -n $ENDPOINT_NAME

Use --all-traffic only for the first deployment. For later versions, create the deployment without it and assign traffic on purpose.

Call an endpoint with a key

ENDPOINT_KEY=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -o tsv --query primaryKey)
SCORING_URI=$(az ml online-endpoint show -n $ENDPOINT_NAME -o tsv --query scoring_uri)
curl -sS -X POST "$SCORING_URI" \
  -H "Authorization: Bearer $ENDPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d @sample-request.json

Never print the key in CI output. Where callers can obtain Entra tokens, prefer aml_token and avoid keys altogether.

Make an endpoint private

az ml online-endpoint update -n $ENDPOINT_NAME --set public_network_access=disabled

Then verify from inside the network that the scoring hostname resolves to a private address, and from outside that the call now fails. If the name does not resolve, look for the missing wildcard A record before anything else.

Tear down an endpoint you no longer need

az ml online-endpoint delete --name $ENDPOINT_NAME --yes --no-wait
az ml compute delete -n cpu-cluster --yes

An endpoint bills for its running instances whether or not it gets traffic, so teardown belongs in the pipeline that retires a model.

Clean a laptop of CLI v1

az extension list
az extension remove -n azure-cli-ml
az extension remove -n ml
az extension add -n ml
az ml -h

CLI v1 support ended on 30 September 2025. Remove it, clean any old ml extension and add the current one.

Reading the signals at each stage

Most debugging time is lost looking in the right place at the wrong stage. Azure ML failures cluster by stage, and each stage has its own evidence.

Before the job runs, the evidence is the submission itself. An az ml job create that fails immediately is almost always a YAML or reference problem: a wrong file path, a missing asset name or version, or an identity without permission. Run it from the folder that contains the YAML, and check that every azureml:name:version reference exists with az ml environment show, az ml data show or az ml model show. A reference to a deleted version fails here, not later.

While the job is Starting or Preparing, the evidence is compute and image state. Look at quota, cluster scaling and whether this is the first use of a changed environment. A job that is slow only the first time is paying for an image build, which is normal. A job that is slow every time with a cluster at zero is paying for node provisioning, and the question becomes whether a warm minimum is worth its idle cost.

While the job is Running, the evidence is your own logs, streamed with az ml job stream, and the metrics you logged with MLflow. If the logs show a long gap before the first line of your code, the time went into reading inputs. If it is a large dataset, check how it is being provided to the container before blaming the algorithm.

At deployment, the evidence is the error code and get-logs. The code tells you the class, such as quota, image, argument, readiness or secrets, and the logs tell you what the container said. Reproduce locally when the class is readiness.

At scoring time, the evidence is the HTTP status. A 502 sends you to your code, a 503 to capacity and a 500 to the platform. Compare the time run() takes with the timeout before you tune anything else.

Over weeks, the evidence is cost and drift. A bill that creeps up is almost always an idle resource. A model that slowly gets worse will not show up in any of the above, and needs a sample of inputs and, later, outcomes.

The discipline is to name the stage before opening any file. When you say "this is a Preparing problem", you have already excluded your training code, your scoring code and the network.

Reviewing a pull request that touches Azure ML

A short checklist makes reviews faster and more consistent. These are the questions to ask of a change, roughly in order of how often they catch something.

Does every environment and data path in the YAML name an explicit version? A new @latest is the most common regression, and it is easy to miss in a diff.

Does a new deployment come with a traffic plan? A deployment created with --all-traffic replaces the serving version at once. For an update, the plan should say how much traffic moves first, what is watched, and how to go back.

Does the change add or enlarge compute? Check max_instances, instance counts and SKUs against quota, and ask who deletes the resource when it is no longer needed.

Does anything sensitive appear in a definition? A key in a conda file, a connection string in a job YAML or a token in a script is a leak, even in a private repository. Secrets should be references that resolve at runtime.

Does the scoring script have a test? At minimum the body of run() should be exercisable with a fixture payload, and the local deploy-and-invoke smoke test should run in CI.

Does the change touch network settings? public_network_access, managed virtual network rules and private DNS each affect who can reach what. Ask for the intended before and after, not just the diff.

Does the change pin or change a client version? A bump of azure-ai-ml or the CLI extension deserves its own pull request, because schemas and curated environments change, and a failure should be attributable to the bump.

Finally, ask what the rollback is. If the answer is "redeploy the old one", the change is riskier than one whose rollback is a traffic number.

Habits that prevent the next incident

Write the runbook before you need it. For each endpoint, a half page that says where logs are, how to roll back, who owns it and which alerts exist. The three-way triage of 502 for code, 503 for capacity and 500 for the platform belongs in it.

Treat environment changes as deployments. The first job after a change is slow and may fail on the build. Put environment changes in their own pull request so the failure is attributable.

Request quota early. Quota is per VM family, per region, per subscription, and an increase takes longer than a deployment. Ask before launch week, and test a second SKU in advance because regions run short.

Serialise changes to an endpoint. A 409 is not flaky infrastructure. It means two operations overlapped. One pipeline stage per endpoint at a time.

Keep secrets out of definitions. Nothing sensitive in a conda file, an environment YAML or a job YAML. Inject at runtime through the deployment and the endpoint's identity, and rotate anything that was ever committed.

Log the metric you tune on, by one stable name. A sweep compares the logged metric. If the name is wrong or reused, the sweep ends with nothing useful to pick.

Prefer small, versioned components. A single step that prepares, trains and evaluates cannot be reused or cached separately. Version components like code, and pin versions in pipelines so they do not change underneath you.

Mind the local environment. If your laptop sits behind a TLS-inspecting proxy, az may fail with CERTIFICATE_VERIFY_FAILED against login.microsoftonline.com. That is local knowledge rather than Azure ML documentation, and the fix is to trust the proxy's root certificate, not to disable verification.

Quick checklist

• The environment and data asset versions are explicit in the job YAML.
• The job's tags record the data version and the commit.
• score.py has been run with --local before any cloud deployment.
• The new model is a new deployment, with a planned traffic shift and a rollback.
• Capacity was computed from a measured request time and compared with quota.
• A second tested SKU exists for the case where the region is short.
• Secrets reach the container through Key Vault and the endpoint identity.
• Public network access matches the policy, and private DNS has been verified.
• Idle compute, endpoints and old deployments have an owner or have been deleted.
• The SDK and CLI extension versions are pinned, with a re-test planned for each minor release.

Next, the Senior level treats the service as a platform for a team: workspace topology, governance, multi-team quota, incident playbooks and migration policy. Before that, if you want the neighbouring tools, see the MLflow guide for tracking concepts and the Docker guide for the images your environments are built on.

This is part three of three. It takes every topic from Beginner and Mid one level further and adds the work you own once Azure Machine Learning is a platform that other teams build on, rather than a service you happen to use.

Up to this point the questions were about making jobs run and endpoints answer. From here they change: what happens to forty teams when one subscription runs out of GPU quota, who can read an endpoint key, what a private workspace actually keeps private, how a model gets from a dev workspace to production without anyone copying files by hand, and what you do when the thing you built everything on announces an end-of-support date.

Where this picks up

I start with the architecture, because every failure mode in this guide is a statement about where a boundary sits.

Architecture: the boundaries that matter

Azure Machine Learning is a managed service, so there is no server of yours to size or patch. That is a real advantage, and it changes what "owning the platform" means. You do not operate a scheduler or a metadata database. You operate a set of boundaries: subscriptions, workspaces, networks, identities and quotas. Failures in a managed service almost always arrive as a boundary being hit, and a platform owner who cannot name the boundaries debugs by trial and error.

The first boundary is the workspace, the resource of type Microsoft.MachineLearningServices/workspaces. It holds assets, jobs, endpoints and compute definitions, and it is the unit of RBAC and of network configuration. A workspace is associated with a storage account, a key vault, a container registry and an Application Insights instance. Those associated resources may live in a different resource group, but they must share the workspace's subscription and tenant. That constraint matters the day someone proposes a central container registry or key vault in a different subscription for all workspaces to share: the documentation requires the same subscription and tenant, so you want to know that at design time rather than at creation time.

The second boundary is the subscription. Quota is granted per SKU, per region and per subscription, so a subscription is also a blast-radius boundary for capacity. Two workspaces in one subscription compete for the same vCPU quota whether or not they belong to the same team.

The third is the split between the control plane and the data plane, and it is the one most often misunderstood. The control plane is Azure Resource Manager: creating a workspace, submitting a job, creating or updating an endpoint, fetching an endpoint's keys. The data plane is what a client does against a running endpoint: sending a scoring request. The CLI v2 sends your YAML and metadata to ARM over HTTPS with TLS 1.2, which means control-plane calls go through the public ARM endpoint even when scoring is entirely private. A private workspace that keeps the scoring URL off the internet has not stopped a developer's laptop from talking to ARM, and your design has to say so out loud.

The fourth boundary is the identity. Every action is performed by a user, a service principal or a managed identity, and every one of those has an RBAC assignment at some scope. Compute uses a managed identity to reach data. An endpoint has its own identity that reads secrets. When something fails with an authorization message, the question is always the same: which identity was acting, and what role did it hold at which scope?

Finally there is the client surface, which is the only thing with a version. The service has none. What you pin and upgrade is the Python SDK azure-ai-ml (the current release in the changelog is 1.35.0, dated 2026-09-08), the ml extension of the Azure CLI, and the REST API version, which the current training documentation shows as 2025-09-01. Because the service moves underneath you without a version bump, the discipline of a platform owner is to treat each of these three as a dependency you test, not a given.

Failure modes: reading the error as a boundary

The error catalogue in the Microsoft troubleshooting page is long, and at the senior level the useful skill is to compress it. Almost every deployment error is one of five things: capacity, identity, network, code, or concurrency. If you can classify an error in a few seconds you know whose problem it is, and that decides whether you fix it or hand it back with a precise explanation.

Capacity errors are OutOfQuota and QuotaExceeded. The documentation lists several causes under OutOfQuota: CPU quota, cluster quota, disk, memory, role assignment limits, endpoint limits, Kubernetes limits, and region-wide VM capacity. The last of these is the one platform teams forget. Having quota on paper does not guarantee that the region has machines at the moment you ask, so a quota increase can be approved and the deployment can still fail. Your playbook for a GPU launch therefore includes a second region, decided in advance.

Identity errors include SecretsInjectionError, which fires when the endpoint's identity lacks RBAC on a secret or when the secret reference is written in the wrong format, and the BadArgument family that includes an authorization or role-assignment failure at startup. A subtle cause: role assignments take time to propagate. A deployment created seconds after the assignment can fail, and the same deployment retried a few minutes later succeeds. The platform lesson is to create the role assignment in the infrastructure pipeline, wait for it, and only then create the deployment, rather than letting a developer discover the race.

Network errors are the most expensive in engineer hours. When endpoint creation fails because the workspace is in v1_legacy_mode, nothing in the deployment YAML is wrong. When key-based authentication fails to create against a private workspace, the cause the documentation gives is the Key Vault network rule: its bypass setting must be AzureServices, and az keyvault network-rule list -n <key-vault-name> shows what it is. When a scoring endpoint cannot be resolved from inside the network, the documented cause is a missing wildcard record in the privatelink.api.azureml.ms private DNS zone, of the form *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms. A deployment that fails to download an image with egress disabled usually points at a model that sits outside the workspace's own storage account.

Code errors surface as ResourceNotReady: the container crashed while running score.py. The causes are ordinary: a missing import, a syntax error, an init() that throws, a readiness or liveness probe that is too aggressive for a slow model load, a missing dependency. One documented message deserves memorizing because it appears after dependency drift: TypeError: register() takes 3 positional arguments but 4 were given, which is a Flask 2 versus azureml-inference-server-http mismatch. It tells you the environment changed, not the code. az ml online-deployment get-logs shows it, and a local deployment with --local reproduces it on your own machine if Docker is available.

Concurrency errors are the HTTP 409 on a management call, meaning another operation is in progress on that endpoint, and OperationCanceled, which the documentation attributes to a higher-priority operation or a lock. These are caused by automation. Two pipelines updating the same endpoint, or a cleanup job deleting an endpoint while a deployment is being created, will produce them. The fix is not retrying harder. It is making one pipeline the only writer to each endpoint.

At scoring time, the status codes carry the same classification. A 502 means an exception or crash in run() or init() of your script. A 503 means a spike in requests beyond what current replicas can absorb. A 504 is a timeout, and the documentation's figure for the Kubernetes case is five seconds, so do not assume the managed default without checking. A 500 is an Azure-side infrastructure failure, and the right response is an incident with Microsoft rather than a code change. Teach your support rota that 502 belongs to the model owner and 500 belongs to you and the cloud provider.

Scaling: which component breaks first

For Azure ML the honest answer to "what breaks first" is rarely the service. It is, in order: quota, regional capacity, image build and cold start time, and only then the application itself.

Quota comes first because it is a hard gate set outside your platform. It is granted per SKU family, per region, per subscription. A platform with a hundred users who each launch a cluster of eight nodes does not need a clever scheduler, it needs a quota policy. Decide which SKUs are allowed, and give each team a ceiling through the compute cluster's max_instances, so one team's experiment cannot starve everybody's production deployment. Note that this is a convention you enforce in code review and policy, because quota itself is not divided among teams for you.

Compute clusters scale from min_instances of zero up to max_instances, and release nodes after the idle timeout you set. The documentation example uses nodeIdleTimeBeforeScaleDown of PT30M. A longer timeout trades money for fewer cold starts in a busy team's day, and a platform default is usually better than letting every team pick. Serverless compute, where a job simply omits compute, removes cluster management for ad hoc work and is the sensible default for beginners on your platform. It does not remove quota: a serverless job can still fail for lack of cores.

Online endpoints scale in two ways: instance_count on the deployment, and Azure Monitor autoscale on top. The documented capacity formula is worth copying into your design template:

from math import ceil

# from the Azure ML troubleshooting page
instance_count = ceil(
    target_rps * request_process_time / target_utilization / max_concurrent_requests_per_instance
)

# 20 requests per second, 10 seconds each, 70 percent utilization, one concurrent request:
print(ceil(20 * 10 / 0.7 / 1))   # 286

The result for the documentation's example, 286 instances, is a useful shock. A model that takes ten seconds to answer and can handle one request at a time cannot serve twenty requests per second without a very large fleet, and no tuning of autoscale rules rescues a design whose arithmetic is wrong. Senior review means asking for the four inputs before approving the deployment: expected requests per second, per-request time, headroom you are willing to run at, and how many requests one instance can really handle at once. Then multiply by the SKU's cost and check the quota.

Spikes are the second half. The documentation says a 503 appears with large request spikes and recommends lowering autoscale_target_utilization (in the Kubernetes case it names) or raising minimum replicas. The mechanism is that autoscale reacts after load arrives, and a new instance takes time to pull an image and run init(). If your model loads in ninety seconds, a spike lasting sixty seconds is over before help arrives. For traffic you can predict, schedule a higher minimum rather than relying on reaction. For traffic you cannot, the only protection is headroom.

Image build and start time is the quiet limit. A deployment must build or pull an environment image, then start a container, then run init(). The ImageBuildFailure error lists a build timeout as one cause. Large environments, GPU images and models loaded from remote storage all lengthen the cycle that determines how fast you can scale and how fast you can recover. A platform owner reduces it by baking dependencies into registered environments rather than installing at start, and by measuring time from deployment creation to first successful scoring request as a metric you publish.

Security: the trust model

A trust model is the answer to "who can do what, and what does each of them hold that someone else could use". For Azure ML, write it down as four lists: the humans, the automation, the workloads, and the secrets.

Humans hold RBAC roles at workspace scope. The documentation's baseline is that Owner or Contributor on the workspace is what lets someone deploy. That is far more than a data scientist needs for daily work, and it is the reason a platform needs custom roles. The sharpest example is endpoint credentials. Key-based authentication means az ml online-endpoint get-credentials returns a key, and anyone who can run it can call the endpoint as anyone. The documentation points out that you can restrict this through the action Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action in a custom role. Create a role for developers that can create and update jobs and deployments but not fetch keys, and give the key-fetching role to the pipeline identity only.

Automation is a service principal or a federated identity used by your CI/CD system. It is the most powerful identity in your estate, since it writes everything, so it is the one to scope tightly: one per environment, assigned at the workspace of that environment only, never at subscription scope. Production's pipeline identity should not be able to write to the dev workspace, and the dev pipeline should have no rights over production. This single separation prevents a large class of "I ran the wrong command against the wrong workspace" incidents.

Workloads run as managed identities. The documentation recommends managed identity on compute for training jobs. A job that reads from a storage account should do so through the cluster's identity with a narrow role on that account, not through an account key stored somewhere. The endpoint likewise has its own identity, and that identity is what reads secrets at deployment time.

Authentication to the endpoint itself has two modes in the YAML: auth_mode: key and auth_mode: aml_token. Keys are simple and are shared secrets, which means rotating them is a coordination exercise with every caller. Token authentication ties access to an Entra identity and RBAC, so access is revocable per caller and auditable. My recommendation is token authentication for anything called by a service you control, and keys only for callers that cannot obtain a token. The documentation does not state a "recommended default" in the pages I used, so this is a judgment, not a rule from Microsoft.

Secrets deserve their own paragraph because the older mechanism is gone. In SDK v1, set_secret and get_secret stored values in the workspace's key vault. They do not exist in v2, and the migration guide tells you to use the Key Vault client libraries directly. For deployments, the supported pattern is that the deployment YAML references a secret in a workspace connection or in Key Vault and it appears in the container as an environment variable. The endpoint identity needs read permission on that secret. If the reference is malformed or the permission is missing, you get SecretsInjectionError, and if the role assignment is new, you may get it for a few minutes anyway.

The effect of this design is that the secret never appears in your YAML, in the job definition, or in a pull request. That is the property you want to defend in review: the repository contains a reference, never a value. A reviewer who sees a literal key, a connection string with a password, or a SAS token in any YAML file should block the change and trigger rotation, because git history keeps it forever. If you are already running HashiCorp Vault for the rest of your estate, the Vault guide covers its trust model, and the question for Azure ML is where the boundary between the two systems sits. A simple answer is that Azure Key Vault serves workloads that run inside Azure ML, and anything that must be shared with systems outside Azure is a separate decision.

Network isolation: inbound, outbound and the exceptions

Network design for Azure ML is two separate questions, and conflating them is the usual cause of an expensive redesign.

Inbound asks who can reach the scoring URL. For managed online endpoints you set public_network_access on the endpoint to disabled, and callers then reach it through the workspace's private endpoint. Managed endpoints share the workspace private endpoint, so you do not create one per endpoint, and deployments remain network-isolated from each other. You can set the flag in the YAML or override it at the command line with --set public_network_access=disabled, which is handy in a pipeline that applies one environment's difference without a second YAML file.

Outbound asks where a deployment or a job may send traffic. The workspace managed virtual network supports two modes, Allow internet outbound and Allow only approved outbound. The second is the one that addresses data exfiltration: nothing leaves except through private endpoint, FQDN or service tag rules you added. This is a control that security teams like and developers find irritating, since every package index and model hub they relied on becomes an explicit rule. Plan for that by mirroring packages into storage or a registry you control and baking dependencies into environments, so that outbound rules stay short and reviewable.

An older setting still appears in the documentation: egress_public_network_access on the deployment. It is the legacy per-deployment mechanism, replaced by the workspace managed virtual network, so do not design new systems around it.

Three network facts cause most private-workspace incidents.

First, v1_legacy_mode. Workspaces with a private endpoint have it enabled automatically in some situations, and it blocks the v2 APIs and managed online endpoints. The fix is to set it to false, and the documentation advises checking with the network team first, because the flag exists for a reason. Do not flip it on a production workspace as a reflex to unblock a developer.

Second, old private endpoints. Private endpoints created before 24 May 2022 must be recreated before online endpoints can use them. If your estate has workspaces that old, inventory them early.

Third, DNS. Private connectivity is a DNS problem as often as a routing problem. The scoring endpoint needs a wildcard A record in the privatelink.api.azureml.ms zone for *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms, and every network that needs to call the endpoint must resolve through a zone that has it. In a hub-and-spoke design where DNS is centralized, forgetting to link or forward this zone produces the symptom "works from the workspace's network, fails from the application's network", which looks like a firewall problem and is not.

A short diagnostic order saves hours: resolve the name from the caller's network first, then check the route, then check the endpoint's public-access flag, then check authentication. Do not start at the model.

Supply chain: images, environments and what runs your code

Every job and deployment runs inside an environment, which is a Docker image plus a conda specification. Whoever controls the environment controls what code executes with the workload's identity. That makes environments the supply-chain surface of Azure ML, and it deserves the same discipline you would give container images in any other platform. The Docker guide covers image hardening in depth, and the Trivy guide covers scanning. Here is how they apply.

Curated environments are the ones named AzureML-.... They are convenient, and the documentation is candid that their names rotate: EnvironmentNotFound is listed as a symptom of a curated environment that was retired. The two documentation examples use different curated names for the same task. For a platform, that is the argument against @latest in production. Build your own environments from a base image you choose, register them with explicit versions, and promote them like code.

Custom environments build in the workspace's container registry. The ImageBuildFailure error has sub-cases that matter to a platform owner: a registry authorization failure, which the documentation says to fix with az ml workspace sync-keys; an image build compute that is not set for a private workspace; and build timeouts. Another documented trap: jobs fail at startup when the registry uses a custom domain name label. The documentation's fix is to use <registry-name>.azurecr.io and set the domain name label scope to Unsecure. I mention it so that when a security team proposes a vanity registry name, you know the cost.

Pinning is the practical control. Pin the azure-ai-ml version in your pipeline's requirements file. Pin the base image by a version you scan. Register environments with explicit version numbers and reference them as azureml:name:version, not @latest. Note that ARM itself has no @latest for environments, so a pipeline that works through the CLI and one that works through raw REST will resolve differently, which is a source of confusing differences.

Scanning belongs in the pipeline that builds the environment, before registration, so that an environment with a critical vulnerability never becomes something a team can select. I will not claim a built-in Azure ML feature for this because I have not verified one in the pages I used: treat scanning as something your pipeline does with Trivy or the registry's own tooling.

The inference server is part of the supply chain. Deployments run azureml-inference-server-http around your score.py, and the documented TypeError: register() takes 3 positional arguments but 4 were given shows what happens when its dependencies drift. A pinned, tested environment avoids that class of incident altogether.

Finally, there is a subtle point about data: the SDK changelog for 1.33.0 added path traversal protection to unzip operations. It is the kind of fix that reminds you that client libraries are a supply chain as well. Subscribe to the changelog, and read the entry before each upgrade rather than after a failure.

Multi-tenancy: workspaces, registries and promotion

When many teams share Azure ML, the design question is how much each of them shares and where the walls go. The documentation gives you the practical wall: the workspace. RBAC is assigned per workspace, network configuration is per workspace, and compute, endpoints and assets live inside one. Everything below follows from that.

There are three common topologies, and each is right in some situation.

One workspace per environment, shared by teams. Dev, test and production workspaces, with teams separated by naming conventions and compute quotas. This is cheap and easy to operate, and it is weak on isolation: any user with a role on the workspace sees every team's assets, and one team's runaway job competes for the same quota. It suits a small organization with high mutual trust.

One workspace per team per environment. Each team gets its own dev and production workspaces. RBAC is simple, cost attribution is obvious, and one team's mistake stays inside its workspace. The price is duplication of network setup, associated resources and policies, which is only tolerable if you create workspaces from code. A platform that offers this topology without Terraform or Bicep behind it will collapse under its own onboarding queue.

One workspace per environment per sensitivity tier. Teams share a workspace when their data classification matches. This is how regulated workloads in the Gulf and Egypt often end up: the workspace in a specific region and with a managed network set to Allow only approved outbound holds the sensitive models, and everything else lives somewhere freer.

The documentation's guidance on promotion is to keep dev, test and production in separate workspaces, or even separate subscriptions, and to share assets between them through registries. A registry holds models, environments and components that several workspaces can consume. The practical effect is that promotion becomes a reference, not a copy: a model is registered once, the production workspace deploys it from the registry, and its lineage is intact. Compare this with the pattern of downloading a model artifact in one workspace and re-uploading it in another, which breaks lineage and invites "which file did we ship?" arguments.

Cross-tenant use is possible, and the changelog notes that SDK 1.34.0 improved cross-tenant registry endpoint resolution. The documentation I used does not describe cross-tenant as a headline product feature, so treat any design that depends on it as something to prototype before promising it to a customer or a business unit.

Quota fairness is the part that surprises platform teams. The subscription holds the quota, so in a shared subscription one team can consume what another needs. The mitigations are structural: separate subscriptions for production and for experimentation, a max_instances convention for each team's clusters enforced in review, and a policy that GPU SKUs are available in production workspaces only. Azure Policy can enforce some of this, and I would pair any such policy with a documented exception path, because a blocked data scientist who cannot get an exception will find a workaround that is worse than the exception.

Chargeback needs resource tags. Tag workspaces, compute and endpoints with the owning team and cost centre at creation, from the infrastructure code, so the data exists when finance asks. Retrofitting tags is painful because compute created by users in the studio will not carry them.

Governance: YAML in git, pull requests and the single writer

Azure ML is designed to be driven from files. Every entity (job, data asset, environment, component, model, endpoint, deployment, schedule) has a YAML form with a published schema under https://azuremlschemas.azureedge.net/latest/, and the documentation recommends keeping those files in source control and applying them through the CLI in CI/CD. For a platform owner this is the governance mechanism. It is how you get review, history, rollback and an audit trail without building any of them.

The structure that works is a split in two repositories, or two clearly separate directories with separate owners. Infrastructure code, written in Terraform or Bicep (see the Terraform guide), creates workspaces, networks, private endpoints, registries, role assignments and compute definitions. The documentation says the same thing: separate ML code from control-plane code, and create infrastructure with ARM or Bicep. ML code and entity YAML lives with each team: training code, pipeline definitions, deployment YAML. The platform team owns the first and reviews changes to the second where they touch security-relevant fields.

Which fields are security-relevant? A short list is enough to configure code owners on:

• public_network_access and egress_public_network_access in endpoint and deployment YAML
• auth_mode
• any environment_variables that reference secrets, and the form of those references
• instance_type and instance_count, because they drive cost and quota
• traffic percentages on production endpoints
• anything under a network or identity configuration

Use code owners or the equivalent in your host to require platform-team approval whenever those lines change, and let everything else flow through ordinary team review. The GitHub Actions guide covers how to build the pipeline side, with environments, required reviewers and OIDC federation to Azure in place of stored credentials.

The principle that prevents most production incidents is the single writer. Production endpoints are changed by exactly one pipeline, running as exactly one identity, from the main branch. Humans do not run az ml online-deployment update against production from a laptop. This does three things. It makes every change reviewable. It removes the 409 and OperationCanceled collisions that two concurrent writers produce. And it means the audit question "who changed this?" has the answer "this commit".

Some practical points on how to run the pipeline:

• Run az ml commands with explicit --workspace-name and --resource-group arguments, or with az configure --defaults set inside the job, never relying on a developer's local defaults. A pipeline that depends on defaults can hit the wrong workspace.
• Export the existing state to a file with az ml <entity> show --output yaml when you adopt a resource that was created in the studio, then delete the system-generated properties before committing it.
• Use --set to apply a per-environment difference to a shared YAML, for example --set public_network_access=disabled, in preference to maintaining three nearly identical files that drift.
• Roll model updates as a new deployment and shift traffic, never as an in-place edit of the live one. In the CLI, --all-traffic gives a new deployment everything, so in production prefer an explicit, staged traffic change that a reviewer can read.

Cost: designing so the bill is predictable

Cost on a managed platform is a design property. You cannot tune it afterwards with the same leverage, because the structure decides it: how many always-on endpoints, how large their instances, how long clusters idle.

Three levers do most of the work, and all three appear in the documentation's own guidance.

Compute clusters scale to zero when min_instances is zero, and release nodes after the idle timeout. The fact sheet I worked from notes that the documentation's clean-up section says a cluster "continues to bill while it exists, even with zero nodes running"; that contradicts the usual understanding that zero nodes means no virtual machine charge, and I could not reconcile the two without the pricing page, so check your own bill before you build a chargeback model around either reading. Whichever is true, delete clusters nobody uses, because a cluster that exists and is forgotten is a standing risk.

Serverless compute is the second lever. For ad hoc and exploratory jobs, omitting compute makes Azure allocate it, which removes the cluster inventory you would otherwise have to govern. It is a sensible default for exploratory work, and a policy of "production training runs on named clusters, everything else on serverless" is easy to explain.

Endpoints bill for running instances whether or not they receive traffic, as far as I know. I have not confirmed the exact billing statement on the pricing page, so verify it before relying on it. The consequence for a platform is that an endpoint nobody calls is the most expensive kind of forgotten resource, because it never shuts itself down. The control is an inventory with an owner and a last-called date. Azure Monitor metrics give you request counts per endpoint, and a monthly job that lists endpoints with no recent traffic and messages the owner removes more cost than any SKU negotiation.

Other points worth building into design review:

• Right-size by the formula. The capacity formula in the scaling section gives a defensible instance count. A deployment approved with a guessed count of ten is as likely to be over-provisioned as under.
• GPU is the line item to watch. Few SKUs cost as much per hour, and quota for them is scarcest. Reserve GPU for the workloads that need it and put a review step in front of any GPU request.
• Spot and low-priority capacity. The REST example in the documentation shows vmPriority set to Dedicated, which implies another option exists, and I have not verified the current name or behaviour. Investigate it for interruptible training before assuming savings.

Finally, think about cost per prediction, not just cost per month. If a deployment costs a fixed amount per month and serves a growing number of requests, the unit cost falls with traffic. If the model is large and the traffic small, the unit cost is huge, and the right answer may be a batch endpoint, which scores in bulk asynchronously and releases compute when it finishes. Choosing between online and batch is the single biggest cost decision for many teams, and it is usually made by default rather than on purpose.

Upgrades and migrations: owning the v1 deadline

The most important migration in this platform's life is already past its dates, and it is yours to finish if your estate still uses the old client libraries.

The facts, from the migration guide. CLI v1, the azure-cli-ml extension, reached end of support on 30 September 2025. SDK v1, the azureml-core family, was deprecated on 31 March 2025, and its support ended on 30 June 2026. Today is after both dates. Microsoft says existing v1 workflows continue to operate, and warns that they may be exposed to security risks or breaking architectural changes. New features ship only in v2. "v1" and "v2" describe the client API, not the service: there is no workspace upgrade, and one workspace can serve both. That is the good news, since it means migration is a code change, not an infrastructure event.

The mapping is the content of your migration plan:

A v1 dataset works in v2, but a v2 data asset does not work in v1, so migrate in a direction that never leaves a producer on v2 with a consumer on v1. FileDataset maps to uri_folder and TabularDataset maps to mltable. Never mix v1 and v2 code in one script: class names such as Workspace and Model clash across the namespaces.

A workable plan has four steps. Inventory: search every repository for azureml.core, azureml-core and azure-cli-ml, and list every deployed v1 web service. Classify: sort workloads by business criticality and by how much v1-only functionality they use, such as database datastores or ParallelRunStep. Migrate in leaf-first order: consumers before producers, so no v2 asset is read by v1 code. Verify and delete: after each workload moves, remove the old service so that nobody redeploys it.

Remember the private-workspace trap: a workspace with a private endpoint can have v1_legacy_mode enabled, and that blocks the v2 APIs. A migration that fails at the first v2 call in such a workspace is probably not a code problem.

Routine upgrades are smaller but never stop. The SDK releases frequently: 1.32.0 on 2026-03-17, 1.33.0 on 2026-05-11, 1.34.0 on 2026-06-11, 1.34.1 on 2026-07-15 and 1.35.0 on 2026-09-08. Entries include fixes to serialization and to immutable-field updates on deployment templates. The 1.34.1 changelog lists changes under breaking changes, with datastore serialization fixed and batch endpoint defaults now serialized as camelCase; I could not confirm the exact wording, so read that entry yourself if you used either feature. The policy that follows from this cadence is: pin the version, upgrade on a schedule in a test pipeline, read the changelog entry, and only then promote the new pin. Curated environments and schemas change too, which is another reason to test rather than assume.

Observability and incident playbooks

Every job logs to MLflow, which the documentation recommends for tracking, and endpoints expose metrics through Azure Monitor with Application Insights as the workspace-associated sink. Use az ml job stream for jobs and az ml online-deployment get-logs for deployments. Alert on the signals a platform owner can act on: deployment failures, 5xx rates, and quota consumption nearing the ceiling.

Playbook: deployment fails. Read the error code and classify it into capacity, identity, network, code or concurrency. For ResourceNotReady, read the container logs, then reproduce with --local. For OutOfQuota, check quota and regional capacity, then fail over to the pre-agreed second region.

Playbook: scoring returns 5xx. A 502 goes to the model owner, since run() or init() crashed. A 503 means capacity: raise the minimum instance count now, tune autoscale afterwards. A 500 is Azure-side: open a support case with the endpoint name and timestamps.

Playbook: a leaked key. Rotate the endpoint key, search git history for the string, and move the caller to token authentication. Record who held the key-retrieval role.

Playbook: a bad model is live. Shift traffic back to the previous deployment, which still exists because you did not delete it during the release. This is the reason blue/green is worth its overlap cost.

For model quality, drift is a separate problem from infrastructure health. See the Evidently guide for a tool-neutral approach, and the Prometheus and Grafana guides if you already export metrics there.

Where Azure ML stops

Azure ML is a good fit when your organization already lives in Azure, wants managed compute and endpoints, and values Entra identity and private networking. It is a weaker fit in four cases. If you need portability across clouds, a Kubernetes-native stack such as Kubernetes with KServe or BentoML keeps your serving layer independent, although the documentation does allow Kubernetes online endpoints through AKS or Arc. If your center of gravity is a lakehouse, Databricks may be closer to your data. If you want open-source experiment tracking independent of a vendor, MLflow remains the common layer. If your workload is hosted language models rather than your own, Azure OpenAI is a different service with its own operating model. Choose on the dominant constraint, and write it in the design record.

Running it as a platform for other teams

A platform is a product with users. Offer a paved road: a template repository containing a training job YAML, a deployment YAML with secure defaults (token authentication, public access disabled, one instance in dev), a pipeline that applies it, and a curated pinned environment. Teams that follow it get review, security and cost controls for free, and teams that leave it accept the work.

Publish service levels for what you own: workspace creation time, quota request turnaround, supported SDK and CLI versions. Publish a deprecation calendar so nobody learns of an end-of-support date from an outage. Measure adoption, time to first deployment, and migration progress. Run an office hour, because the cheapest fix for a platform is a conversation before the pull request.

The review checklist

For a design document or pull request:

• Which workspace and subscription, and which quota does it consume? Is the capacity formula shown?
• Which identity runs it, with what role, at what scope?
• Is public network access disabled, and is DNS resolvable from the callers?
• Are secrets references, never values? Is auth_mode justified?
• Are environments pinned and scanned? Is azure-ai-ml pinned?
• Is there exactly one writer, and is the rollback a traffic shift?
• Is there an owner, a cost tag, and a plan to delete it?
• Does anything still depend on SDK v1 or CLI v1?

The complete picture

Azure ML as a platform is a set of boundaries: subscription quota, workspace isolation, network paths split between control and data planes, identities with scoped roles, and a client surface you must pin and upgrade. Everything else is process: YAML in git, a single writer, registries for promotion, and a migration you finish on purpose.

Where the series leaves you

You can now design, secure, scale and operate Azure ML for other teams, argue its limits honestly, and finish the v1 migration. Take the checklist into your next review, and verify every cost, region and SKU claim against your own subscription before you promise it.

Sources

• Azure Machine Learning documentation
• Upgrade from v1 to v2
• What is v2
• Install and set up CLI v2
• Train models
• Deploy online endpoints
• Troubleshoot online endpoints
• Network isolation for online endpoints
• azure-ai-ml changelog
• MLOps v2 accelerator

Senior interviews for an Azure Machine Learning platform role are rarely about syntax. The interviewer wants to know whether you can design a system with the service's real boundaries in mind, diagnose an outage from a single error code, and defend a decision to someone who controls the budget. This file gives you prompts of each kind, with the reasoning a strong answer contains and the points where weaker answers go wrong. Use it by covering the answer, speaking your own aloud, then comparing.

The two-minute answer

If you are asked "how would you describe Azure ML to a new engineering director?", say something like this. Azure ML is a managed service for training, tracking and serving models. You do not run its scheduler or database. What you own is the set of boundaries around it: the subscription that holds quota, the workspace that holds assets and access control, the network paths for control and for scoring, the identities that act, and the client libraries you pin. Operating it well means keeping those boundaries explicit, keeping every entity as YAML in git, applying changes from one pipeline, and promoting models through registries.

Then stop. A concise answer that names the real boundaries signals experience. A tour of every feature signals a brochure.

System design prompt: a private ML platform for twelve teams

Prompt. A bank in the Gulf wants twelve data science teams to train and serve models on Azure ML. Data must not leave a private network. Walk me through your design.

What a strong answer covers, in order.

Start by asking questions, because the constraints decide the design. How sensitive is the data, and is there one classification or several? Which Azure region must hold it? How many models are online versus batch? What is the expected request rate for the largest one? Who approves production releases today?

Then propose the topology. For a regulated customer, I would separate environments into different workspaces, and ideally different subscriptions for production, so that quota and access are not shared with experimentation. Whether each team gets its own workspace depends on how much isolation the bank requires. If the data classification differs between teams, one workspace per team per environment is the safer answer, provided workspaces are created from Terraform or Bicep. Say that last clause aloud, since a design that requires manual workspace creation does not scale to twelve teams.

Network: set the workspace managed virtual network to Allow only approved outbound, which the documentation presents as the mode that addresses data exfiltration, and set public_network_access to disabled on online endpoints so that scoring goes through the workspace private endpoint. A strong candidate then volunteers the exception: control-plane calls go to Azure Resource Manager over the public network, so either the bank accepts that, or adds an ARM private link, which requires subscription owner and a role at the root management group. Mention private DNS: the wildcard record in the privatelink.api.azureml.ms zone for the inference hostname, and that callers on other networks must resolve through it.

Identity: managed identity on compute for training, a separate endpoint identity for secret reads, one pipeline identity per environment scoped to that workspace only, and a custom role without the Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action permission for developers so they cannot fetch endpoint keys. Token authentication for callers wherever possible.

Promotion: models and environments go into a registry, and each workspace deploys from the registry, so lineage survives. All entities are YAML in git, applied by a single pipeline, with code owners guarding network, authentication and instance-count fields.

Capacity: ask for the request rate and the per-request time and apply the documented formula, ceil(target_rps * request_process_time / target_utilization / max_concurrent_requests_per_instance). Check quota for the SKU in the chosen region, and name a second region for failover.

Where weak answers go wrong. They propose one giant workspace for everyone and rely on naming conventions for isolation. They say "private endpoint" without mentioning DNS or the ARM path. They forget v1_legacy_mode, which blocks v2 APIs and managed online endpoints in a private workspace until it is set to false, and which the documentation says to check with the network team first. And they never mention quota.

Follow-up you should expect. "How do you onboard a thirteenth team in a day?" The answer is the paved road: a template repository and an infrastructure module that creates the workspace, registry access, role assignments, tags and a starter pipeline from one pull request.

System design prompt: real-time versus batch scoring

Prompt. A retailer wants nightly demand forecasts for four million products and also a price-suggestion model called from a website at peak. Do they need the same serving design?

Strong answer. No. The nightly forecast is asynchronous and bulk, which is what batch endpoints are for, so compute is allocated for the run and released afterwards. The documentation names batch endpoints as the replacement for the old ParallelRunStep. The price model needs a managed online endpoint, with a deployment sized by the capacity formula and autoscale on top.

Discuss cost: an online endpoint costs for as long as its instances run, as far as I know, and I would verify that against the pricing page rather than assert it in an interview with numbers. Then the interviewer will ask about the spike. Autoscale reacts after load arrives, and a new instance must pull an image and run init(), so for known peaks you schedule a higher minimum, and for unknown peaks you hold headroom. A 503 during a spike is the documented symptom of this.

Add blue/green: put the new model in a deployment named green behind the same endpoint, shift traffic in steps, keep blue until confident, and then delete it, because both bill while both exist.

Weak answer. Uses an online endpoint for everything because it is the one they know, then discovers that a four-million-row scoring request does not belong in a synchronous call.

System design prompt: promoting a model safely

Prompt. Design the release path from a data scientist's notebook to production. Where are the gates?

Strong answer. The notebook is for exploration only. Training is a job defined in YAML, run by a pipeline, tracked in MLflow, and its output model is registered with a version. A gate checks the metrics against a threshold, and a second gate checks that the environment is pinned and scanned. The model is then shared through a registry, and a pull request changes the production deployment YAML to reference the new version. Reviewers see the diff in model, environment and traffic. The release pipeline, running as the only identity with write rights to production, creates the new deployment and moves traffic in stages, with a rollback that is one traffic change back to the previous deployment.

Note the human gates deliberately: approval on the traffic shift for high-risk models, none for low-risk ones. A platform that forces the same ceremony on everything gets bypassed.

Weak answer. Re-uploads the model artifact into the production workspace by hand, which breaks lineage, or edits the live deployment in place, which removes the rollback.

Incident prompt: deployments suddenly fail with OutOfQuota

Prompt. At 9am several teams report that deployments fail with ERROR: OutOfQuota. Nothing changed. What do you do?

Strong answer. First read which kind of quota. The documentation lists CPU quota, cluster quota, disk, memory, role assignment limits, endpoint limits, Kubernetes limits and region-wide VM capacity under that one code, so the first move is to read the message detail, not to file a generic request. If it is subscription vCPU quota for a SKU family, check which workspace consumes it: a forgotten cluster, a team that raised max_instances, or a pipeline that retried in a loop. Free capacity by scaling down idle compute. Then request an increase. If it is regional capacity, quota is irrelevant: the region cannot supply the machines right now, so use another region or another SKU.

Close the loop with prevention: a quota alert at a threshold, a max_instances convention, and separate subscriptions for production so experimentation cannot starve it.

Weak answer. Files a quota request and waits, without determining which limit was hit.

Incident prompt: scoring returns 502, then 503

Prompt. An endpoint that worked yesterday returns 502 for a fraction of requests and 503 for others.

Strong answer. Treat them as two different problems. 502 means run() or init() raised or crashed, so look at the container logs with az ml online-deployment get-logs and compare the failing requests with the passing ones: a particular input shape, a missing feature value, a dependency that changed. Also check whether the environment was rebuilt, because dependency drift produces errors like TypeError: register() takes 3 positional arguments but 4 were given, which indicates a Flask versus inference server mismatch. 503 points at capacity under spikes, so check instance count and autoscale settings against traffic, and raise the minimum while you investigate.

The two may be linked: if instances crash and restart, capacity falls, and the survivors are overloaded. A strong candidate says so and orders the work: stop the crash first, since capacity cannot recover while instances keep dying.

Then mitigation: shift traffic to the previous deployment if one exists, which is why you keep it.

Incident prompt: the endpoint cannot be reached from the application network

Prompt. The endpoint works from inside the workspace network. The application in another virtual network gets name resolution failures.

Strong answer. This is DNS until proven otherwise. The documentation's cause for an unresolved scoring endpoint is a missing wildcard A record in the privatelink.api.azureml.ms zone, of the form *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms. Check the answer returned from the application's network, then check that its DNS path can see the private zone through a link or a forwarder. Only after name resolution works do you check routes, then the endpoint's public-access flag, then authentication.

Weak answer. Opens the firewall, which is both slow and unsafe, because the problem was never the firewall.

Incident prompt: a key was committed to a public repository

Strong answer. Treat it as compromised on discovery. Rotate the key, find out which callers used it and move them to token authentication, and check access logs for the exposure window. Remove it from history, knowing that removal does not undo exposure. Then ask why a developer could retrieve it: the custom role that omits the key-retrieval action, and the single-writer pipeline, are the structural fixes. Add secret scanning to the repository so the next one is caught at push time.

Trade-off prompt: key authentication or token authentication

Keys are simple, since any HTTP client can send one, and they are shared secrets that must be rotated in concert with every caller and that cannot be tied to a person. Token authentication, aml_token, uses Entra identity and RBAC, which makes access revocable per caller and auditable, at the cost of callers needing to obtain tokens. My default for service-to-service traffic in the same tenant is tokens, and I would keep keys only for callers that cannot do otherwise, with rotation rehearsed. Note that the documentation does not name a recommended default in the pages I relied on, so say plainly that this is your judgment.

Trade-off prompt: one workspace or many

Fewer workspaces mean less to operate and more shared blast radius, with RBAC and quota shared among teams. More workspaces give isolation and simple cost attribution at the price of duplicated setup, which you only afford if workspaces are created from code. The deciding questions are data classification, the incident you fear most, and the size of the platform team. Say that you would start with the isolation the data demands and no more.

Trade-off prompt: serverless compute versus named clusters

Serverless removes cluster management and suits ad hoc work: a job omits compute and Azure allocates it. Named clusters give predictable capacity, an explicit max_instances ceiling, a chosen idle timeout and a clear owner, which matters for production training. A policy of serverless for exploration and named clusters for scheduled production runs is easy to defend.

Trade-off prompt: Azure ML or a Kubernetes-native stack

Azure ML wins when the organization is in Azure, wants managed compute and endpoints, and values Entra and private networking. A Kubernetes-native stack, covered in the Kubernetes guide and the KServe guide, wins on portability and fine control, and costs you the operation of the cluster. The documentation also allows Kubernetes online endpoints through AKS or Arc, so it is a spectrum, not a binary. Name the dominant constraint before choosing.

Leadership prompt: justify the platform investment

Prompt. The CFO asks why the platform team needs two more engineers when the cloud provider manages the service.

Strong answer. Managed does not mean unowned. The provider operates the service. Someone must own quota across subscriptions, network and identity design, release governance, cost control, and the migration from the v1 SDK and CLI, whose support ended on 30 September 2025 and 30 June 2026. Without that owner, each team solves these alone, differently, and expensively. Frame it in outcomes the CFO measures: forgotten endpoints retired, time from approved idea to first deployment, incidents caused by quota or misconfiguration, and security findings closed. Offer to report those numbers quarterly, and do not promise savings you cannot measure.

Leadership prompt: argue for delaying a feature to finish the v1 migration

Lead with risk and a date. v1 workflows still run, so nothing breaks on a given day, but Microsoft says they may be exposed to security risks or breaking architecture changes, and new features ship only in v2. Propose a time-boxed plan: inventory, a leaf-first order, a hard end date, and a weekly count of remaining v1 references. Offer the leader a choice between a smaller, planned cost now and an unplanned incident later, and do not pretend the cost is zero.

Leadership prompt: tell a team their request is denied

A team wants a large GPU fleet with unrestricted internet egress. Answer with a path, not a refusal. Explain the control behind each limit: quota is shared and finite, and unrestricted egress defeats the exfiltration protection. Offer what they can have: a reviewed quota allocation, a time-boxed exception, an approved outbound rule set, or a mirror of the packages they need. Leaders and teams accept constraints they understand and can appeal.

Rapid-fire questions

What is the difference between an endpoint and a deployment? An endpoint is a stable URL and authentication. A deployment is a model, code and environment on specific instances behind it. One endpoint holds several, with traffic percentages.

Why register a model? For versioning, lineage and sharing through a registry.

Does v2 data work in v1? No. A v1 dataset works in v2, but a v2 data asset does not work in v1.

What blocks v2 APIs in a private workspace? v1_legacy_mode, which is fixed by setting it to false after checking with the network team.

What does SecretsInjectionError mean? The endpoint identity lacks RBAC on the secret or the reference format is wrong, and new role assignments can take time to propagate.

Which key-vault network setting breaks key authentication? The bypass value must be AzureServices.

How do you roll back? Shift traffic to the previous deployment.

How do you know a model's capacity need? Apply the documented formula and check quota.

What replaces HyperDriveConfig? Sweep jobs.

What strong answers share

Across all of these, the strongest answers do five things. They ask a clarifying question before designing. They name the boundary involved, whether quota, workspace, network path or identity. They classify errors rather than memorize them. They state what they have not verified, such as pricing, instead of inventing a number. And they end with how they would measure that the fix worked. Practise ending every answer that way.

Sixty-second self-test

Say the answers aloud without looking.

1. Name the five boundaries you own on a managed service.
2. Which path does a private workspace not close, and what closes it?
3. State the capacity formula and its four inputs.
4. Give one custom-role action that stops developers fetching keys.
5. What is the single-writer rule, and which two errors does it prevent?
6. Which two deadlines have passed for v1, and what is the practical risk?
7. Why is a registry better than a copy for promotion?
8. Which error code family covers quota, and why is a quota increase sometimes not enough?
9. What does a 502 mean, and who owns it?
10. When would you not choose Azure ML?

If you hesitated on more than two, return to the matching section of the detailed guide and read it again before an interview.

This file is the field notes of someone who runs Azure Machine Learning for other teams. It collects the traps that appear only once a platform has many users, the habits that keep ownership manageable, and the checklists to apply to pull requests and design documents. Each trap has a symptom, a cause and a fix, because at this level you are usually called after the symptom has appeared.

Production traps that only appear at scale

Quota is the first thing to break, and it breaks for everyone

Symptom. Deployments that worked last week fail with OutOfQuota or jobs with QuotaExceeded, across several teams at once.

Cause. Quota is granted per SKU family, per region and per subscription. Teams that share a subscription share the pool, and nothing divides it for you. A single team that raises max_instances on a cluster, or a pipeline that retries in a loop, can consume what production needs.

Fix. Separate production into its own subscription. Enforce a max_instances convention for clusters in review. Alert on quota consumption before it hits the ceiling, and keep a quota request template ready. Remember that the OutOfQuota code also covers regional VM capacity, so approved quota does not guarantee machines: decide a second region in advance for anything critical.

The capacity number nobody computed

Symptom. An endpoint is approved with a guessed instance count, then returns 503 under load, or sits at ten percent utilization for a year.

Cause. Nobody did the arithmetic. The documented formula is ceil(target_rps * request_process_time / target_utilization / max_concurrent_requests_per_instance), and the documentation's own example, 20 requests per second at 10 seconds each with 70 percent utilization and one concurrent request, gives 286 instances.

Fix. Make the four inputs a required section of every design document for an online endpoint. Re-run the calculation when traffic or latency changes materially. If the result is absurd, the design is wrong, and the answer may be a faster model, higher concurrency or a batch endpoint.

Autoscale arrives after the spike

Symptom. 503 errors for the first minutes of every peak, then recovery.

Cause. Autoscale reacts to load that has already arrived. A new instance must pull an image, start a container and run init(), and a heavy model can take longer to load than the spike lasts.

Fix. For predictable peaks, raise the minimum before they begin. For unpredictable ones, hold headroom and reduce load time by shrinking the environment image and loading the model efficiently. Measure time from instance creation to first successful request and publish it.

Two writers on one endpoint

Symptom. Intermittent HTTP 409 on management calls, deployments cancelled with OperationCanceled, and a production endpoint whose state matches no commit.

Cause. A pipeline and a human, or two pipelines, are modifying the same endpoint. The documentation attributes OperationCanceled to a higher-priority operation or a lock, and 409 to another operation in progress.

Fix. One pipeline, one identity, one branch writes to each production endpoint. Remove write roles from humans in production workspaces. Treat studio edits as incidents to be reconciled in git.

The portal edit that silently diverged

Symptom. The next pipeline run reverts a change nobody remembers, or fails because the live object does not match the file.

Cause. Someone fixed a problem by clicking in the studio. The YAML in git no longer describes reality.

Fix. Make reading the default and writing the exception. Where an emergency edit is allowed, require a follow-up commit within a day. Export the live state with az ml <entity> show --output yaml to see the drift, remembering to delete system-generated properties before committing.

A private workspace that is not quite private

Symptom. A security review finds that developers can still create deployments from the public internet, although scoring is private.

Cause. Control-plane calls go to Azure Resource Manager over the public network, even when scoring is private. Disabling public access on the endpoint closes the data path only.

Fix. State the decision in the design: accept public ARM access with strong identity controls, or add an ARM private link, which requires the subscription owner and a role at the root management group. Do not claim the workspace is fully private without saying which path you mean.

v1_legacy_mode on a private workspace

Symptom. Managed online endpoint creation fails in a workspace that has a private endpoint, and the v2 APIs seem unavailable.

Cause. Workspaces with a private endpoint can have v1_legacy_mode enabled automatically, which blocks the v2 APIs and managed online endpoints.

Fix. Set it to false, after checking with the network team, as the documentation advises. Record the change. Also inventory private endpoints created before 24 May 2022, which must be recreated before online endpoints can use them.

DNS that works from one network only

Symptom. Scoring works from inside the workspace network and fails to resolve from the application's network.

Cause. The wildcard record for *.<workspace-guid>.inference.<region>.privatelink.api.azureml.ms is missing from, or not reachable through, the application's DNS path.

Fix. Resolve the name from the caller's network first, always. Link or forward the private zone for every network that needs to call the endpoint, and include DNS in your onboarding checklist.

Key-based authentication that cannot be created

Symptom. Creating a key-authenticated endpoint in a private workspace fails.

Cause. The Key Vault network rule's bypass value is not AzureServices.

Fix. Check with az keyvault network-rule list -n <key-vault-name>, correct the rule through your infrastructure code, and add a policy test so it does not drift.

Secret injection that fails right after the role assignment

Symptom. SecretsInjectionError on a deployment whose role assignment looks correct.

Cause. Either the endpoint identity lacks read permission on the secret, the reference format is wrong, or the assignment has not propagated yet.

Fix. Create the role assignment in the infrastructure pipeline, wait for propagation, then deploy. Validate the reference format in a pull request check, and retry once before escalating.

Curated environment names that vanish

Symptom. EnvironmentNotFound for an environment that worked for months.

Cause. Curated AzureML-... environments rotate, and the documentation's own examples use different names for the same task. Referencing @latest makes behaviour change underneath you.

Fix. Register your own environments with explicit versions, scan them, reference them as azureml:name:version, and list ml_client.environments.list() before relying on any curated one. Note that ARM has no @latest for environments, so REST-based and CLI-based pipelines can resolve differently.

Dependency drift inside the inference server

Symptom. Containers crash at startup with TypeError: register() takes 3 positional arguments but 4 were given, or ModuleNotFoundError.

Cause. Unpinned dependencies. The TypeError is documented as a Flask 2 versus azureml-inference-server-http mismatch.

Fix. Pin everything, including azureml-inference-server-http, in the environment's conda file, and test the built image locally with az ml online-deployment create --local before the cloud deployment.

The idle endpoint

Symptom. The monthly bill contains endpoints nobody has called for weeks.

Cause. Endpoints, as far as I know, bill for running instances regardless of traffic, which I have not verified on the pricing page. They do not shut themselves down.

Fix. Maintain an inventory with owner, cost centre and last-called date, produced from az ml online-endpoint list --output table and Azure Monitor. Delete or archive on a schedule, with a notification to the owner first.

The blue deployment that never got deleted

Symptom. Costs doubled after a release and stayed that way.

Cause. Blue/green keeps two sets of instances until someone deletes the old one.

Fix. Put "delete the old deployment after the observation window" into the release pipeline itself as a final, delayed step, so it does not depend on memory.

Mixing v1 and v2 in one script

Symptom. Confusing import errors or wrong-class behaviour, such as a Model that has the wrong attributes.

Cause. Class names such as Workspace and Model clash across the v1 and v2 namespaces.

Fix. Never combine them. Migrate a workload completely and in leaf-first order, since a v1 dataset works in v2 but a v2 data asset does not work in v1.

Platform-ownership habits

Treat every entity as code. Jobs, data assets, environments, components, models, endpoints, deployments and schedules all have YAML forms under https://azuremlschemas.azureedge.net/latest/. If it is not in git, it is not governed.

Separate control-plane code from ML code. Infrastructure goes through Terraform or Bicep, as covered in the Terraform guide, and teams own their training and deployment YAML. Different owners, different review rules, different pipelines.

Pin and upgrade on a schedule. Pin azure-ai-ml in requirements. The SDK ships often, and recent entries have included serialization fixes and immutable-field fixes for deployment templates. Upgrade in a test pipeline, read the changelog entry, and promote the new pin only after tests pass. For the ml CLI extension, use az extension update -n ml in a controlled job, not ad hoc on laptops.

Make secure defaults the paved road. A template deployment YAML with auth_mode set deliberately, public access disabled, secrets by reference and one instance in dev means most teams never make the risky choice. Spend your time on the template, not on policing.

Publish what you support. Maintain a page listing supported SDK and CLI versions, regions, SKUs, and a deprecation calendar. Include the v1 dates: CLI v1 support ended on 30 September 2025, SDK v1 support ended on 30 June 2026.

Measure the platform, not just the models. Track time to first deployment, quota requests and their turnaround, incidents by category (capacity, identity, network, code, concurrency), count of endpoints with no traffic, and remaining v1 references. Review them monthly with the teams.

Write runbooks from incidents. After each incident, add the error code, its category, the first command and the owner to the runbook. Over a year this becomes the most valuable document you own.

Run a game day twice a year. Break a test deployment on purpose, revoke a role, remove a DNS record, and exhaust a quota in a sandbox. The goal is to learn how long diagnosis takes with only the runbook.

Keep a verification list. Some facts, such as prices, spot availability, region support per SKU and autoscale commands, change faster than documentation reviews. When you state one in a design document, mark the date you verified it and the page you used.

Look after the human side. Hold office hours, answer in the team channels where people already ask, and say no with an alternative. A platform that people avoid has failed, however good its controls.

Mind your local tooling. If engineers' laptops sit behind a TLS-inspecting gateway such as Cloudflare Gateway, command-line tools with their own trust stores can fail with certificate errors, including CERTIFICATE_VERIFY_FAILED from az against login.microsoftonline.com. The right fix is to trust the gateway's root certificate authority, not to disable verification. This is local operational experience, not Microsoft documentation, so document the fix for your own estate.

Checklist for a pull request

Use this when a team changes entity YAML or pipeline code.

Identity and access

• Which identity applies this, and is it scoped to this environment only?
• Does the change add or widen a role assignment? Is the scope the narrowest that works?
• Is there any literal secret, key, connection string password or SAS token anywhere in the diff? Treat a hit as a rotation event.

Network and exposure

• Does public_network_access change? Does egress_public_network_access appear in new code?
• Does the change need a new outbound rule, and can an internal mirror replace it?

Authentication

• What is auth_mode, and why? Who calls this endpoint, and can they use tokens?

Reproducibility

• Are environments referenced by explicit version, not @latest?
• Is azure-ai-ml pinned, and is azureml-inference-server-http pinned in the environment?
• Does the deployment work with --local before the cloud run?

Capacity and cost

• Is the instance type justified, and is the capacity formula result in the description?
• Does the PR create a second deployment? Where is the step that deletes the old one?
• Is the resource tagged with owner and cost centre?

Release

• Does the traffic change go in stages, and is the rollback a single traffic shift to a deployment that still exists?
• Is exactly one pipeline responsible for applying this to production?

Migration

• Does the diff add any import from azureml.core or use of azure-cli-ml? It should not.

Checklist for a design document

Use this before approving a new workload.

Purpose and fit. What decision does the model support, and is online, batch or neither the right shape? Why Azure ML rather than the alternatives, and what is the dominant constraint?

Topology. Which subscription, workspace and region, and why? Which data classification applies? Which quota will be consumed, and what happens if the region lacks capacity?

Network. Inbound and outbound design. Whether control-plane traffic crosses the public network and whether that is accepted. How DNS reaches every caller. Whether the workspace is in v1_legacy_mode and who decided.

Identity. Human roles, pipeline identity, compute identity, endpoint identity. Which custom role keeps key retrieval away from developers.

Secrets and supply chain. Where secrets live and how they are injected. Environment base image, pinning, scanning, and the registry used.

Capacity and cost. Four inputs to the formula, resulting instance count, autoscale plan for known peaks, monthly cost estimate with its verification date, and an owner for the cost.

Release and rollback. Blue/green plan, traffic steps, observation window, rollback command, deletion of the old deployment.

Observability. Where logs, metrics and traces go. The alerts, their thresholds and their owners. How drift will be detected, linking to the Evidently guide or your own tooling.

Lifecycle. Who owns the endpoint, when it will be reviewed, and what triggers its deletion.

Unknowns. The facts the author could not verify and who will verify them. A design document that lists its unknowns is more trustworthy than one that has none.

Practice cards

Card one: quota drill. In a sandbox subscription, find your remaining quota for one SKU family in your main region, create a cluster whose max_instances exceeds it, and read the resulting error. Write down the exact text and the category it belongs to.

Card two: key lockdown. Create a custom role that omits the onlineEndpoints/token/action permission, assign it to a test user, and confirm that az ml online-endpoint get-credentials is refused. Then confirm the same user can still submit a job.

Card three: DNS from two networks. For a private endpoint, resolve the scoring hostname from two different virtual networks and fix whichever fails.

Card four: the rollback. Deploy blue, then green with a small traffic share, then move all traffic back to blue using one command or one SDK update to endpoint.traffic. Time it.

Card five: migration inventory. Count the files and images in your organization that reference azureml-core, azureml.core or azure-cli-ml. Share the count with the teams and agree an end date.

Card six: cost sweep. List online endpoints, find the quiet ones, and contact their owners. Measure what you saved.

Where to go next

Pair this guide with the MLflow guide for tracking and the model registry, the GitHub Actions guide for the pipeline that applies your YAML, and the Vault guide if secrets cross cloud boundaries. Keep the two checklists in your repository's pull request template and your design template, and refine them after every incident.