The Complete Pydantic AI Guide

This is part one of three. It covers everything you need to build a real, working AI agent with Pydantic AI, not a teaser. By the end you can call a language model from typed Python, force it to answer in a structure your code can trust, give it tools it can call, pass it your database connection safely, hold a multi-turn conversation, stream its answer, and test all of it without spending a cent. Mid-level and Senior take the same topics further; nothing here is thrown away.

Each section ends with a Try it task. Do them as you go. They take a few minutes each, and an agent only starts to make sense once you have watched your own one call a tool, fail validation, and recover.

One warning before we begin. Pydantic AI released its second major version in June 2026, and a great deal of the code on blogs and in older tutorials was written for version 1. This guide targets Pydantic AI 2.52. Where an old habit now breaks, the text says so and shows the current form, and there is a short section near the end on recognising version 1 code.

What Pydantic AI is, and the problem it solves

A language model, on its own, takes text in and returns text out. That is enough for a chat window and not nearly enough for software. Software wants a number it can add up, a record it can store, a decision it can branch on. Software also wants the model to look things up, call a service, or check a database, and to do so without a developer hand-writing the same fragile loop for the hundredth time.

Pydantic AI is a Python library for building agents on top of language models. An agent, in this library, is a small object that sends your instructions and a question to a model, lets the model call Python functions you wrote, checks everything that comes back, and keeps going until it has an answer in exactly the shape you asked for. The whole thing is typed, so your editor and your type checker understand what goes in and what comes out.

The library comes from the team behind Pydantic, the validation library that sits underneath a large part of the Python ecosystem, and that heritage is the point. Pydantic turns "a dictionary that is probably what I expected" into "an object I have verified". Pydantic AI applies the same idea to the least predictable component you will ever put in a program, a language model. The model proposes; Pydantic validates; if the proposal is wrong, the validation error goes back to the model and it tries again.

What came before? In the early days you called a vendor's HTTP API directly, built a prompt string by hand, and parsed the reply with regular expressions and hope. Then came larger frameworks that offered chains, graphs, memory modules and dozens of integrations. They are powerful, but newcomers often find that a simple task needs a lot of vocabulary. Pydantic AI takes a narrower line: ordinary Python, ordinary type hints, ordinary functions as tools, and a small number of concepts. If you have used FastAPI, the feeling is similar, and the team says so openly; the documentation compares agents to a FastAPI app, something you create once and reuse.

Three things about this design explain most of what follows.

The model is untrusted input. Whatever a model returns is just text that looks plausible. Pydantic AI treats it the way a web framework treats a form submission: validate it, and reject it if it does not fit. This is why the library feels strict compared with simply calling a model and printing the result.

Everything is driven by types. The shape of the answer is a Python class. The arguments of a tool are the parameters of a Python function. The data you hand to your tools is a dataclass. The library reads those types and builds the schemas it sends to the model, so you write each fact once.

It is a library, not a service. There is no server to run, no daemon, no database to install. An agent lives inside your own program, whether that is a script, a notebook, a FastAPI endpoint or a background worker. It calls a model provider over the network and that is the only external moving part.

You need little to follow along: Python 3.10 or newer, a terminal, and an account with one model provider. If you have no key yet, do not worry. A built-in test model lets you do most of this guide with no key and no network at all.

The mental model: five nouns

Pydantic AI has a larger surface than this guide, but you can do real work by understanding five ideas. Learn these and the rest of the documentation becomes easy to navigate.

Agent. The Agent class is the central object. It holds a default model, your instructions, the tools the model may use, the type of result you expect, and the type of the dependencies you will pass in. It is generic: the full type is Agent[DepsT, OutputT], which reads "an agent that takes dependencies of this type and produces output of that type". You create it once, usually at module level, and call it many times. It keeps no memory between calls on its own.

Model. The model is the language model that does the thinking, and the library speaks to many of them: OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Groq, Mistral and others, including models on your own machine through Ollama. You usually name one with a short string of the form provider:model-name, such as openai:gpt-5.2. Behind that string sits a class that knows one vendor's API.

Tool. A tool is a Python function the model is allowed to call. You write the function and decorate it; the library describes it to the model from its signature and its docstring. When the model decides it needs the function, it names it and supplies arguments, the library validates those arguments, runs your function, and sends the return value back to the model.

Output. The output is the final, typed answer. By default it is a plain string. You can instead ask for a number, a list, or any Pydantic model, dataclass or typed dictionary, and the library makes sure the model's answer fits before you ever see it.

Dependencies. Dependencies, or deps, are the things your tools need from the outside world: a database connection, an HTTP client, an API key, the identity of the current user. You pass them in for each call and your tools read them from a context object. This keeps tools testable and keeps secrets out of global variables.

Around those five sits a sixth term you will meet constantly, the run. A run is one execution of the agent: you give it a prompt, it loops through model calls and tool calls, and it ends with a result or an error. A conversation is simply several runs where you pass the earlier messages back in.

Here is the loop in more detail, because understanding it makes every error message easier to read.

First the agent sends the model your instructions, any earlier messages, and the new prompt. The model answers either with text or with a request to call one or more tools. If it asked for tools, the library checks the arguments against each tool's schema. If they are valid, it runs the functions and sends the results back as a new message, then asks the model again. If the arguments are invalid, it sends the validation error back instead and asks again. This repeats until the model produces something that satisfies the output type, or a limit is reached.

A final detail that surprises people: when you ask for structured output, the library normally delivers the output schema to the model as a tool as well. The model "calls" a tool whose arguments are your result fields, and validation of that call is what produces your typed object. You never need to think about this when things work, but it explains why output validation errors look like tool errors.

Installing and checking your setup

Pydantic AI needs Python 3.10 or newer and installs with pip or uv. Both work on Linux, macOS and Windows, because the package is pure Python. A virtual environment keeps the library and its many dependencies away from your system Python, so use one.

On Linux or macOS:

python3 -m venv .venv
source .venv/bin/activate
pip install pydantic-ai

On Windows, in PowerShell:

py -m venv .venv
.venv\Scripts\Activate.ps1
py -m pip install pydantic-ai

If you prefer uv, the fast Python package manager, the equivalent is:

uv init
uv add pydantic-ai

The plain pydantic-ai package is the convenient bundle. As of version 2 it installs the core library plus support for OpenAI, Anthropic and Google models, the command-line tool, MCP support (a protocol for plugging in external tools), the evaluation library, a small web interface, retry helpers and Logfire, the observability service from the same company. It does not include the extras for Amazon Bedrock, Groq, Mistral, Cohere, xAI and Hugging Face. If you want one of those, name it in square brackets:

pip install "pydantic-ai[bedrock,groq]"

There is also a slimmer package, pydantic-ai-slim, where you choose exactly what you want. It is useful in production images where size matters:

pip install "pydantic-ai-slim[openai]"

Because the project releases a new minor version every day or two, it is good practice to see what you got. Check the version from Python:

python -c "import pydantic_ai; print(pydantic_ai.__version__)"

You should see 2.52.0 or something newer in the 2.x line. Then run the smallest possible agent. It uses a built-in fake model named test, so it needs no API key and makes no network call:

from pydantic_ai import Agent

agent = Agent('test')
result = agent.run_sync('hello')
print(result.output)

If that prints a short placeholder string, your installation works. The test model is real documented behaviour, not a hack: it exists so that you and your test suite can exercise an agent without a provider.

If you work in Jupyter, Google Colab or a similar notebook, there is one difference. A notebook already runs an event loop, so calling run_sync can fail with RuntimeError: This event loop is already running. In a notebook, write result = await agent.run('hello') at the top level instead. We explain the difference between those two calls shortly.

Your first real agent

Now to talk to an actual model. You need a key from one provider, and the library reads it from an environment variable named for that provider. For OpenAI it is OPENAI_API_KEY; for Anthropic, ANTHROPIC_API_KEY; for the Gemini API, GOOGLE_API_KEY. Set it in your shell rather than writing it into code, because anything in code ends up in version control sooner or later.

export OPENAI_API_KEY='paste-your-key-here'

On Windows PowerShell the equivalent is $env:OPENAI_API_KEY="paste-your-key-here". Now the agent:

from pydantic_ai import Agent

agent = Agent(
    'openai:gpt-5.2',
    instructions='Be concise.',
)

result = agent.run_sync('What is the capital of Italy?')
print(result.output)

Run it with python first_agent.py. The answer is a short sentence naming Rome. Take the program apart, because every later example is a variation on it.

The first argument, 'openai:gpt-5.2', picks the model. The part before the colon is the provider, and the part after is the model's own name. Since version 2 the provider prefix is required. Writing just 'gpt-5.2', which worked in version 1, now fails with a UserError saying the model is unknown.

The instructions argument tells the model how to behave on every run. We will say more about it shortly.

run_sync runs the whole loop and waits for it to finish. It returns an AgentRunResult, and the final answer is in its .output attribute. That result also carries the token usage and the full message exchange, which we use later.

There are in fact five ways to run an agent. As a beginner you need only the first three, but it helps to know the family exists.

run_sync is a convenience wrapper around run. Use run_sync for scripts and learning, and run inside any async def.

Now look at what came back besides the text.

from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2', instructions='Be concise.')
result = agent.run_sync('What is the capital of Italy?')

print(result.output)
print(result.usage)
print(len(result.all_messages()))

result.usage reports how many requests were made and how many input and output tokens were used, and it is where cost accounting starts. Note that in version 2 it is a property, so there are no parentheses. Version 1 wrote result.usage(), and copying that into 2.x raises an error. result.all_messages() returns the full list of messages exchanged, a request and a response in the simplest case, which is the raw material for conversations.

Choosing a model and providing credentials

The string provider:model is the everyday way to pick a model, so it is worth knowing the common prefixes. In version 2 they are openai, anthropic, google for the Gemini API, google-cloud for Gemini through Google Cloud, bedrock, groq, mistral, xai, openrouter and ollama, among others. clai --list-models, which we meet later, prints the names the library knows about.

Two renames from version 1 catch people out. The old google-gla: and google-vertex: prefixes are now google: and google-cloud:. And the bare openai: prefix now means OpenAI's newer Responses API. If you specifically want the older Chat Completions API, which many OpenAI-compatible servers implement, write openai-chat:. For most beginners the default does exactly what you want, but when you point the library at a third-party server that copies the older API, openai-chat: is the one to try.

Each provider reads its credentials from environment variables. The most common are:

To pass a key explicitly, or to point at a different endpoint such as an OpenAI-compatible server in your own network, build the provider yourself.

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    'my-model-name',
    provider=OpenAIProvider(
        base_url='http://localhost:8000/v1',
        api_key='not-needed-locally',
    ),
)
agent = Agent(model)

That pattern is how you use a self-hosted model, which matters for teams in regulated sectors who cannot send prompts outside their own network or country. The library does not care whether the endpoint is a cloud service or a server in your own data centre, as long as it speaks a supported API. If you run models locally with Ollama, there is a dedicated ollama: prefix, and the neighbouring guide on Ollama shows how to serve a model on your own machine. If you want one gateway in front of many providers, the guide on LiteLLM covers that idea.

What happens when you forget the key? The library tells you plainly. For OpenAI the error is a UserError that begins "Set the OPENAI_API_KEY environment variable or pass it via OpenAIProvider(api_key=...) to use the OpenAI provider." It then adds a hint: "To try Pydantic AI without an API key, use the built-in test model: Agent('test')." That is the library reminding you of the smoke test from the installation section. The same pattern holds for the other providers, naming their own variable.

Instructions: telling the agent how to behave

Instructions are the standing orders you give the model: its role, its tone, what it should and should not do. Pydantic AI offers two related features, instructions and system prompts, and the documentation recommends instructions for almost everything.

The difference is subtle but practical. Instructions are re-evaluated on every run, and only the current agent's instructions are sent. If you pass earlier messages in as history, instructions stored inside that history are not sent again. A system prompt, by contrast, is kept in the message history and travels with it. Beginners should reach for instructions, because it means you can change an agent's behaviour by editing code, and old conversations will follow the new behaviour rather than a stale prompt.

The simplest form is a string:

from pydantic_ai import Agent

agent = Agent(
    'openai:gpt-5.2',
    instructions=(
        'You are a support assistant for an online bookshop. '
        'Answer politely and in at most three sentences. '
        'If you do not know, say so instead of guessing.'
    ),
)

Sometimes the instruction depends on information that only exists at run time, such as the name of the user asking. For that, decorate a function with @agent.instructions. It receives a RunContext, an object we describe in the dependencies section, and returns a string:

from dataclasses import dataclass
from pydantic_ai import Agent, RunContext

@dataclass
class Deps:
    customer_name: str

agent = Agent('openai:gpt-5.2', deps_type=Deps)

@agent.instructions
def personalise(ctx: RunContext[Deps]) -> str:
    return f"The customer's name is {ctx.deps.customer_name}. Greet them by name."

result = agent.run_sync('Hi there', deps=Deps(customer_name='Mona'))
print(result.output)

You can combine a constructor string with one or more decorated functions.

Structured output: answers your code can trust

This is the feature that most beginners come to Pydantic AI for. Instead of a string you must parse, you declare the shape of the answer as a Pydantic model and the agent returns an instance of it.

A Pydantic model is a class whose fields have type annotations. Pydantic checks that data matches those types and converts it where it sensibly can. Pass it as output_type:

from pydantic import BaseModel
from pydantic_ai import Agent

class City(BaseModel):
    city: str
    country: str

agent = Agent(
    'google:gemini-3-flash-preview',
    output_type=City,
    instructions='Answer with the city and country.',
)

result = agent.run_sync('Where were the 2012 Olympics held?')
print(result.output)
print(result.output.city)

The printed output is a City object, shown as City(city='London', country='United Kingdom'), and result.output.city is plain 'London'. Your editor knows result.output is a City because the agent is generic in its output type, so autocompletion and type checking work on the answer.

What happens if the model replies with something that does not fit, such as a missing field or a number where text was expected? Pydantic raises a validation error, the library turns that into a message to the model describing what was wrong, and the model gets another chance. You did not write any retry code. By default the output has a small retry budget; if the model still cannot comply, you get an UnexpectedModelBehavior error, and we discuss reading it in the errors section.

You can add richer rules with the usual Pydantic tools. Field descriptions help the model understand what you mean, and constraints reject nonsense:

from pydantic import BaseModel, Field
from pydantic_ai import Agent

class Review(BaseModel):
    sentiment: str = Field(description='one of: positive, negative, neutral')
    score: int = Field(ge=1, le=5, description='rating from 1 to 5')
    summary: str = Field(description='one sentence summary')

agent = Agent('openai:gpt-5.2', output_type=Review)
result = agent.run_sync('The delivery was late but the book was wonderful.')
print(result.output.score)

If the model returns a score of 9, validation fails on le=5 and the model is told to correct it. This is the heart of the library: the constraints you already know how to write in Pydantic become guardrails around the model.

Output types are not limited to classes. You can ask for a plain int, a list[str], a dataclass or a typed dictionary. When the answer could be one of several shapes, pass a list of types, which means "any one of these":

from pydantic import BaseModel
from pydantic_ai import Agent

class Box(BaseModel):
    width: int
    height: int

agent = Agent('openai:gpt-5.2', output_type=[Box, str])
result = agent.run_sync('A box that is 10 wide and 20 high')
print(result.output)

Here the model may return a Box, or a string if it cannot. Use the list form rather than Box | str, because the type checker mypy has trouble with the pipe form in this position.

Function tools: letting the model act

A model knows only what it was trained on and what you put in the prompt. It cannot know today's date, an order's status, or the contents of your database. A tool fixes that: you write a Python function, and the model can ask the agent to run it.

There are two decorators. Use @agent.tool_plain for a function that needs nothing from the outside, and @agent.tool for one that needs the run context, which gives access to your dependencies. Begin with the plain one:

import random
from pydantic_ai import Agent

agent = Agent(
    'openai:gpt-5.2',
    instructions="Use the tool to roll the die, then tell the player what came up.",
)

@agent.tool_plain
def roll_dice() -> str:
    """Roll a six-sided die and return the result."""
    return str(random.randint(1, 6))

result = agent.run_sync('Roll the die for me.')
print(result.output)

Walk through what happens. The library inspects roll_dice and tells the model there is a tool with this name and this description. The docstring becomes the description, so write it for the model: say what the function does and when to use it. The model replies with a request to call roll_dice; the library runs it and sends back the string it returned; the model then writes the final answer. The program prints a sentence like "You rolled a 4."

Tools can take arguments, and the type hints become the argument schema. Documenting the parameters in the docstring helps the model fill them in correctly. The library understands the common docstring styles (Google, NumPy and Sphinx):

from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2')

ORDERS = {'A100': 'shipped', 'A101': 'processing'}

@agent.tool_plain
def order_status(order_id: str) -> str:
    """Look up the status of a customer order.

    Args:
        order_id: the order identifier, such as A100
    """
    return ORDERS.get(order_id, 'unknown order')

result = agent.run_sync('Where is order A101?')
print(result.output)

The arguments the model supplies are validated against the signature before your function runs. If the model sends a number where a string is required, or omits a required argument, validation fails and the model is told what was wrong, so a bad call never reaches your code.

A tool can also be async def, and for anything that waits on a network or a database you should prefer that. Synchronous tools still work, since the library runs them in a thread pool, but async tools let many runs share one event loop efficiently.

Tools can fail on purpose. If a tool receives a value that is syntactically valid but wrong, such as an order that does not exist, raise ModelRetry with a message, and the model is asked to try again with that guidance:

from pydantic_ai import Agent, ModelRetry

agent = Agent('openai:gpt-5.2')

@agent.tool_plain
def order_status(order_id: str) -> str:
    """Look up the status of a customer order."""
    if not order_id.startswith('A'):
        raise ModelRetry('Order ids start with the letter A, for example A100.')
    return 'shipped'

The message you raise is sent back as if it were a tool result, so write it the way you would explain the mistake to a colleague. The default budget is a single retry per tool, which you can raise with @agent.tool_plain(retries=3) when a tool is genuinely hard to call correctly.

Dependencies and RunContext

Tools usually need something: a database connection, an HTTP client, a user id. You could use global variables, but globals are hard to test and easy to get wrong when many requests run at once. Pydantic AI offers dependency injection instead, in the same spirit as FastAPI.

The recipe has three steps. Define a class that holds what your tools need, usually a dataclass. Tell the agent its type with deps_type. Then pass an instance in at run time with deps=, and read it inside tools through ctx.deps:

from dataclasses import dataclass
from pydantic_ai import Agent, RunContext

class FakeDb:
    async def get_order(self, order_id: str) -> str:
        return {'A100': 'shipped', 'A101': 'processing'}.get(order_id, 'unknown')

@dataclass
class Deps:
    db: FakeDb
    customer: str

agent = Agent(
    'openai:gpt-5.2',
    deps_type=Deps,
    instructions='You are a helpful support assistant.',
)

@agent.tool
async def lookup_order(ctx: RunContext[Deps], order_id: str) -> str:
    """Look up an order's status.

    Args:
        order_id: the order identifier
    """
    status = await ctx.deps.db.get_order(order_id)
    return f'Order {order_id} for {ctx.deps.customer} is {status}.'

result = agent.run_sync(
    'Where is order A100?',
    deps=Deps(db=FakeDb(), customer='Mona'),
)
print(result.output)

Look at the first parameter of lookup_order: ctx: RunContext[Deps]. This is the run context, a small object the library passes to any function that wants it. The [Deps] part tells the type checker what ctx.deps is, so ctx.deps.db autocompletes and a typo is caught before you run anything. The context does not appear in the schema the model sees; the model only sees order_id.

An important detail: deps_type=Deps is only for type checking. It does not create the dependencies. You must still pass deps=Deps(...) on every run, and if a tool needs them and you forgot, you will get an AttributeError on ctx.deps at the first use rather than a helpful message.

Why go to this trouble? Three reasons. Tools stay free of hidden global state, so they are easy to read. Each request can carry its own user and its own connection, which is exactly what a web service handling many customers needs. And in tests you can pass a fake database, as the example does, and never touch the real one.

Conversations: carrying history between runs

A single run has no memory of earlier runs. Ask "Where were the 2012 Olympics held?" and then, in a separate call, "And its population?", and the second call has no idea what "its" refers to. To continue a conversation you pass the earlier messages back in.

Every result carries its messages. result.new_messages() returns only the messages from the run that just finished, and result.all_messages() returns everything including any history you passed in. The usual pattern is to hand the new messages of one run to the next:

from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2', instructions='Be concise.')

first = agent.run_sync('Where were the 2012 Olympics held?')
print(first.output)

second = agent.run_sync(
    'And what is its population?',
    message_history=first.new_messages(),
)
print(second.output)

The second call receives the first exchange as context, so "its" resolves to London. Because instructions are re-evaluated each run and are not stored in history, the second run still obeys the current instructions.

To keep going for many turns, collect the messages as you go. A common beginner pattern is a simple loop:

from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2', instructions='You are a friendly tutor.')
history = []

while True:
    question = input('You: ')
    if question.strip().lower() in {'quit', 'exit'}:
        break
    result = agent.run_sync(question, message_history=history)
    history = result.all_messages()
    print('Agent:', result.output)

Each turn, all_messages() already contains the earlier history plus the new turn, so assigning it back to history keeps the whole conversation.

There are two things to understand about history. First, it grows, and every message is sent to the model again on every turn, so long conversations cost more and eventually exceed the model's context window. Production systems trim or summarise old messages, a topic for the Mid-level guide. Second, history is yours to store. The agent keeps nothing between runs. If you want a conversation to survive a restart, save the messages, for example as JSON, and load them later. The library gives you a type adapter for that:

from pydantic_ai import ModelMessagesTypeAdapter
from pydantic_core import to_json

blob = to_json(result.all_messages())          # bytes you can store anywhere
history = ModelMessagesTypeAdapter.validate_json(blob)

Store the bytes in a file, a database column or a cache, and turn them back into messages with validate_json. Treat saved history as trusted data you produced yourself. If a browser or other untrusted client can send you history, the Senior guide explains why that is dangerous and how to clean it.

Streaming: showing the answer as it arrives

Language models produce text a piece at a time, and waiting for the whole reply makes an application feel slow. Streaming lets you show text as it is generated. Streaming uses an async context manager, so it lives inside an async def:

import asyncio
from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2', instructions='Be concise.')

async def main():
    async with agent.run_stream('Explain what an API is.') as response:
        async for chunk in response.stream_text():
            print(chunk, end='', flush=True)
    print()

asyncio.run(main())

By default stream_text() yields the text accumulated so far each time, so you may see the sentence grow rather than receive only the new piece. If you want only the new fragment each time, pass delta=True:

async with agent.run_stream('Explain what an API is.') as response:
    async for piece in response.stream_text(delta=True):
        print(piece, end='', flush=True)

There is a trap in streaming that the documentation calls out. run_stream() treats the first output that matches your output type as the final answer. With the default settings, if the model writes some text before calling a tool, the stream can end at the text, and the tool call that follows never runs. If you need the full tool loop to continue while you watch events, use run_stream_events() or iter(), which the Mid-level guide covers. For a simple streaming chat with no tools, run_stream is exactly right.

Limits, retries and reading the errors

Agents loop, and loops need brakes. Pydantic AI ships sensible defaults, and understanding them turns scary tracebacks into short conversations.

The most important default is the request limit: a single run may make at most 50 model requests. A run that keeps calling tools in circles will stop with a UsageLimitExceeded error whose message begins "The next request would exceed the request_limit of 50". You can set your own limits with UsageLimits:

from pydantic_ai import Agent, UsageLimits, UsageLimitExceeded

agent = Agent('openai:gpt-5.2')

try:
    result = agent.run_sync(
        'Write a long essay about rivers.',
        usage_limits=UsageLimits(output_tokens_limit=200),
    )
except UsageLimitExceeded as exc:
    print('Stopped:', exc)

You can limit requests, tool calls, input tokens, output tokens, total tokens and an approximate dollar cost. Treat the cost limit as a guide, not a billing guarantee, since it depends on bundled price data. Pair it with a spend cap set at your provider.

The next default is the retry budget, which is one. A tool or an output that fails validation once gets one more chance; if it fails again the run stops. Raise it on the agent with Agent(..., retries=3), or per tool with @agent.tool(retries=3).

Now the errors, one by one, because you will meet all of them.

When a run fails in a way you cannot explain, the single most useful tool is capture_run_messages. It records every message exchanged, even when the run raises:

from pydantic_ai import Agent, capture_run_messages, UnexpectedModelBehavior

agent = Agent('openai:gpt-5.2', output_type=int)

with capture_run_messages() as messages:
    try:
        agent.run_sync('Name a colour.')
    except UnexpectedModelBehavior:
        for message in messages:
            print(message)

Reading those messages shows you what the model actually said and what validation told it. Nine times in ten, the cause is visible there: the model returned text when you asked for an integer, or called a tool with the wrong argument name.

An error that surprises people after the move to version 2 is a TLS certificate failure, SSL: CERTIFICATE_VERIFY_FAILED. The library's own web requests now verify certificates against the operating system's trust store instead of a bundled list. In a slim container image, or behind a corporate proxy that inspects encrypted traffic and re-signs it with its own certificate authority, that authority may be missing from the system store. The fix is to install the certificate into the operating system's store, or to pass your own configured HTTP client to the provider.

Testing without a model

An agent that calls a paid, unpredictable model is awkward to test. Pydantic AI solves this with test models that run locally, so your test suite is fast, free and repeatable.

TestModel is a stand-in that calls your tools with generated arguments and returns a valid placeholder answer. You swap it in with agent.override, which affects only the code inside the with block:

from pydantic_ai import Agent, models
from pydantic_ai.models.test import TestModel

models.ALLOW_MODEL_REQUESTS = False   # fail loudly if anything reaches a real model

agent = Agent('openai:gpt-5.2')

@agent.tool_plain
def order_status(order_id: str) -> str:
    """Look up an order."""
    return 'shipped'

def test_agent_runs():
    with agent.override(model=TestModel()):
        result = agent.run_sync('Where is my order?')
    assert result.output is not None

The line models.ALLOW_MODEL_REQUESTS = False is a safety net. If any test accidentally reaches a real model, it raises RuntimeError: Model requests are not allowed, since ALLOW_MODEL_REQUESTS is False instead of quietly spending money. When you want a model whose replies you script yourself, FunctionModel lets you pass a function that returns whatever response you like.

The clai command line

Installing Pydantic AI also gives you clai, a command-line chat client. It is the quickest way to talk to a model from a terminal and a good way to check that your key and model name work before you write code.

clai

With no arguments it starts an interactive session. Type a question and press Enter. A prompt in quotes runs one question and exits:

clai "Explain DNS in two sentences"

Choose a model with -m, and list the model names the library knows with -l:

clai -m anthropic:claude-sonnet-4-6 "Explain DNS in two sentences"
clai --list-models

If you do not want streamed output, add --no-stream. Inside an interactive session, the slash commands include /exit to leave, /markdown to switch markdown rendering, /multiline for multi-line input, /cp to copy the last answer, and /usage to see token usage.

You can also run clai against your own agent. The -a option takes module:variable, so if support.py defines agent = Agent(...), then this opens a chat with it:

clai -a support:agent

That is useful for trying instructions and tools by hand without writing a chat loop. clai also has a web command that opens a browser chat, which is convenient, but it is for local development only. Never expose it to a network. Earlier versions of the web interface had security weaknesses that were patched in releases 2.28 and 2.30, and even a patched development server is not designed to face users.

Configuration and everyday settings

Beyond instructions and tools, a few settings shape how a model behaves, and you pass them as model_settings. They are a plain dictionary:

from pydantic_ai import Agent

agent = Agent(
    'openai:gpt-5.2',
    model_settings={'temperature': 0.0, 'max_tokens': 500},
)

result = agent.run_sync(
    'Summarise: Pydantic AI is a Python agent library.',
    model_settings={'max_tokens': 100},
)

The common fields are temperature, which controls randomness (a low value makes answers more repeatable), max_tokens, which caps the length of the reply, top_p, timeout, seed and parallel_tool_calls. Settings can be given when you build the model, on the agent, and on each run, and later layers win, so a run-level value overrides the agent's. Not every provider supports every field, so if a setting seems ignored, check that provider's page in the documentation.

Two other agent arguments are worth knowing at this level. name labels the agent, which helps in logs. retries sets the default retry budget, as discussed. There are many more, such as toolsets, capabilities and end_strategy, but you can leave them alone until you have a reason.

For observability, which means seeing what your agents do, Pydantic AI emits standard OpenTelemetry traces and works with Logfire, the company's own service, as well as with any OpenTelemetry-compatible backend. The simplest setup with Logfire is two lines after you authenticate:

import logfire

logfire.configure()
logfire.instrument_pydantic_ai()

After that, each run shows up as a trace you can inspect, including every model call and tool call. If you already use a tracing tool for language models, the guide on Langfuse shows how traces help you understand and debug agent behaviour. Tracing is optional for a beginner, but turning it on early makes mysterious behaviour much easier to explain.

Recognising version 1 code

Because Pydantic AI moved to version 2 in June 2026, tutorials, forum answers and AI-generated snippets often show version 1 code. Recognising it saves hours. Here are the differences a beginner is most likely to hit.

The theme of version 2 is that many options once passed to the Agent constructor moved onto capabilities, reusable bundles of tools and behaviour that you pass as capabilities=[...]. You will not need them yet, but you will see the word often in current documentation. If you do inherit a version 1 codebase, the official advice is to upgrade to the latest 1.x release first, fix every deprecation warning, and only then move to version 2. Messages you saved with version 1 still load in version 2.

Pin the version in anything you deploy, for example pydantic-ai==2.52.*, because the project releases every day or two.

Putting it all together: a small support assistant

Let us combine everything into one program: typed output, a tool that reads dependencies, instructions, history, limits and a test. The scenario is a bookshop assistant that answers questions about orders and always returns a structured reply.

import asyncio
from dataclasses import dataclass

from pydantic import BaseModel, Field
from pydantic_ai import Agent, ModelRetry, RunContext, UsageLimits


class OrderDb:
    """A stand-in for a real database client."""

    ORDERS = {
        'A100': {'status': 'shipped', 'eta_days': 2},
        'A101': {'status': 'processing', 'eta_days': 5},
    }

    async def find(self, order_id: str) -> dict | None:
        return self.ORDERS.get(order_id)


@dataclass
class Deps:
    db: OrderDb
    customer: str


class Reply(BaseModel):
    answer: str = Field(description='a polite answer of at most three sentences')
    needs_human: bool = Field(description='true if a person should follow up')


agent = Agent(
    'openai:gpt-5.2',
    deps_type=Deps,
    output_type=Reply,
    retries=2,
)


@agent.instructions
def brief(ctx: RunContext[Deps]) -> str:
    return (
        'You are a support assistant for an online bookshop. '
        f'You are speaking to {ctx.deps.customer}. '
        'Use the lookup tool for any question about an order. '
        'If the order cannot be found, set needs_human to true.'
    )


@agent.tool
async def lookup_order(ctx: RunContext[Deps], order_id: str) -> str:
    """Look up an order by its identifier.

    Args:
        order_id: the order identifier, for example A100
    """
    if not order_id.startswith('A'):
        raise ModelRetry('Order ids start with the letter A, for example A100.')
    order = await ctx.deps.db.find(order_id)
    if order is None:
        return 'No such order.'
    return f"Status {order['status']}, arriving in {order['eta_days']} days."


async def main() -> None:
    deps = Deps(db=OrderDb(), customer='Mona')
    limits = UsageLimits(request_limit=10, total_tokens_limit=5000)
    history = []

    for question in ['Where is order A100?', 'And when will it arrive?']:
        result = await agent.run(
            question,
            deps=deps,
            message_history=history,
            usage_limits=limits,
        )
        history = result.all_messages()
        print(result.output)


if __name__ == '__main__':
    asyncio.run(main())

Read it from the top. OrderDb is the thing your tools need, and Deps bundles it with the customer's name. Reply is the shape of every answer, with a boolean so that calling code can route hard cases to a person. The agent is created once, with a retry budget of two.

The brief function builds the instructions at run time, using the customer's name from the dependencies. The lookup_order tool does the real work: it rejects a malformed id with ModelRetry, which sends the guidance back to the model, reads the database through ctx.deps, and returns a short string.

In main, the same deps object and the same limits are reused for both questions, and history carries the first exchange into the second, which is why "it" in "when will it arrive?" makes sense. The limits make sure that one runaway run cannot spend more than ten requests or five thousand tokens.

Now a test, so a change to the code is checked without a model:

from pydantic_ai import models
from pydantic_ai.models.test import TestModel

from assistant import Deps, OrderDb, agent

models.ALLOW_MODEL_REQUESTS = False


def test_assistant_returns_a_reply():
    with agent.override(model=TestModel()):
        result = agent.run_sync(
            'Where is order A100?',
            deps=Deps(db=OrderDb(), customer='Mona'),
        )
    assert isinstance(result.output.needs_human, bool)

Run it with pytest. It proves the wiring: the tool registers, the dependencies are read, the output type is satisfied. Then run python assistant.py with a real key to see real behaviour.

What you can now do, and what comes next

You can now install Pydantic AI, check that it works with no key, and talk to a real model through a provider-prefixed name. You can write instructions, ask for typed output as a Pydantic model, and rely on validation to keep the model honest. You can give the model tools, pass dependencies through the run context, hold a conversation by passing messages, stream a reply, set limits, read the common errors, and test the lot with TestModel. You can also spot version 1 code and translate it.

A short recap of the questions you should be able to answer without looking:

Mid-level takes every one of those topics further: the choice between output modes and when to use each, output validators, how retries really layer, human approval for risky tools, managing and trimming history, the five run methods in depth, connecting external tools through MCP, evaluating agent quality, and capabilities and hooks. If your agents will sit behind a web service, the guide on FastAPI covers the other half of that stack, and the guide on the Model Context Protocol explains the tool standard Pydantic AI can plug into.

Senior then covers what you own when agents are a platform for a team: the trust model for client-supplied history and approvals, secrets and per-user credentials, multi-tenant isolation, concurrency and rate limiting, fallback across providers, durable execution so a run survives a crash, cost control, version upgrades, and the cases where a different tool is the better choice.

Sources

• Pydantic AI overview
• Installation
• Troubleshooting
• Migration guide, version 1 to version 2
• Agents
• Dependencies
• Output
• Message history
• Retries
• Function tools
• Models overview
• OpenAI models
• Testing
• Command-line interface
• Logfire integration
• Version policy
• Changelog

This is the interview companion to the beginner guide. It assumes you have built at least one small agent. The aim is not to memorise phrases but to be able to explain, calmly and in your own words, what Pydantic AI does, why it is designed the way it is, and what goes wrong. Interviewers for AI engineering roles tend to probe three things: whether you understand the loop under the hood, whether you treat model output as untrusted, and whether you can debug when it misbehaves. Everything below is aimed at those three.

All answers target Pydantic AI 2.52, the current release at the time of writing. If an interviewer describes older behaviour, you can score points by noting what changed in version 2.

The thirty-second answer

Pydantic AI is a Python library for building agents on top of language models. An agent sends instructions and a question to a model, lets the model call Python functions you wrote as tools, validates everything with Pydantic, and returns a typed result in exactly the shape you declared.

It is a library rather than a service, so it runs inside your own application, and it works with many providers through a provider:model string such as openai:gpt-5.2. What sets it apart is that types do the work: the output type, the tool arguments and the dependencies are all ordinary Python types, so editors and type checkers understand them and bad model output is caught at the boundary.

If you are asked to say it even shorter: "typed, validated agents in plain Python, in the style of FastAPI".

The model you should have in your head

Be ready to draw this on a whiteboard. There are five nouns and one verb.

The nouns are the agent (the reusable, generic container Agent[DepsT, OutputT]), the model (the language model, named by a string), the tools (Python functions the model may call), the output (the typed final answer) and the dependencies (what your tools need, passed per run and read through ctx.deps). The verb is the run: one execution of the loop.

The loop is the most commonly asked-about part. The agent sends instructions, history and the prompt to the model. The model replies with text or with tool calls. The library validates the tool arguments, runs the tools, and sends the results back. This repeats until the model produces a valid output or a limit stops it. If validation fails at any point, the error is sent back to the model so it can correct itself, which is the library's built-in retry.

Two details raise the quality of your answer. First, structured output is normally delivered through a hidden output tool, so a validation failure on the answer looks like a tool retry. Second, a run is limited by default to 50 model requests, and the default retry budget is one.

Commands and snippets to have ready

You will rarely type these in an interview but you should recognise each one and say what it does.

pip install pydantic-ai
python -c "import pydantic_ai; print(pydantic_ai.__version__)"
clai --version
clai "Explain DNS in two sentences"
clai -m anthropic:claude-sonnet-4-6
clai -a support:agent

The first installs the bundle, the second checks the version, and the clai lines use the command-line client: a one-shot prompt, a chosen model, and a chat against your own agent defined as agent in support.py.

The code you should be able to write from memory is a minimal agent with typed output and one tool:

from dataclasses import dataclass
from pydantic import BaseModel
from pydantic_ai import Agent, RunContext


@dataclass
class Deps:
    db: object


class Answer(BaseModel):
    text: str
    confident: bool


agent = Agent(
    'openai:gpt-5.2',
    deps_type=Deps,
    output_type=Answer,
    instructions='Answer briefly. Use tools for facts.',
)


@agent.tool
async def get_price(ctx: RunContext[Deps], sku: str) -> str:
    """Return the price of a product.

    Args:
        sku: the product code
    """
    return '9.99'


result = agent.run_sync('How much is SKU 7?', deps=Deps(db=None))
print(result.output)
print(result.usage)

Be able to point at each line and say what it is for. deps_type is for the type checker only. @agent.tool takes a RunContext as its first parameter; @agent.tool_plain does not. result.output is an Answer. result.usage is a property in version 2, not a method.

And the testing snippet, which interviewers love because many candidates skip it:

from pydantic_ai import models
from pydantic_ai.models.test import TestModel

models.ALLOW_MODEL_REQUESTS = False

with agent.override(model=TestModel()):
    result = agent.run_sync('How much is SKU 7?', deps=Deps(db=None))

The classic traps and why they happen

These are the mistakes that come up most, with the reasoning behind each. Explaining the reason, not just the fix, is what distinguishes a strong answer.

Forgetting the provider prefix. In version 1 you could write Agent('gpt-5'). In version 2 that raises a UserError saying the model is unknown. The prefix tells the library which provider class to build. The fix is openai:gpt-5.

Calling result.usage() with parentheses. In version 2, usage is a property. Code written for version 1 fails when it calls the result. This is a good example to cite when asked about the version 2 migration.

Believing deps_type supplies the dependencies. It does not. It only tells the type checker what ctx.deps is. You still have to pass deps= on every run. The reason is that dependencies are per-run data, such as a user or a connection, and an agent is created once and shared.

Assuming the agent remembers. It does not. Each run starts clean. To continue a conversation you pass message_history=previous.new_messages(). The reason is design: the agent is stateless so that one instance can serve many users at once, and state is yours to store.

Using run_sync in a notebook. It raises RuntimeError: This event loop is already running, because a notebook already runs an event loop. Use await agent.run(...).

Trusting valid output as correct output. Pydantic checks shape and constraints, not truth. A well-formed answer can still be wrong. The reason is that a model generates plausible text; validation only restricts which text is accepted.

Streaming with tools. run_stream() treats the first output matching the output type as final, so tool calls after streamed text may never run. For a streamed tool loop use run_stream_events() or iter().

Tool docstrings left empty. The docstring becomes the tool description the model reads. A tool with no description is chosen badly or called with poor arguments. The reason is simple: the docstring is part of the prompt.

Missing extras after installing the slim package or upgrading. If you see ImportError: Please install the ... package, an optional dependency is missing. Version 2's default install no longer includes Bedrock, Groq, Mistral, Cohere, xAI or Hugging Face, so add them, for example pip install "pydantic-ai[groq]".

TLS errors in containers or behind a proxy. Since the move to a newer HTTP library, requests verify certificates against the operating system's store. A slim image or a corporate inspecting proxy may lack the right authority. Install the certificate into the system store or pass a configured HTTP client.

Common interview questions

What problem does Pydantic AI solve that calling a model API directly does not?

Calling an API directly gives you text. You then have to parse it, handle malformed replies, write the loop that runs tools and feeds results back, and keep it all type safe. Pydantic AI does that loop for you, validates every model output against Python types, retries with the validation error when the model gets it wrong, and presents a typed result. The benefit is not that the model is smarter; it is that the surrounding plumbing is correct and consistent.

Why is it described as "like FastAPI"?

Because the design philosophy is similar: plain Python, type hints as the source of truth, dependency injection through a context object, and decorators to register functions. An agent is meant to be created once at module level and reused, as you do with a FastAPI app. If you know FastAPI, you already know the shape of deps_type, RunContext and decorators.

What is the difference between instructions and system_prompt?

Instructions are re-evaluated on every run, and only the current agent's instructions are sent. If you reuse message history, the old instructions stored there are not resent. A system prompt is kept in the history and travels with it. The documentation recommends instructions unless you specifically need the prompt to persist in history. The practical advantage is that you can change behaviour by changing code and have even old conversations follow it.

How do you get structured output?

Pass output_type= a Pydantic model, a dataclass, a typed dictionary, a scalar, or a list of such types for a choice. The library gives the model a schema, validates the answer, and returns an instance. If validation fails, the error goes back to the model for another attempt. I would add field descriptions and constraints, such as ge and le, so that the rules I care about are enforced by validation.

What are the output modes?

The default is tool output: the schema is sent as a hidden tool. There is also native output, which uses a provider's own structured-output feature; prompted output, which describes the schema in the prompt and parses the text; and text output, which passes plain text through a function. As a beginner I use the default and only switch if a model handles it poorly.

What is a tool, and how does the model know about it?

A tool is a Python function registered with @agent.tool or @agent.tool_plain. The library builds a schema from the signature and the docstring and sends it to the model. The model can then request a call. The library validates the arguments before running the function, so invalid arguments never reach my code, and the result goes back to the model.

When do you use @agent.tool and when @agent.tool_plain?

@agent.tool when the function needs the run context, for example to read dependencies. Its first parameter is ctx: RunContext[Deps]. @agent.tool_plain when it needs nothing from the context. The context parameter is hidden from the model's view of the tool.

What is RunContext?

An object the library passes to functions that ask for it. It carries deps, the usage so far, the message history, the current retry number and the run step. It is parameterised, RunContext[Deps], so the type checker knows what ctx.deps is.

Why use dependency injection instead of global variables?

Globals make tools hard to test and unsafe when many requests run at once, because they share state. With injection, each run carries its own database client, user or key, and tests can pass a fake. It is the same reason FastAPI uses dependencies.

How does a conversation work, given that agents are stateless?

I keep the messages and pass them back. result.new_messages() gives the messages from the last run, and result.all_messages() gives everything including earlier history. I pass them as message_history= on the next call. To survive a restart I serialise them with ModelMessagesTypeAdapter and to_json, and store them wherever suits, keyed by a conversation id.

What are the downsides of passing the whole history every time?

It grows, so each turn costs more and eventually exceeds the context window. A real application trims or summarises old messages. Also, history is a trust boundary: if a client can send me history, it could forge tool calls, so I would only accept history that I stored myself.

What are the five ways to run an agent?

run (async), run_sync (blocking), run_stream (stream text or output), run_stream_events (stream every event) and iter (step through the graph nodes). Beginners mostly use the first three.

How do you stop an agent looping forever?

There is a default cap of 50 model requests per run, which raises UsageLimitExceeded. I would set my own UsageLimits for requests, tool calls and tokens, and fix the cause, which is usually a tool that never gives the model what it needs. There is also a cost limit, but it is best-effort rather than a billing guarantee, so I would pair it with a provider spend cap.

What does "retries" mean here?

When a tool raises ModelRetry, or the model's arguments or output fail validation, the error is sent back to the model for another attempt. The default budget is one. You raise it on the agent with retries=, per tool with retries= on the decorator, or per output. When the budget is spent you get UnexpectedModelBehavior.

How would you debug "Tool exceeded max retries"?

I would wrap the run in capture_run_messages() and print the messages. That shows what arguments the model sent and what error it was given. Usually the docstring is vague or the argument type is awkward. I would fix the description, add an example, and only then consider raising the retry count.

How do you test an agent without paying for model calls?

Set models.ALLOW_MODEL_REQUESTS = False so accidental real calls fail, and use agent.override(model=TestModel()) inside a with block. TestModel calls tools and returns a valid placeholder. For scripted replies there is FunctionModel. These tests verify wiring, not model quality, so I would add a small set of real-model evaluations separately.

How do you choose a model provider, and what about data residency?

The library supports many providers behind the same API, so I choose on quality, price and where the data may go. For employers with residency rules in the Gulf or Egypt, I would use a provider region that satisfies the rule, or a model I host myself, which I reach through an OpenAI-compatible endpoint or the ollama: prefix. The library itself sends nothing anywhere except to the provider I configure.

What changed in version 2?

The headline is that capabilities became the main extension mechanism, and several Agent arguments moved onto them. Practical changes for a beginner are that the provider prefix is required, result.usage is a property, the default install is slimmer, the bare openai: prefix means the Responses API, and the MCP classes were unified into MCPToolset. The advice for upgrading is to go to the latest 1.x first, fix deprecation warnings, then move to 2.

How do you keep up with a library that releases this often?

Pin an exact version, read the compatibility notes on each release, run tests with warnings visible, and upgrade deliberately. The project promises no intentional breaking changes in minor releases, but defaults and trace attributes are allowed to change, so a lockfile and a test suite are the real protection.

Is clai web safe to run on a server?

No. It is for local development only. Earlier versions had issues where a malicious web page could reach a locally running instance, which were fixed in releases 2.28 and 2.30, but it is still not designed to face users. For production I would build my own endpoint around the agent.

What a strong answer covers that a weak one misses

Weak answers describe the library as "a wrapper around OpenAI". Strong answers mention provider independence, the validation-and-retry loop and the typed result. Weak answers say "the agent remembers the conversation"; strong ones say it is stateless and history is passed in. Weak answers treat tool docstrings as comments; strong ones know they are part of the prompt. Weak answers stop at "I tested it by running it"; strong ones describe TestModel and ALLOW_MODEL_REQUESTS. And strong answers always mention limits: the request cap, the retry budget, and the fact that cost limits are best-effort.

If you are asked a question you do not know, say what you would check. "I would look in the official docs for the version I am pinned to, because this library changes quickly" is a good answer, and it is true.

Questions to ask the interviewer

Good questions show that you think about operation, not only demos. Ask which provider and region the team uses and why. Ask how they test agents and whether they evaluate with real models. Ask how they pin and upgrade the library. Ask what tools their agents are allowed to call and who reviews that list. Ask whether traces contain customer data and where they are stored. Ask what happens when a run exceeds its limits. A team that has answers to these has already met the problems this guide describes.

For neighbouring topics, the guides on Langfuse for tracing, FastAPI for serving, and the Model Context Protocol for external tools are the natural next reads.

Sixty-second self-test

Answer each aloud, then check.

1. Name the five nouns. Agent, model, tool, output, dependencies.
2. Write the model string for Anthropic's Sonnet 4.6. anthropic:claude-sonnet-4-6, with the provider prefix.
3. What does deps_type do? It informs the type checker only; you must pass deps= yourself.
4. How do you continue a conversation? Pass message_history= from new_messages() or all_messages().
5. Is result.usage a method or a property in version 2? A property.
6. What is the default request limit per run? Fifty model requests.
7. What is the default retry budget? One.
8. How do you test with no model? agent.override(model=TestModel()) and ALLOW_MODEL_REQUESTS = False.
9. Where does the model see a tool's description? In the docstring.
10. Why can a streamed run skip a tool call? run_stream treats the first matching output as final.
11. How do you run an agent in a notebook? await agent.run(...), not run_sync.
12. Can you expose clai web to users? No, local development only.

If you got ten or more without hesitating, you are ready for a beginner-level conversation. Where you stumbled, reread the matching section of the detailed guide, rerun the example, and say the answer aloud once more.

This file collects the mistakes beginners actually make with Pydantic AI, each with the symptom you will see, the cause underneath, and the fix. Then come the habits worth adopting on your first day and a short checklist. It targets Pydantic AI 2.52. A lot of what you find online was written for version 1, so several of the mistakes below are simply version 1 habits meeting version 2.

Read the section titles first. If one of them describes what is happening to you right now, jump straight to it.

Mistakes beginners actually make

Writing the model name without a provider

Symptom. You write Agent('gpt-5.2') and get a UserError beginning "Unknown model".

Cause. In version 1 a bare name was guessed to belong to OpenAI. Version 2 removed the guess, so the library needs to be told which provider class to build.

Fix. Use provider:model: Agent('openai:gpt-5.2'), Agent('anthropic:claude-sonnet-4-6'), Agent('google:gemini-3-flash-preview'). If the old prefixes google-gla: or google-vertex: appear in an example, use google: and google-cloud: instead.

Forgetting the API key, or putting it in the code

Symptom. A UserError that says to set the OPENAI_API_KEY environment variable, or, worse, no error because you hard-coded the key and pushed it to a public repository.

Cause. Providers read credentials from environment variables, and beginners try to shortcut that by pasting the key into a file.

Fix. Export the variable in your shell, or load it from a local file that is listed in .gitignore. If a key ever reaches version control, revoke it at the provider and create a new one; deleting the commit is not enough, because copies persist. The error message itself mentions the keyless alternative, Agent('test'), which is a good way to keep working while you sort out credentials.

Calling result.usage() or using old result attributes

Symptom. An AttributeError or a TypeError about something not being callable when you print token usage.

Cause. In version 2, result.usage is a property, not a method. Very old examples use result.data, which is now result.output.

Fix. Write result.usage and result.output, with no parentheses on the first. When you copy a snippet, check the version it was written for before trusting any attribute name.

Thinking deps_type creates the dependencies

Symptom. A tool raises an AttributeError when it touches ctx.deps, or the value is not what you expect.

Cause. deps_type=Deps is for the type checker only. It does not build anything. The actual object comes from deps= on the run call, and you forgot it, or you passed it to the constructor instead.

Fix. Pass deps=Deps(...) on every run, run_sync and stream call. Make it a habit to write the type and the first call together, so they never drift apart.

Using the wrong decorator

Symptom. A tool is rejected at registration, or the model is shown an argument named ctx that you never meant it to supply.

Cause. @agent.tool expects the first parameter to be a RunContext. @agent.tool_plain expects none. Mixing them up produces confusing results.

Fix. Ask one question: does this function need dependencies or other run information? If yes, use @agent.tool with ctx: RunContext[Deps] first. If not, use @agent.tool_plain.

Leaving the docstring empty or vague

Symptom. The model never calls your tool, calls it at the wrong time, or fills in arguments badly.

Cause. The docstring is the tool's description, and the parameter documentation guides the arguments. The model has nothing else to go on.

Fix. Write what the function does, when to use it, and one example of each argument. Use the Args section of a Google-style docstring. When the model misuses a tool, improve the docstring before you touch any code.

Expecting memory without passing history

Symptom. The second question loses context, and the answer says it does not know what "it" refers to.

Cause. Agents are stateless. Each run starts from scratch.

Fix. Pass message_history=previous.new_messages() on the next call, or keep all_messages() in a list as in the chat-loop example. Store the messages yourself if the conversation must outlive the process.

Letting history grow without limit

Symptom. Replies get slower and more expensive as a chat goes on, and eventually the provider rejects a request as too long.

Cause. Every message is sent again on each turn, so cost and size grow with the length of the conversation.

Fix. At the beginner stage, cap the number of turns you keep, or start a fresh conversation for a new topic. Later you will summarise or trim history in a more careful way, which the Mid-level guide covers.

Running run_sync where an event loop already exists

Symptom. RuntimeError: This event loop is already running, usually in Jupyter, Colab or inside an async web handler.

Cause. run_sync starts its own loop, and it cannot start one inside a loop that is already running.

Fix. In a notebook or any async def, write result = await agent.run(...). Reserve run_sync for plain scripts and tests.

Calling run_sync from inside a tool

Symptom. A UserError explaining that Agent.run_sync() cannot be used inside a synchronous tool because it can deadlock.

Cause. You tried to let one agent call another from a normal def tool by running it synchronously.

Fix. Make the tool async def and await other_agent.run(...). Pass usage=ctx.usage so the token counts add up across both agents.

Streaming and losing the tool call

Symptom. The streamed text appears, but a tool the model was about to call never runs.

Cause. run_stream treats the first output that matches your output type as final. If the model emits text first, that text may be taken as the answer.

Fix. For streaming with tools, use run_stream_events() or iter(), which keep the loop going. For a plain chat with no tools, run_stream is fine.

Believing valid means true

Symptom. The answer parses perfectly and is wrong.

Cause. Validation checks types and constraints. It does not check facts.

Fix. For anything that matters, verify against a trusted source inside a tool, add constraints that reject impossible values, and keep a person in the loop for risky decisions. Ask for a confident flag or a needs_human flag in the output type so that calling code can route uncertain cases.

Hitting "exceeded max retries" and just raising the number

Symptom. UnexpectedModelBehavior: Tool 'x' exceeded max retries count of 1, or "Exceeded maximum output retries".

Cause. The model kept sending invalid arguments, or an answer that never satisfied your output type. The default budget is one retry for tools.

Fix. Look first, raise second. Wrap the run in capture_run_messages(), print the messages, and see what the model sent and what error came back. Then fix the cause: a clearer docstring, a simpler output type, a gentler constraint. Raising retries is the last step, not the first, because it only gives a confused model more chances to be confused.

A runaway loop and a surprise bill

Symptom. A run takes a long time, spends many tokens, and ends in UsageLimitExceeded about the 50-request limit.

Cause. A tool keeps returning something the model cannot use, so it keeps asking again. Fifty requests is the default ceiling, and fifty large requests are not cheap.

Fix. Find the loop in the messages. Make the tool's return value informative, especially for the "not found" case. Set your own UsageLimits with small numbers for development, and set a spend cap at your provider as a second layer.

Missing packages after installing or upgrading

Symptom. ImportError: Please install the ... package to use the ... provider.

Cause. You installed pydantic-ai-slim, or upgraded to version 2 where the default install dropped Bedrock, Groq, Mistral, Cohere, xAI, Hugging Face and some others.

Fix. Install the extra by name: pip install "pydantic-ai[groq]" or pip install "pydantic-ai-slim[openai]".

Certificate errors in containers and behind proxies

Symptom. SSL: CERTIFICATE_VERIFY_FAILED or a "self-signed certificate in chain" message, often in a Docker image or on a corporate network, while a browser on the same machine works.

Cause. Recent versions verify certificates against the operating system's trust store. A minimal image has few authorities, and a company proxy that inspects traffic re-signs it with its own authority that the system may not trust.

Fix. Install ca-certificates and your organisation's root certificate into the system store, or pass a configured HTTP client to the provider. Do not turn verification off; that removes the protection that the check exists to give. If you build images, the Docker guide shows how to add system packages cleanly.

Pasting version 1 code into a version 2 project

Symptom. A pile of unrelated errors: mcp_servers, MCPServerStdio, builtin_tools and history_processors are all unrecognised.

Cause. Those names were removed or moved in version 2. Many arguments that once went on Agent(...) now go through capabilities=[...], and the MCP classes became MCPToolset.

Fix. Check the migration guide in the official documentation, or ask your editor's type checker, which will flag every removed name. Treat any code sample older than the June 2026 release with suspicion.

Using clai web as if it were a product

Symptom. You bind the development web interface to a public address so that colleagues can try your agent.

Cause. It is convenient, and it looks finished.

Fix. Do not. It is for local development only. Earlier versions had security flaws that let a malicious page drive a locally running instance, which were fixed in 2.28 and 2.30, and even a patched copy is not meant to face users. Build a proper endpoint around the agent for anyone else.

Giving a tool more power than it needs

Symptom. An agent deletes, sends or changes something you did not expect.

Cause. The model chose to call a tool that was available. It was never told not to, and the tool did not check.

Fix. Start with read-only tools. Validate inputs inside the tool, using your own code. Make destructive actions require explicit approval, which the Mid-level guide shows. Remember that anything the model can call, it will eventually call at the wrong moment.

Putting private data into prompts and traces without thinking

Symptom. A privacy review asks where customer data went, and you do not know.

Cause. Everything you put into a prompt is sent to the provider, and tracing may store prompts and tool arguments in a third-party service.

Fix. Decide in advance which data may be sent, to which provider and region. For employers with data-residency rules, choose a regional endpoint or a model you host. Configure tracing so it excludes message content when the data is sensitive.

Habits worth adopting on day one

Start every project with the test model

Begin with Agent('test') or agent.override(model=TestModel()), and build the wiring first: tools, deps, output type. Switch to a real model once the plumbing passes a test. You will waste far fewer tokens, and when something fails you will know whether the cause is your code or the model.

Set ALLOW_MODEL_REQUESTS = False in your test setup

One line, models.ALLOW_MODEL_REQUESTS = False, turns any accidental real model call in a test into an immediate, readable failure. It protects your wallet and keeps your test suite fast and deterministic.

Pin the version

The library releases every day or two. Write pydantic-ai==2.52.* in your requirements, or use a lockfile such as the one uv produces, and upgrade on purpose. Read the compatibility notes on the release page before you do. The project avoids breaking the documented API in minor releases, but defaults and trace attribute names are allowed to change.

Keep the agent at module level

Create the agent once, near the top of a module, and call it from functions. This is what the documentation intends, and it is what makes agent.override possible in tests. An agent built inside a function is hard to test and wasteful to recreate.

Write instructions like you would brief a new colleague

Short, specific and testable. "Answer in at most three sentences. If you do not know, say so" beats "be helpful". Keep a handful of fixed test questions, and rerun them whenever you change a sentence, so you can see what each edit did.

Give every field a description

On output models, Field(description=...) is read by the model. A field named score with the description "rating from 1 to 5" is used far better than the bare name. Add ge and le constraints wherever a range is known.

Return helpful strings from tools

Tools talk to a model, not a person. "No such order. Order ids look like A100." is much more useful than an empty string or an exception, because the model can act on it. For a clearly wrong argument, raise ModelRetry with a plain explanation.

Prefer async tools for anything that waits

If a tool calls a network service or a database, make it async def. Synchronous tools work, since the library runs them in threads, but async tools scale better and avoid blocking. Remember that you cannot forcibly stop a synchronous tool once it has started.

Look at the messages before guessing

When something is odd, print result.all_messages(), or wrap the run in capture_run_messages() if it raises. The conversation between your code and the model is the ground truth. Most mysteries dissolve within a minute of reading it.

Set limits while developing

Use UsageLimits with small values during development, so a bug costs cents, not dollars. You can relax the numbers later. At the same time, set a spending limit in your provider's console. A cost limit inside the library is an estimate, not a guarantee.

Turn on tracing early

Even a free tier of an observability tool, or a local OpenTelemetry collector, repays the setup within a day. Seeing each model call, each tool call and each token count turns guesswork into reading. The guide on Langfuse shows what good traces look like for language-model applications.

A beginner's debugging routine

When a run does something you did not expect, work through the same short routine rather than changing things at random.

First, reproduce it with the test model. If it still fails, the problem is in your wiring: a missing deps=, the wrong decorator, an output type that cannot be satisfied. Fix that before involving a real model.

Second, if it only fails with a real model, print the messages. Look at the order: your request, the model's response, any tool calls and their returns, any retry messages. Ask what the model saw, not what you meant it to see.

Third, change one thing. Edit one sentence of the instructions, or one line of a docstring, or one constraint, and rerun the same prompt. If you change three things you will not know which helped.

Fourth, if the behaviour is random, lower the temperature in model_settings to make runs repeatable while you investigate, then raise it again if you want variety.

Fifth, check the version. If an attribute, import or argument is not found, compare the snippet with the migration notes. Half of the "bugs" a beginner meets in 2026 are version 1 code meeting version 2.

Practice exercises

Short exercises that cement the guide. Do them in order.

1. Smoke test. Install the library in a fresh environment and run Agent('test'). Print the version.
2. Typed answer. Write a model with three fields, one with a constraint, and run it against five sentences.
3. One tool. Add a tool that looks things up in a dictionary. Read the tool call in all_messages().
4. Dependencies. Move the dictionary into a dataclass passed as deps, and read it through ctx.deps.
5. Conversation. Build a chat loop that keeps history, and save it to disk so it survives a restart.
6. Limits. Add UsageLimits and trigger UsageLimitExceeded deliberately.
7. Test. Write a test with TestModel and ALLOW_MODEL_REQUESTS = False.
8. Translate. Take an old tutorial snippet and rewrite it for version 2.

Checklist

Before you call a small agent finished, go down this list.

• The model string has a provider prefix.
• The API key is in the environment, not in the code or the repository.
• The version is pinned.
• The agent is created once at module level.
• Instructions are short, specific and testable.
• The output type has field descriptions and sensible constraints.
• Every tool has a docstring that says what it does and what its arguments mean.
• Tools return helpful strings, and raise ModelRetry for fixable mistakes.
• Dependencies are passed with deps= on every call.
• Conversation history is passed explicitly, and its growth is capped.
• Limits are set, and a spend cap exists at the provider.
• There is at least one test using TestModel, with real requests disabled.
• Nothing sensitive is sent to a provider or recorded in traces without a decision.
• clai web is not exposed to anyone but you.

If every line is ticked, you have a solid foundation. The Mid-level guide builds on it with output modes, validators, approval for risky tools, MCP, evaluation and capabilities.

Where this picks up

You already build agents with typed output, a function tool or two, and a dependencies dataclass. This level is for the moment an agent leaves your laptop and becomes something other people depend on: it has to survive bad model output, cost a predictable amount, be testable without a network, be observable, and keep working when the library moves under it. This guide is written against Pydantic AI 2.52 (the V2 line, released as stable in June 2026), so every snippet uses current names.

Each section states the problem first and ends with a Try it that you can run against a real project in a few minutes.

The run loop, precisely

Beginner described an agent as a thing that calls a model and runs tools. To debug it at work you need the exact sequence, because every limit, retry and hook attaches to one step of it.

A run is a small state machine, implemented on top of pydantic_graph. It has three node types that repeat: a request node builds the next message to the model and sends it, a call-tools node inspects the response, and an end node holds the final result. The model response contains parts: text, tool calls, sometimes thinking. For each tool call Pydantic validates the arguments against the tool's JSON schema, runs the function, and packages the return value (or a retry request) into the next request. The loop ends when the model produces an output that validates against output_type, or when something raises.

UserPromptNode  ->  ModelRequestNode  ->  CallToolsNode  ->  End
                         ^                     |
                         |   tool results /    |
                         +---- retry prompts --+

There are five ways to drive that machine, and choosing the wrong one is a common source of "my tool never ran":

Two details about the end of the loop matter in production. First, end_strategy decides what happens when the model returns a valid output tool call and other function tool calls in the same response. V2 changed the default from 'early' to 'graceful': the other function tools now run, in the order the model emitted them, so their side effects happen. If you upgraded from V1 and a tool such as "send email" suddenly fires on runs where it used to be skipped, this is why. Set end_strategy='early' to restore the V1 behaviour, or 'exhaustive' to run everything.

Second, every run has a run_id, and a conversation that spans many runs has a conversation_id. You pass the latter to run() and then filter traces and logs by it. Treat it as the key you also use for your history store, which we get to shortly.

The result object gives you the pieces you will use constantly. result.output is typed from output_type. result.usage is a property in V2 (a RunUsage with input_tokens, output_tokens, requests, tool_calls and cost); calling it as usage() is a V1 habit that now fails. result.all_messages() returns the full history including this run's, and result.new_messages() only what this run added.

Instructions, settings and what the model actually sees

The prompt you send is assembled from several layers, and at this level you stop guessing which layer a surprise came from.

Instructions versus system prompt. instructions can be given as an argument, with @agent.instructions for dynamic text that reads RunContext, or per run with instructions=. They are re-evaluated on every run, and only the current agent's instructions are sent. If you load a message_history produced by a different agent, that agent's instructions do not come along. A system_prompt, by contrast, is stored as a SystemPromptPart in the history and persists when you replay it. The official guidance is to prefer instructions unless you specifically want the prompt frozen into the stored conversation. The practical consequence: when you change the instructions and deploy, old conversations pick up the new behaviour immediately, which is usually what you want and occasionally a surprise.

Static instructions are ordered before dynamic ones. That is deliberate, because providers cache on prompt prefixes, and a stable prefix is what makes caching hit. If you put a timestamp at the top of a dynamic instruction function, you have not broken correctness but you have likely collapsed your cache hit rate. Keep volatile content at the end, and keep it out of the static block.

from dataclasses import dataclass
from datetime import date
from pydantic_ai import Agent, RunContext

@dataclass
class Deps:
    tenant: str
    user_name: str

agent = Agent(
    'anthropic:claude-sonnet-4-6',
    deps_type=Deps,
    instructions='You are a support assistant. Be concise and cite order ids.',
)

@agent.instructions
def per_request(ctx: RunContext[Deps]) -> str:
    # dynamic, so it is ordered after the static text above
    return f'The customer is {ctx.deps.user_name}. Today is {date.today():%Y-%m-%d}.'

Model settings merge in layers. ModelSettings is a TypedDict with fields such as max_tokens, temperature, timeout, parallel_tool_calls, tool_choice, thinking, extra_headers and extra_body. Settings can be attached at four levels, and they merge with the later level winning: the model, then the agent, then capabilities, then the individual run. Each level can also be a callable that receives the run context and returns settings, which lets you vary temperature or token budget per tenant. Provider-specific subclasses such as GoogleModelSettings or OpenAIChatModelSettings carry options that only that API understands.

from pydantic_ai import Agent

agent = Agent('openai:gpt-5.2', model_settings={'temperature': 0.0, 'max_tokens': 800})

# the run-level layer wins for max_tokens, temperature is inherited
result = agent.run_sync('Summarise this ticket.', model_settings={'max_tokens': 200})

Thinking has its own knob. You can set it in model_settings={'thinking': 'high'} or attach the Thinking(effort='high') capability. The accepted values are True, False, and the effort levels 'minimal', 'low', 'medium', 'high' and 'xhigh'. If you are on Anthropic, note that the old budget_tokens style of extended thinking is deprecated on Opus 4.6 and removed on Opus 4.7 and Sonnet 5 and later; adaptive thinking through these settings is the supported route.

Model strings changed meaning in V2. A string is always '<provider>:<model>', and the prefix-less V1 form raises UserError: Unknown model. Two prefixes deserve a note. The bare openai: prefix now selects the Responses API (OpenAIResponsesModel); use openai-chat: if you need Chat Completions, which is what many OpenAI-compatible gateways implement. And Google's old google-gla: and google-vertex: prefixes became google: and google-cloud:. If you point at a gateway or a self-hosted OpenAI-compatible server, build the model explicitly so there is no ambiguity:

from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    'my-served-model',
    provider=OpenAIProvider(base_url='http://llm-gateway.internal:4000/v1', api_key='sk-internal'),
)

If your gateway is LiteLLM, the LiteLLM guide covers running it; Pydantic AI treats it as just another OpenAI-compatible base URL, or you can use the litellm: prefix.

Tools in depth: schemas, retries and failure

A tool is the part of your agent most likely to be called with garbage. The model chooses the arguments, so a tool is an untrusted API endpoint with a very friendly caller. Treat its definition as an interface contract.

The schema is your docstring. Pydantic AI builds the tool's JSON schema from the function signature and takes descriptions from the docstring (Google, NumPy and Sphinx styles are all parsed). The first paragraph becomes the tool description and each argument line becomes a parameter description. A tool called get(id: str) with no docstring forces the model to guess; a tool with a precise description and typed parameters such as Literal['open', 'closed'] or a constrained Annotated[int, Field(ge=1, le=100)] gets better calls and cheaper retries, because validation rejects a bad value before your code runs.

from typing import Annotated, Literal
from pydantic import Field
from pydantic_ai import Agent, RunContext, ModelRetry

agent = Agent('openai:gpt-5.2', deps_type=Deps)

@agent.tool(retries=3)
async def find_orders(
    ctx: RunContext[Deps],
    status: Literal['open', 'shipped', 'cancelled'],
    limit: Annotated[int, Field(ge=1, le=50)] = 10,
) -> list[dict]:
    """Find this customer's orders.

    Args:
        status: Only return orders in this state.
        limit: Maximum number of orders to return.
    """
    rows = await ctx.deps.db.orders(ctx.deps.tenant, status, limit)
    if not rows:
        raise ModelRetry(f'No {status} orders. Try a different status.')
    return rows

The retry budget is small by default. Every retry path draws from a budget of one unless you raise it. When the model sends invalid arguments, or your code raises ModelRetry, Pydantic AI sends a RetryPromptPart back and the model tries again. If the budget runs out you get UnexpectedModelBehavior: Tool 'find_orders' exceeded max retries count of 1. You can set the budget agent-wide with Agent(retries=3) or split it with retries={'tools': 3, 'output': 2}; you can override it per tool with @agent.tool(retries=N) or Tool(max_retries=N), per output tool with ToolOutput(T, max_retries=N), and per run with agent.run(..., retries={...}). When several are set, the most specific wins: per-tool beats per-toolset, then an override block, then the per-run argument, then the agent, then the built-in default. Raise budgets deliberately, because each retry is a full model request that you pay for.

Not every failure is the model's fault. ModelRetry says "you called me wrongly, try again", which is right for a missing record or a bad argument. It is wrong for "the database is down". For that, raise ToolFailed, which reports the failure to the model without consuming the retry budget, so the model can explain the problem or choose another path. It applies to function tools only. And for genuinely unexpected exceptions, let them propagate: your run fails loudly rather than the model inventing an answer around an error you did not model.

Timeouts are per attempt and per tool. Agent(tool_timeout=...) sets a default and a tool can set its own timeout=. On expiry the model receives a retry prompt reading Timed out after N seconds. There is a trap hidden here that Beginner could not show you: a plain def tool is run in a thread, and a thread cannot be forcibly stopped. The timeout lets the run continue, but the function keeps running and any side effects it performs are not rolled back. Prefer async def for anything that does I/O, and design side-effecting tools to be idempotent.

Concurrency inside a response. Models often return several tool calls at once and Pydantic AI runs them concurrently. In V2, sequential=True on one tool is a barrier for that tool only, not a switch that serialises the whole batch. If you need every tool in a run to execute one at a time, use agent.parallel_tool_call_execution_mode('sequential'), or set ModelSettings(parallel_tool_calls=False) to ask the provider not to emit parallel calls in the first place.

When a tool misbehaves in testing, capture_run_messages() is your first instrument. Wrap the run in it and you see the exact request and response parts, including the retry prompts, which tells you whether the model or your validation is the one disagreeing.

from pydantic_ai import capture_run_messages, UnexpectedModelBehavior

with capture_run_messages() as messages:
    try:
        agent.run_sync('Find my shipped orders', deps=deps)
    except UnexpectedModelBehavior:
        for m in messages:
            print(m)

In V2 this also captures a partial request or response from an interrupted run, marked state='interrupted', which is useful after a cancellation or timeout.

Output modes, validators and streaming side effects

Structured output looks like one feature but is really four mechanisms, and which you choose changes reliability, latency and which models you can use.

You normally pass the type itself and let the default apply. You reach for the wrappers when you need to change a name or description, set a per-output retry budget, or switch mode:

from pydantic import BaseModel
from pydantic_ai import Agent, NativeOutput, ToolOutput

class Fruit(BaseModel):
    name: str
    color: str

class Vehicle(BaseModel):
    name: str
    wheels: int

# several shapes, each its own output tool with its own retry budget
tool_agent = Agent('openai:gpt-5.2', output_type=[ToolOutput(Fruit, max_retries=3), Vehicle])

# provider-native structured output
native_agent = Agent('openai:gpt-5.2', output_type=NativeOutput([Fruit, Vehicle]))

If extraction fails often, try NativeOutput first on a model that supports it, and simplify the schema second. Deeply nested optional unions are hard for any model, and Exceeded maximum output retries (N) is nearly always a schema that is too clever.

Output validators are for rules the type system cannot express: a referenced id must exist, a total must match line items. Define one with @agent.output_validator; it receives the run context and the parsed output, may be async, and raises ModelRetry to send the model a corrective message.

@agent.output_validator
async def check_order_exists(ctx: RunContext[Deps], out: Invoice) -> Invoice:
    if not await ctx.deps.db.order_exists(out.order_id):
        raise ModelRetry(f'Order {out.order_id} does not exist. Use an id from the tool results.')
    return out

Streaming adds a hazard. When you stream structured output, the validator and any output function can be invoked repeatedly on partial data. Output functions are called multiple times while streaming, so a side effect inside one (writing a row, sending a message) will happen several times. The guard is ctx.partial_output: return early when it is true, and do the real work only on the final call.

async def save_summary(ctx: RunContext[Deps], summary: Summary) -> Summary:
    if ctx.partial_output:
        return summary
    await ctx.deps.db.insert_summary(summary)
    return summary

Finally, an output type of a union is best written as a list, output_type=[Box, str], because type checkers handle the list form better than Box | str.

Dependencies, history and the trust boundary

An agent is built once and reused, like a FastAPI app. Everything that varies per request therefore has to travel in deps, and everything that persists between requests has to live in your own store. Pydantic AI is a library, not a server: it keeps no state between runs.

Deps are runtime objects. deps_type= on the agent is only for the type checker; the real object is passed as deps= on each run and reached as ctx.deps in tools, instructions, validators and hooks. A frozen dataclass holding the tenant, the authenticated user, and clients (database pool, HTTP client) is the idiom. Build the heavy clients once at startup and put references in the per-request deps, so a request does not open new connections.

Conversations are lists of messages you store. After a run, result.new_messages() holds only what that run added. Append them to your store for the conversation, and on the next turn pass the stored list as message_history=. To persist, serialise with the type adapter rather than hand-rolling JSON, because messages are a tagged union of many part types:

from pydantic_core import to_json
from pydantic_ai import ModelMessagesTypeAdapter

def save(conversation_id: str, messages) -> None:
    store.put(conversation_id, to_json(messages))   # bytes of JSON

def load(conversation_id: str):
    blob = store.get(conversation_id)
    return ModelMessagesTypeAdapter.validate_json(blob) if blob else []

Histories serialised under V1 still deserialise under V2 through the same adapter, so a migration does not strand stored conversations. Two cautions follow from how history works. Because instructions are not carried in history, changing them changes the behaviour of every stored conversation on its next turn. And a long conversation grows without bound, so plan trimming; the ProcessHistory capability is the V2 place to filter or summarise messages before each request.

Tenancy follows the same logic. Put the tenant identifier in deps, tag the run with metadata={'tenant': ...} so traces can be filtered, store history under a tenant-and-conversation key, and filter which tools a caller may see from the deps rather than trusting the prompt to refuse.

Capabilities and hooks: the V2 extension point

In V1 you configured behaviour with a long list of Agent(...) arguments. In V2 most of those moved onto capabilities, reusable bundles of tools, lifecycle hooks, instructions and settings that you pass as capabilities=[...]. If you can read this section you can read most of the V2 migration guide.

The map from old arguments to capabilities is regular: history_processors became ProcessHistory, prepare_tools became PrepareTools, builtin_tools became NativeTool (and the module builtin_tools became native_tools), instrument became Instrumentation, and the agent-level event_stream_handler became ProcessEventStream. Even Thinking, web search and MCP access are capabilities: Thinking(effort='high'), WebSearch(), WebFetch() and MCP(url=...).

For your own cross-cutting behaviour, the quickest tool is Hooks, which lets you attach functions to lifecycle events with decorators. Typical uses are logging every model request, redacting content, adding a header, or recording tool latency.

from pydantic_ai import Agent
from pydantic_ai.capabilities import Hooks, Thinking

hooks = Hooks()

@hooks.on.before_model_request
async def log_request(ctx, request_context):
    print('request step', ctx.run_step)
    return request_context

agent = Agent(
    'openai:gpt-5.2',
    capabilities=[hooks, Thinking(effort='medium')],
)

The event names follow a pattern: before_*, after_*, wrapping * forms and *_error variants, over run, node run, model request, tool validation, tool execution, output validation and output processing, plus an event hook. A hook accepts timeout=, and exceeding it raises HookTimeoutError. One subtle rule worth remembering: raising ModelRetry from a before_model_request hook is not retried; it propagates as an error. If you want to influence the model after a response, use after_model_request or wrap the call with wrap_model_request.

When a hook becomes a reusable unit with its own tools or instructions, build a capability. The declarative form needs an id and a description, and then accepts tools just like an agent:

from pydantic_ai.capabilities import Capability

ops = Capability(id='ops', description='Operational lookups', instructions='Prefer the ops tools for status questions.')

@ops.tool_plain
def service_status(name: str) -> str:
    """Return the current status of a named service."""
    return f'{name}: ok'

Setting defer_loading=True makes a capability on-demand: it stays collapsed in the prompt until the model decides to load it. That keeps tool definitions out of every request, which matters for cost once you have dozens of tools. There is also a ToolSearch capability for the same reason. For subclassing, AbstractCapability exposes the full hook set.

Limits, cost and the retry layers that multiply

A loop that calls a paid API needs limits, and the defaults will not match your budget. The first thing to know is that one exists: every run has a default request_limit of 50 model requests. A tool loop that never converges stops there with UsageLimitExceeded: The next request would exceed the request_limit of 50. That is a fuse, not a budget.

UsageLimits lets you express the budget you actually have:

from decimal import Decimal
from pydantic_ai import Agent, UsageLimits, UsageLimitExceeded

agent = Agent('anthropic:claude-sonnet-4-6')

limits = UsageLimits(
    request_limit=15,
    tool_calls_limit=30,
    total_tokens_limit=60_000,
    cost_limit=Decimal('0.50'),
)

try:
    result = agent.run_sync('Triage this incident report...', usage_limits=limits)
except UsageLimitExceeded as exc:
    print('stopped:', exc)

Other fields are input_tokens_limit, output_tokens_limit, per_request_input_tokens_limit and count_tokens_before_request for providers that can count ahead. cost_limit is calculated from bundled price data (the genai-prices package), so it is best-effort, not a billing guarantee: prices change, new models may be missing (you get a CostNotFoundWarning), and the check happens as requests complete. Refresh prices with from pydantic_ai import prices and prices.update_in_background(), and pair the in-process limit with a provider or gateway spend cap that actually stops money leaving.

Watch the Anthropic default. As of 2.52, AnthropicModel defaults max_tokens to the model's maximum output, not 4096, and streams such requests behind the scenes. That raises your worst-case cost per request, so set max_tokens explicitly in settings when you have a ceiling in mind.

Retries stack, and the product is what you pay for. There are three independent layers: an HTTP transport that can retry (the retries extra provides tenacity-based transports), the provider SDK's own retry count, and Pydantic AI's model-level retries triggered by validation failures. They multiply. The official worked example, 3 transport attempts times 3 SDK attempts times 2 model requests, is 18 wire requests for what looks like one question. UsageLimits.request_limit counts only model requests, so it does not see the other layers. When you debug a 429 storm or a surprisingly large bill, count each layer.

Fallback is the tool for provider outages. FallbackModel(primary, secondary) moves to the next model when the first raises ModelAPIError (configurable with fallback_on). For it to react quickly you must set the SDK clients' max_retries=0, otherwise the primary spends its own retries before fallback starts. If every model fails you get a FallbackExceptionGroup whose .exceptions list shows each cause.

from pydantic_ai import Agent
from pydantic_ai.models.fallback import FallbackModel

model = FallbackModel('anthropic:claude-sonnet-4-6', 'openai:gpt-5.2')
agent = Agent(model)

There is no whole-run timeout. ModelSettings(timeout=...) applies per attempt, tool_timeout per tool, MCP has its own initialisation and read timeouts, but nothing bounds the entire run. Wrap the call in asyncio.timeout() (or anyio.fail_after()), or hand the run a CancellationToken and cancel it from a timer. A cancelled run raises RunCancelled, which carries the messages gathered so far.

Human approval and deferred tools

Some tool calls should not happen without a person. Others must be completed somewhere else, such as a browser or another service. Pydantic AI models both as deferred tools: the run ends early, hands you the pending calls, and you resume it later with results.

An approval-required tool is declared with requires_approval=True, or by raising ApprovalRequired from inside the tool when only some arguments need a human. An external tool raises CallDeferred, or lives in an ExternalToolset, and you supply its result yourself. For either kind the agent must allow DeferredToolRequests as an output type, so the run can end with that instead of text.

from pydantic_ai import (
    Agent, DeferredToolRequests, DeferredToolResults, ToolDenied,
)

agent = Agent('openai:gpt-5.2', output_type=[str, DeferredToolRequests])

@agent.tool_plain(requires_approval=True)
def delete_file(path: str) -> str:
    """Delete a file."""
    ...

first = agent.run_sync('Remove /tmp/old.log')
if isinstance(first.output, DeferredToolRequests):
    results = DeferredToolResults()
    for call in first.output.approvals:
        ok = ask_human(call.tool_name, call.args)
        results.approvals[call.tool_call_id] = True if ok else ToolDenied('Not permitted')
    final = agent.run_sync(
        'continue',
        message_history=first.all_messages(),
        deferred_tool_results=results,
    )

You can approve with True, deny with ToolDenied('reason') so the model knows why, or approve while editing the arguments with ToolApproved(override_args=...). The V2 attributes to use are .calls and .approvals; the old DeferredToolCalls name and .tool_calls are gone. Since 2.42, invalid values in DeferredToolResults.approvals are rejected rather than silently accepted. If the decision can be made in-process (for example by a policy function), the HandleDeferredToolCalls capability resolves requests inline and avoids the two-phase dance.

Because the second call needs the first call's history, approval flows pair naturally with the stored-history pattern above, and with a durable workflow engine when human decisions take hours.

MCP toolsets in V2

MCP, the Model Context Protocol, lets an agent use tools that run in another process. If you want the protocol itself, read the MCP guide. This section is about how Pydantic AI consumes it, which changed substantially in V2.

All the V1 client classes (MCPServerStdio, MCPServerSSE, MCPServerStreamableHTTP, MCPServerHTTP and FastMCPToolset) became one class, pydantic_ai.mcp.MCPToolset, which infers the transport. You pass it as a toolset, and async with agent: opens and closes every connection (this replaces run_mcp_servers()).

from pydantic_ai import Agent
from pydantic_ai.mcp import MCPToolset

toolset = MCPToolset('http://localhost:8000/mcp')        # Streamable HTTP, recommended

agent = Agent('openai:gpt-5.2', toolsets=[toolset])

async def main():
    async with agent:
        result = await agent.run('What is 7 plus 5?')
        print(result.output)

An address ending in /sse is inferred as the older SSE transport, which is deprecated; prefer Streamable HTTP. For a subprocess server you pass a stdio transport from the FastMCP client library. Install the mcp extra for any of this; if it is missing you get an ImportError asking you to install the FastMCP client.

Three production points follow. First, defaults differ from V1, with init_timeout of 5 seconds, read_timeout of 300 seconds and tool_error_behavior='retry', so re-check them against what your servers need. Second, config files in the {"mcpServers": {...}} shape (with ${VAR} and ${VAR:-default} expansion) are loaded with load_mcp_toolsets, and they spawn processes and expand environment variables, so treat them as trusted code, never as user input. Third, and most important for a service: a shared MCPToolset is one session and one identity for every concurrent run. If each user needs their own credentials, build the toolset per run with a dynamic toolset function so the token comes from ctx.deps:

@agent.toolset(per_run_step=False)
def user_mcp(ctx: RunContext[Deps]):
    return MCPToolset(ctx.deps.mcp_url, auth=ctx.deps.token)

If you used the old MCP(url=...) capability expecting the provider to call the server, note that in V2 it runs the server locally; add native=True for the provider-side behaviour.

Native tools, web access and agent specs

Two features reduce code you would otherwise write, and both have V2 sharp edges.

Native tools are tools the model provider executes on its side: web search, code execution, web fetch, image generation and similar. You attach them with the NativeTool(...) capability, using classes from pydantic_ai.native_tools (the V1 module was builtin_tools, and the keyword was builtin=). Because the provider runs them, there is no function of yours to test, and TestModel cannot emulate them, which is why the override in the testing section removes them. The higher-level WebSearch() and WebFetch() capabilities choose between a native implementation and a local one. In V2 they are native-only by default and raise on a model that lacks the feature, so a fallback must be asked for explicitly with WebSearch(local='duckduckgo') or WebFetch(local=True). The local fetch tool blocks private IP ranges and cloud metadata endpoints by default and caps downloads at 50 MiB; do not weaken those defaults (allow_local_urls) without a reason, because a fetch tool that an attacker can steer is a server-side request forgery primitive.

Agent specs describe an agent in YAML or JSON, which is convenient when prompts and model choice are changed by people who do not deploy code. Agent.from_file('agent.yaml') loads one, and Agent.from_spec({...}, deps_type=...) builds from a dictionary. The keys are model, instructions, model_settings and capabilities, where a capability with arguments is written as a mapping:

model: anthropic:claude-sonnet-4-6
instructions: Answer briefly and cite sources.
model_settings:
  temperature: 0.0
capabilities:
  - WebSearch:
      local: duckduckgo

Specs need the spec extra, which is not part of the V2 default install, so a service that loads one needs pip install "pydantic-ai[spec]". Treat spec files as trusted configuration reviewed like code: a capability list can grant tools, so a spec that anyone can edit is a way to change what your agent is allowed to do.

When you are unsure whether a behaviour comes from your code, a capability or the provider, reduce the agent. Run it with Agent('test') and then one capability at a time. The pydantic_ai.direct module (model_request and its streaming variants) sends a single request to a model with no agent loop at all, which isolates provider behaviour from framework behaviour in one call.

Testing without a network

Agents are non-deterministic, but your code around them need not be. The goal of testing is to prove that your wiring, tools, validators and error handling work, using a fake model, and to leave "is the model smart enough" to evals.

Start with a global safety net, so a forgotten override can never spend money in CI:

from pydantic_ai import models

models.ALLOW_MODEL_REQUESTS = False

With that set, a test that reaches a real model fails with RuntimeError: Model requests are not allowed, since ALLOW_MODEL_REQUESTS is False. Then replace the model inside the test with agent.override, which also accepts deps, toolsets, tools, instructions, settings and retries so you can swap any collaborator.

TestModel calls every tool your agent has with generated arguments and then returns a valid output of the right type, which makes it perfect for a "does the whole loop wire up" smoke test. FunctionModel gives you a function that receives the messages and returns a response, so you can script an exact conversation, including a malformed tool call to exercise your retry path.

import pytest
from pydantic_ai import capture_run_messages
from pydantic_ai.models.test import TestModel
from pydantic_ai.models.function import FunctionModel, AgentInfo
from pydantic_ai.messages import ModelMessage, ModelResponse, TextPart

from myapp.agent import agent, Deps

def test_wiring():
    with agent.override(model=TestModel(), deps=Deps(tenant='t1', user_name='Sam', db=FakeDb())):
        result = agent.run_sync('anything')
    assert result.output is not None

def test_scripted_answer():
    def respond(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse:
        return ModelResponse(parts=[TextPart('Your order ships Monday.')])

    with agent.override(model=FunctionModel(respond)):
        result = agent.run_sync('When does it ship?', deps=Deps('t1', 'Sam', FakeDb()))
    assert 'Monday' in result.output

If your agent uses native (provider-executed) tools, TestModel cannot emulate them, so override them out: agent.override(model=TestModel(), native_tools=[]).

Two habits make these tests useful rather than decorative. Assert on capture_run_messages() when behaviour depends on the sequence, such as "the tool was called once and the validator rejected the first output". And run your suite with deprecation warnings promoted to errors, -W error::pydantic_ai.exceptions.PydanticAIDeprecationWarning, so a feature scheduled for removal in V3 fails a test today instead of an upgrade later.

For quality rather than wiring, use pydantic_evals: a Dataset(name=..., cases=[Case(...)]) evaluated with evaluate_sync, optionally with an LLMJudge evaluator. Note that V2 requires the dataset name and makes the evaluate arguments keyword-only. For RAG-style scoring and metrics beyond this, the RAGAS and DeepEval guides cover the neighbouring tools.

Observability that survives upgrades

When an agent misbehaves in production, a trace is worth more than a log line: it shows each model request, its tool calls, token use and timing as a tree. Pydantic AI emits standard OpenTelemetry GenAI spans, so any OTLP backend works, and the simplest path is Pydantic's own service, Logfire.

import logfire
from pydantic_ai import Agent

logfire.configure()
logfire.instrument_pydantic_ai()

agent = Agent('openai:gpt-5.2')

Without Logfire, create an OpenTelemetry TracerProvider with an OTLP exporter and call Agent.instrument_all(); per agent, attach capabilities=[Instrumentation(settings=InstrumentationSettings(...))]. The OpenTelemetry guide covers the collector side, and traces from Pydantic AI can be sent to Langfuse the same way.

Four V2 details cause real dashboard breakage. Instrumentation defaults to format version 5. On run spans, token usage is reported under gen_ai.aggregated_usage.* while model-request spans keep gen_ai.usage.*, so a dashboard that reads token counts off the run span with the old attribute names shows zeros. The version policy lists span attribute changes as not breaking in minor releases, so pin InstrumentationSettings(version=5) if dashboards depend on it. Tag runs with metadata= (a dict or a callable) so tenant and feature are queryable. And set include_content=False where prompts or tool arguments contain personal data; make sure you run 2.44 or later, because earlier versions could still leak exceptions and instructions into spans under that setting. That matters for any team handling customer data under Gulf or Egyptian residency rules, since span content leaves your process.

Serving an agent behind FastAPI

The common deployment is an agent inside a web service. The FastAPI guide covers the framework; here is what is specific to agents.

Create the agent at module level, once. Build deps per request in a dependency function. Make the endpoint async and await agent.run(...), never run_sync, so the event loop is not blocked. Close long-lived resources on shutdown with async with agent: in the lifespan, so MCP sessions and owned HTTP clients are released.

from fastapi import FastAPI, Depends
from pydantic import BaseModel

app = FastAPI()

class Ask(BaseModel):
    conversation_id: str
    question: str

@app.post('/ask')
async def ask(body: Ask, user=Depends(current_user)):
    history = load(body.conversation_id)
    result = await agent.run(
        body.question,
        deps=Deps(tenant=user.tenant, user_name=user.name, db=pool),
        message_history=history,
        conversation_id=body.conversation_id,
        usage_limits=LIMITS,
        metadata={'tenant': user.tenant},
    )
    save(body.conversation_id, history + result.new_messages())
    return {'answer': result.output}

For chat front ends, the UI adapters stream events in a protocol a browser library understands: VercelAIAdapter for the Vercel AI SDK, AGUIAdapter for AG-UI. Each exposes dispatch_request(request, agent=agent), accepts on_complete and on_cancel callbacks, needs the ui or ag-ui extra (neither is in the default install), and since 2.44 requires a JSON Content-Type on the request.

Concurrency needs a ceiling. Agent(max_concurrency=10) limits simultaneous runs, and a ConcurrencyLimit(max_running=10, max_queued=100) adds a queue whose overflow raises ConcurrencyLimitExceeded; map that to a 429 or 503 for your callers. A ConcurrencyLimitedModel limits HTTP requests to the provider across runs.

Upgrades, V2 traps and the corporate network

The package moves fast, with a minor release every one to three days. The version policy says minors carry no intentional breaking changes, but it explicitly excludes new message parts, new stream events, new optional fields, span attributes and repr changes, and it allows fixes that break code relying on undocumented behaviour. So pin, and read the compatibility notes of each GitHub release before bumping. Beta features live in a beta module and may change at any time.

If you are on V1, the supported path has three steps: upgrade to the latest V1 (1.100 or later; 1.107.x is the maintained security line), clear every PydanticAIDeprecationWarning, then move to V2 and handle the behaviour changes that never warned. The silent ones are the ones that hurt:

• A bare pip install pydantic-ai is slimmer. It no longer includes Bedrock, Groq, Mistral, Cohere, xAI, Hugging Face, Temporal, AG-UI, ui or spec, so the first symptom is an ImportError naming a missing package. Add the extra: pip install "pydantic-ai[bedrock,groq]".
• end_strategy is 'graceful' and the bare openai: prefix is the Responses API, both described above.
• WebSearch() and WebFetch() are native-only and raise on models that do not support them. Restore fallbacks with WebSearch(local='duckduckgo') and WebFetch(local=True).
• prepare callbacks that return None raise TypeError; return [].
• ModelProfile is a TypedDict, so isinstance checks and .update() break; use merge_profile and dictionary access.
• Streaming names are singular and run_stream_events is now a context manager: async with agent.run_stream_events(...) as events: async for e in events:.

Durable execution changed shape in 2.14. For Temporal, DBOS and Prefect, wrap an agent with the TemporalDurability, DBOSDurability or PrefectDurability capability and call agent.run() inside your own workflow. The wrapper classes TemporalAgent, DBOSAgent and PrefectAgent are deprecated and will be removed in V3. Durability keeps a run alive across crashes; it is not storage, so you still keep your own history. If you orchestrate with Prefect, see the Prefect guide.

Security fixes are part of upgrading too. The current line contains the fixes for the dev web UI issues, SSRF and parsing fixes in the web-fetch tool, and a 2.52 fix for resource exhaustion on deeply nested HTML. A 50 MiB cap on downloads is the default. Run 2.52.0 or the 1.107.7 security release as a minimum.

Putting it all together

Here is the shape of a production agent that uses most of this level at once: typed output with a validator, a tool with a deliberate retry budget, limits, a fallback model, tenant deps, tracing and a test.

from dataclasses import dataclass
from decimal import Decimal
from pydantic import BaseModel
from pydantic_ai import Agent, RunContext, ModelRetry, ToolFailed, UsageLimits
from pydantic_ai.models.fallback import FallbackModel

@dataclass
class Deps:
    tenant: str
    db: object

class Answer(BaseModel):
    text: str
    order_id: str | None = None

agent = Agent(
    FallbackModel('anthropic:claude-sonnet-4-6', 'openai:gpt-5.2'),
    deps_type=Deps,
    output_type=Answer,
    instructions='Answer using the tools. Cite the order id when relevant.',
    retries={'tools': 2, 'output': 2},
    model_settings={'temperature': 0.0, 'max_tokens': 800},
)

@agent.tool
async def get_order(ctx: RunContext[Deps], order_id: str) -> dict:
    """Fetch one order for the current tenant.

    Args:
        order_id: The order identifier, such as A-1042.
    """
    try:
        row = await ctx.deps.db.get(ctx.deps.tenant, order_id)
    except ConnectionError as exc:
        raise ToolFailed('Order system unavailable') from exc
    if row is None:
        raise ModelRetry(f'No order {order_id}. Ask the user to confirm the id.')
    return row

@agent.output_validator
async def must_cite_real_order(ctx: RunContext[Deps], out: Answer) -> Answer:
    if out.order_id and await ctx.deps.db.get(ctx.deps.tenant, out.order_id) is None:
        raise ModelRetry('order_id must come from a tool result')
    return out

LIMITS = UsageLimits(request_limit=12, total_tokens_limit=40_000, cost_limit=Decimal('0.25'))

At the edge, your handler calls await agent.run(question, deps=..., message_history=..., usage_limits=LIMITS, metadata={'tenant': ...}), wraps it in asyncio.timeout(), stores new_messages(), and maps UsageLimitExceeded, UnexpectedModelBehavior and FallbackExceptionGroup to distinct responses. In tests, ALLOW_MODEL_REQUESTS=False plus agent.override(model=FunctionModel(...)) exercise every branch without a network. In operations, one instrumented trace per run, tagged by tenant, tells you where the time and tokens went.

Where you are now

You can predict what an agent will do rather than hoping: you know the loop, which layer a setting or instruction came from, how retry budgets and usage limits interact, and why retries multiply. You can keep an agent honest with output validators, approvals and server-side authorisation, and you can test it with a fake model that cannot spend money. You can trace it, serve it, and upgrade it without being surprised by V2's silent changes.

The next level treats Pydantic AI as a platform for a team: dynamic toolsets and multi-tenant isolation, durable execution for long-running work, concurrency design, fallback strategy under provider incidents, cost governance and the review checklist for agent changes. If you are choosing between frameworks, the neighbouring guides for LangGraph and the OpenAI Agents SDK show how the same problems look elsewhere.

Sources

• Pydantic AI overview
• Installation
• Troubleshooting
• Migration guide to V2
• Changelog
• Version policy
• Agents
• Dependencies
• Output
• Hooks
• Message history
• Retries
• Timeouts
• Capabilities overview
• Tools
• Advanced tools
• Toolsets
• Deferred tools
• Models overview
• MCP client
• Testing
• Logfire integration
• UI integrations
• Durable execution
• Harness
• Exceptions API

How mid-level interviews differ

At this level nobody asks you what an agent is. They give you a broken or half-designed system and watch how you reason. The questions below are scenarios and trade-offs on Pydantic AI 2.52, with the reasoning a strong answer contains. Read each question, answer it aloud, then compare.

Two habits separate strong answers from weak ones. A strong answer names the mechanism (for example "the default request_limit of 50 stopped the loop") rather than a vague cause ("the model got confused"). And it says what it would check first and what evidence would change its mind.

Scenario questions

1. A tool never runs when we stream

Question. "A chat endpoint uses run_stream(). The model sometimes answers a sentence and the database tool never gets called. Non-streaming works. Why?"

Strong answer. run_stream() treats the first output that matches the output type as the final result. If output_type allows str, a leading text part counts as the answer, and tool calls that follow are not executed under default settings. The fix depends on the need: use run_stream_events() or iter(), which keep running the loop until the end, or make the output type structured so plain text cannot count as final. I would reproduce with capture_run_messages() to confirm no ToolCallPart was processed.

What weak answers miss. They blame the model or the prompt, and never mention that the streaming API has a different notion of "done".

2. The run died with "exceeded max retries count of 1"

Question. "A tool fails in production with UnexpectedModelBehavior: Tool 'x' exceeded max retries count of 1. What do you do?"

Strong answer. The default retry budget is one, so a single invalid argument set followed by a second failure ends the run. I would first see why it fails, by capturing the messages: is the model sending invalid arguments (a schema or docstring problem), or is my code raising ModelRetry for something the model cannot fix? If the arguments are the problem I improve the types and the docstring, for example a Literal instead of a free string, and then raise retries for that tool with @agent.tool(retries=3). If my code raises ModelRetry for an infrastructure failure, that is wrong: it should be ToolFailed, which reports the problem without consuming the retry budget. Raising the budget blindly costs a model request per retry.

Follow-up they may ask. "Which retry setting wins?" The most specific: per-tool, then per-toolset, then an override block, then the per-run argument, then the agent, then the default.

3. The bill is far higher than request counts suggest

Question. "Our provider dashboard shows many more requests than our traces. Where do you look?"

Strong answer. Retries stack in layers: an HTTP transport can retry, the provider SDK retries on its own, and Pydantic AI makes additional model requests when validation fails. They multiply; the documentation's example of 3 times 3 times 2 gives 18 wire requests for one apparent question. UsageLimits.request_limit only counts model requests, so it does not see the lower layers. I would count each layer, then decide where retrying belongs. If I use a FallbackModel I set the SDK client's max_retries=0 so failure moves to the next model quickly instead of burning retries on the first.

Bonus. Mention that cost_limit is best-effort based on bundled price data, so it needs a provider-side spend cap behind it, and that Anthropic's default max_tokens is the model maximum since 2.52, which raises worst-case cost per request unless you set it.

4. A dashboard shows zero tokens after an upgrade

Question. "After moving from V1 to V2, our Grafana panel for tokens per run reads zero. Nothing in the code changed."

Strong answer. Instrumentation defaults to format version 5, and run spans now report usage under gen_ai.aggregated_usage.*, while model-request spans keep gen_ai.usage.*. A query that reads the old attribute on the run span finds nothing. Fix the query, or aggregate from request spans. To stop it recurring, pin InstrumentationSettings(version=5) and note that the version policy classifies span-attribute changes as non-breaking in minor releases, so they can change again.

5. Tests are slow, flaky and sometimes spend money

Question. "How would you make the agent test suite deterministic and free?"

Strong answer. Set models.ALLOW_MODEL_REQUESTS = False globally so any un-mocked call fails loudly. Use agent.override(model=TestModel()) for wiring tests, since it calls every tool with generated arguments and returns valid output, and FunctionModel for scripted conversations including a bad tool call to cover the retry path. Override deps with fakes. If the agent uses native tools, override them out because TestModel cannot emulate them. Assert on capture_run_messages() when order matters. Keep model quality measurement separate, in pydantic_evals, run on a schedule rather than in every pull request.

6. One tenant saw another tenant's data through MCP

Question. "Your agent uses an MCP server for CRM access. A user reports seeing records that belong to a different customer. Hypothesis?"

Strong answer. A shared MCPToolset is one session with one identity for all concurrent runs, so credentials from whichever run connected first are used for everyone. The fix is a per-run toolset built from ctx.deps, using the @agent.toolset decorator, so each run authenticates as its user. I would also check that tool filtering is done from deps, not from the prompt, and add tenant tags in metadata to confirm in traces.

7. A deploy to a container fails on TLS only

Question. "The agent works locally but in the Docker image every call raises CERTIFICATE_VERIFY_FAILED."

Strong answer. Since 2.32 and 2.33 Pydantic AI's HTTP layer uses httpx2 and verifies against the operating system trust store, not certifi. A slim image without ca-certificates, or a network with a TLS-inspecting proxy whose root CA is not installed, fails exactly like this. Install the CA into the image's OS store, or pass an httpx2.AsyncClient configured with the right verify as http_client. A good answer adds the wrinkle that Groq and Cohere still use the older client with certifi, so a mixed-provider service can half work.

8. After upgrading, a "send email" tool fires on runs where it didn't

Question. "A tool with side effects started executing in cases where it was previously skipped."

Strong answer. The end_strategy default changed from 'early' to 'graceful' in V2. When the model returns a successful output tool call together with other function tool calls, those function tools now run, in emission order. Either restore end_strategy='early', or, better, make the tool safe: idempotent, gated behind requires_approval=True, and not issued alongside the final answer.

9. The agent loops until it hits a limit

Question. "Runs sometimes fail with The next request would exceed the request_limit of 50. Is that a bug?"

Strong answer. It is the fuse working. The default of 50 model requests per run stops a non-converging tool loop. I would find the loop with the captured messages (typically a tool returning a result the model keeps re-querying, or a validator that always rejects), fix the cause, and then set an explicit UsageLimits with a number appropriate to the task rather than raising the default blindly. I would also alert on UsageLimitExceeded since it indicates a defect, not normal load.

10. A sync tool delegates to another agent

Question. "A tool calls sub_agent.run_sync() and the process raises UserError. Why, and what should it be?"

Strong answer. run_sync and run_stream_sync are rejected inside functions called during a run because they can deadlock the loop. Make the tool async def and await sub_agent.run(..., usage=ctx.usage). Passing ctx.usage accumulates the child's tokens into the parent's totals so limits apply across the whole tree. For blocking non-agent work inside an async tool, use asyncio.to_thread().

Trade-off questions

11. ToolOutput, NativeOutput or PromptedOutput?

Strong answer. ToolOutput is the default and works across most tool-capable models, but the model may answer in text. NativeOutput uses the provider's JSON-schema mode and has the strongest adherence where supported, with limits on schema shapes and models. PromptedOutput places the schema in the prompt and parses text, which is the fallback for models without either feature, with the weakest reliability and extra prompt tokens. I would start with the default, measure retries, move to NativeOutput when extraction is flaky and the model supports it, and simplify the schema before anything else, because Exceeded maximum output retries is usually a schema that is too clever.

12. instructions or system_prompt?

Strong answer. The docs recommend instructions. They are re-evaluated every run and only the current agent's are sent, so a prompt change takes effect for stored conversations, and prompts from a different agent do not leak in through history. system_prompt is stored in history, which is right only if you need a conversation frozen with the prompt it started with. Also mention caching: static instructions sort ahead of dynamic ones, so keep volatile text at the end.

13. Fallback model versus retries

Strong answer. They solve different problems. Retries handle transient errors and validation failures from the same model; a fallback handles an outage or capacity problem at a provider. The trap is combining them badly: if the SDK still retries several times, the fallback only starts after the delay. So set SDK max_retries=0, let the fallback model react immediately, and keep Pydantic AI's retries for validation. I would also test the fallback model on real prompts, since a different model may need different output handling, and handle FallbackExceptionGroup for the all-failed case.

14. Approval versus authorisation

Strong answer. Approval via requires_approval pauses the run for a decision, which protects against the model acting alone. It is not an authorisation boundary: if the client that approves is controlled by an untrusted user, they can forge approvals, because message_history is trusted server state and is not signed. Authorisation must be checked inside the tool against server state. I would use approvals for human-in-the-loop safety on risky actions, and authorisation for who may do what.

15. Durable execution or just store the history?

Strong answer. Storing history lets a conversation continue on the next turn. Durable execution (Temporal, DBOS, Prefect and others) lets a single long run survive a crash midway and resume. The docs say plainly that durability is not storage. I would choose durability when runs take minutes to hours, involve human waits or expensive steps I cannot repeat, and plain stored history when runs are short and a retry from the start is acceptable. In V2 durability is a capability (TemporalDurability and friends) and you call agent.run() inside your own workflow; the wrapper classes are deprecated.

16. Capabilities, hooks or subclassing the agent?

Strong answer. Do not subclass Agent. Use a Hooks() capability for one-off cross-cutting behaviour such as logging or header injection, a declarative Capability when you want a reusable bundle of tools and instructions, and AbstractCapability when you need the full lifecycle. Look at pydantic-ai-harness before building one, because many common capabilities already exist. Use defer_loading=True for capabilities the model rarely needs, to keep prompts small.

17. Should a service use Pydantic AI, or another framework?

Strong answer. Pydantic AI suits teams that want typed outputs, dependency injection and testability in plain Python, with an extension model (capabilities) and first-class OpenTelemetry. A graph-heavy workflow with explicit state may be better served by a graph-centric tool such as LangGraph; Pydantic AI also has its own graph library, pydantic_graph. A provider-specific SDK may fit if you are tied to one vendor. A good answer weighs fit, team skills and the pace of change, and does not claim one is universally better.

Rapid-fire checks

Be ready to answer these in one or two sentences each.

• What does deps_type do at runtime? Nothing; it is for type checking. The object arrives through deps=.
• What is the default request_limit? 50 model requests per run.
• How do you restore the V1 early-stop behaviour? end_strategy='early'.
• What replaced mcp_servers= and run_mcp_servers()? toolsets=[MCPToolset(...)] and async with agent:.
• What replaced result.usage()? The result.usage property.
• Which prefix means the Responses API? Bare openai:; use openai-chat: for Chat Completions.
• How do you stop an unbounded run in time? asyncio.timeout() or a CancellationToken, because there is no built-in whole-run timeout.
• Why pin versions? Releases arrive every one to three days and minors may change spans, events and message parts.

What a strong answer covers that a weak one misses

Strong answers name the V2 mechanism, such as end_strategy, the aggregated usage attributes or the OS trust store. They distinguish who is at fault (model, tool, provider or network) before proposing a fix. They put limits and fakes in place before an incident rather than after. They treat history, approvals and MCP credentials as security boundaries. And they admit what they would verify rather than guessing, for example "I would check the release's compatibility notes".

Weak answers reach for "raise the retries", "use a bigger model" or "add more prompt text", which treat symptoms and cost money. They describe V1 APIs that no longer exist, they trust the client with state, and they cannot explain how they would test the system without calling a real provider.

One more habit: say what you would verify

Interviewers at this level like to hear the edge of your knowledge. The library moves every few days, so a sentence such as "I would confirm that in the release notes for the version we pin" is a sign of maturity, not weakness. Pair it with the concrete place you would look: the migration guide for V1 to V2 changes, the version policy for what minors may change, and the exceptions reference for which error type to catch. Then say how you would confirm the answer in a staging environment with tracing on.

Finally, practise concision. Give the headline cause in one sentence, the evidence in two, and the fix in one more. Interviewers can always ask for depth.

Practice prompts

Try these aloud for two minutes each.

1. Design the error handling for an agent with a flaky inventory API and a strict budget. Which exceptions do you raise inside tools, which limits do you set, and what do you log?
2. A teammate wants to return the full message history to the browser and accept it back each turn. What do you say?
3. You must upgrade 40 services from V1 to V2 this month. Describe the sequence, the tests you add first and the three default changes you would grep for.
4. Your agent's traces contain customer phone numbers. What do you change, and which version do you need?
5. Explain, with the retry layers, why an agent at "3 retries" cost 18 requests during an outage.

When you can answer these with mechanisms, numbers and the name of the specific setting, you are interviewing at the right level.

How to use this page

This is the field guide for people who already run Pydantic AI 2.52 at work. It lists the errors that appear once real traffic arrives, the practices that pay off most (in order of leverage), practice cards to rehearse, and operational recipes you can paste into a runbook. It assumes you have read the mid-level guide, and it repeats nothing from it that is not needed to act.

Errors at this level

Each entry has the symptom, the cause and the fix.

Tool 'x' exceeded max retries count of 1

Cause. The default retry budget is one. The model sent invalid arguments twice, or your code raised ModelRetry twice. Fix. Look at the messages with capture_run_messages() before touching a number. If the model keeps getting arguments wrong, tighten the types (Literal, Field(ge=..)) and the docstring. Then raise the budget for that tool only: @agent.tool(retries=3). If the failure is infrastructure, raise ToolFailed instead of ModelRetry, since it does not consume the budget.

Exceeded maximum output retries (N)

Cause. The output schema or an output validator rejected the model's answer repeatedly. Fix. Simplify the schema (fewer nested optionals and unions), try NativeOutput on a model that supports it, make validator messages specific enough that the model can act on them, and only then raise retries={'output': N}.

UsageLimitExceeded: The next request would exceed the request_limit of 50

Cause. A tool loop never converged and hit the default fuse. Fix. Find the loop in the captured messages. Common causes are a tool that returns an empty result the model keeps retrying, and an output validator that always rejects. Fix that, then set a deliberate UsageLimits for the task.

UserError: ... cannot be used inside a synchronous tool

Cause. A def tool called run_sync() on another agent. Fix. Make the tool async def and await sub_agent.run(..., usage=ctx.usage). Move blocking work into asyncio.to_thread().

UserError: Unknown model: gpt-5

Cause. A V1-style prefix-less model string, or an old prefix. Fix. Use 'openai:gpt-5'. Rename google-gla: to google:, google-vertex: to google-cloud: and grok: to xai:.

ImportError: Please install the ... package

Cause. The V2 default install is slimmer. Bedrock, Groq, Mistral, Cohere, xAI, Hugging Face, Temporal, AG-UI, ui and spec are no longer included. Fix. Install the extra: pip install "pydantic-ai[bedrock,groq]". Put the extras you use in your lockfile so a clean build does not discover this in production.

SSL: CERTIFICATE_VERIFY_FAILED only in containers or behind a proxy

Cause. Since 2.32 and 2.33, HTTP verifies against the OS trust store, not certifi. Slim images and TLS-inspecting proxies lack the CA. Fix. Install ca-certificates and your proxy's root CA into the OS store of the image, or pass a configured httpx2.AsyncClient as http_client. Do not disable verification.

RuntimeError: Event loop is closed

Cause. An agent's provider connection pool was created under an event loop that has since closed, common when scripts call asyncio.run repeatedly with a module-level agent. Fix. Create the model and provider (or a fresh http_client) inside the loop that will use them, and keep one loop for the process lifetime.

RuntimeError: This event loop is already running

Cause. run_sync() in Jupyter, Colab or Marimo. Fix. Use await agent.run(...) at top level. In legacy environments nest_asyncio.apply() is the documented workaround.

RuntimeError: Model requests are not allowed, since ALLOW_MODEL_REQUESTS is False

Cause. A test reached a real model, which is the safety net doing its job. Fix. Wrap the test in agent.override(model=TestModel()) or FunctionModel(fn).

TypeError from a prepare callback

Cause. The callback returned None. V2 turned the V1 warning into an error. Fix. Return [] when you want no tools.

ModelHTTPError and ContentFilterError

Cause. The provider returned a 4xx or 5xx, or its safety filter blocked output. Fix. For a 429, read exc.retry_after, back off, and add a FallbackModel. For a filter, adjust the prompt or provider safety settings, and catch ContentFilterError so a blocked answer produces a polite message, not a 500.

Dashboards read zero tokens after upgrading

Cause. Run spans now report gen_ai.aggregated_usage.*; only model-request spans use gen_ai.usage.*. Fix. Update the queries and pin InstrumentationSettings(version=5).

Practices that pay off most, in order of leverage

1. Turn off real model calls in tests. Setting models.ALLOW_MODEL_REQUESTS = False in conftest.py costs one line and removes a whole class of flaky, expensive CI runs. It is the highest-leverage change on this page.
2. Pin versions and read the compatibility notes. Releases land every one to three days. A lockfile plus a ritual of reading the "Compatibility Notes" of each release you skip over turns surprises into scheduled work.
3. Set limits on purpose. Choose a request_limit, a token limit and a cost_limit per use case, and put a provider-side spend cap behind them, since the in-process cost check is best-effort.
4. Bound the whole run. There is no built-in wall-clock limit. Wrap run calls in asyncio.timeout() at the service edge.
5. Make tools precise. A good docstring and strict parameter types do more for reliability than any prompt wording, and they make failures cheap.
6. Separate retryable from non-retryable failures. ModelRetry for "the model can fix this", ToolFailed for "the world is broken", plain exceptions for bugs.
7. Trace from day one, with tenant metadata. One trace per run, tagged with metadata, answers most production questions faster than logs.
8. Treat history and approvals as server-side. Store history yourself, key it by tenant and conversation, and authorise inside tools.
9. Run with deprecation warnings visible. -W error::pydantic_ai.exceptions.PydanticAIDeprecationWarning in CI tells you about V3 removals now.
10. Keep volatile text last. Static instructions first lets provider prompt caching work.

Practice cards

Work each card for about ten minutes on a scratch project. They are small on purpose.

Card 1: the fuse. Create an agent with a tool that always returns an empty list and a prompt that tells the model to keep searching. Run it with UsageLimits(request_limit=4) and catch UsageLimitExceeded. Print result.usage-style information from the exception message and the captured messages. Goal: recognise a runaway loop in under a minute.

Card 2: retry or fail. Write one tool that raises ModelRetry('Order not found') for unknown ids and ToolFailed('Order system down') when a fake database raises a ConnectionError. Run both paths, and check which consumes the retry budget. Goal: internalise the difference.

Card 3: scripted conversation. Use FunctionModel to return a malformed tool call first and a valid one second. Assert that the run succeeded and that exactly one retry prompt appears in the captured messages. Goal: test failure handling without a provider.

Card 4: history round trip. Run two turns. Serialise the first turn's messages with to_json, reload them with ModelMessagesTypeAdapter.validate_json, and feed them to the second turn in a new process. Then change the agent's instructions and confirm the new text applies. Goal: know what history does and does not carry.

Card 5: retry arithmetic. Write down your service's transport retries, SDK retries and Pydantic AI retries. Multiply them. Compare to your latency budget and your provider rate limit. Goal: find the number before an outage does.

Card 6: tenant leak. Build a dynamic toolset with @agent.toolset that reads a token from ctx.deps. Run two concurrent runs with different tokens and log which credential each tool call used. Goal: prove isolation instead of assuming it.

Card 7: the upgrade grep. On a real repository, search for mcp_servers=, builtin_tools=, history_processors=, result.usage(), OpenAIModel( and prefix-less model strings. Write the V2 replacement beside each hit. Goal: a migration plan in an afternoon.

Operational recipes

Recipe 1: a safe test setup

import pytest
from pydantic_ai import models
from pydantic_ai.models.test import TestModel

models.ALLOW_MODEL_REQUESTS = False

@pytest.fixture
def fake_model():
    return TestModel()

Use with agent.override(model=fake_model, deps=fake_deps): in each test. For agents with native tools, add native_tools=[] to the override.

Recipe 2: a run wrapper with limits, timeout and error mapping

import asyncio
from pydantic_ai import UsageLimitExceeded, UnexpectedModelBehavior, UsageLimits

LIMITS = UsageLimits(request_limit=12, total_tokens_limit=40_000)

async def answer(agent, question, deps, history):
    try:
        async with asyncio.timeout(45):
            return await agent.run(
                question,
                deps=deps,
                message_history=history,
                usage_limits=LIMITS,
                metadata={'tenant': deps.tenant},
            )
    except TimeoutError:
        raise ServiceBusy('The assistant took too long')
    except UsageLimitExceeded:
        raise ServiceBusy('The request was too large')
    except UnexpectedModelBehavior:
        raise ServiceFailed('The assistant could not produce a valid answer')

Map each exception to a distinct, countable outcome. UsageLimitExceeded rising is a defect signal; a timeout rising is a provider or capacity signal.

Recipe 3: debug a misbehaving run

from pydantic_ai import capture_run_messages

with capture_run_messages() as messages:
    try:
        agent.run_sync('the failing prompt', deps=deps)
    finally:
        for m in messages:
            print(type(m).__name__, [type(p).__name__ for p in m.parts])

Read it as a script: request, response with tool call, request with tool return or retry prompt, and so on. The retry prompts show who disagreed. If a trace backend is available, find the same run by conversation_id and compare timings.

Recipe 4: isolate model from framework

When you cannot tell whether a strange answer is the model or your loop, send one request without an agent:

from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest

response = model_request_sync('openai:gpt-5.2', [ModelRequest.user_text_prompt('Say hello')])
print(response.parts)

If the response is fine here and wrong inside the agent, the cause is in instructions, tools or output settings.

Recipe 5: containers and corporate proxies

For a slim image, install the OS CA bundle and any proxy root CA before running anything:

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
    && rm -rf /var/lib/apt/lists/*
COPY corp-root-ca.crt /usr/local/share/ca-certificates/corp-root-ca.crt
RUN update-ca-certificates

Then test one real call during the image build pipeline, not the first production request. See the Docker guide for layer and cache hygiene.

Recipe 6: a pre-upgrade routine

pip install "pydantic-ai==2.52.*"
python -W error::pydantic_ai.exceptions.PydanticAIDeprecationWarning -m pytest
python -c "import pydantic_ai; print(pydantic_ai.__version__)"

Run the suite with deprecation warnings as errors first, then read the GitHub release notes for every version between your pin and the target, looking only at the compatibility sections. Roll out to a staging service with tracing on, and compare token and latency numbers before and after.

Recipe 7: keep secrets out of traces

Instrument with content excluded where prompts may contain personal data, and run 2.44 or later so exceptions and instructions are also redacted:

from pydantic_ai import Agent
from pydantic_ai.capabilities import Instrumentation
from pydantic_ai.models.instrumented import InstrumentationSettings

agent = Agent(
    'openai:gpt-5.2',
    capabilities=[Instrumentation(settings=InstrumentationSettings(include_content=False))],
)

Never put API keys in deps objects that get logged or serialised. Keep provider keys in environment variables or a secret manager and let the provider read them. For region-sensitive data, check where your trace backend is hosted before enabling content capture.

Habits for the first month on a shared agent codebase

Most of the leverage in a team setting comes from a few agreements, written down once and enforced in review.

Agree on one place for limits and settings. When every endpoint invents its own UsageLimits and max_tokens, nobody can answer "what is the worst this feature can cost". Put limits in a small module, name them after the use case (SUPPORT_LIMITS, EXTRACTION_LIMITS), and reference them from handlers. Reviewers then check one file when a number changes, and an incident review can point to a single line.

Keep agents at module level and deps small. Building an agent per request repeats schema generation and breaks connection reuse, and it is the usual cause of the "Event loop is closed" error in scripts. Keep the agent global and build only the deps object per request. Resist adding the whole request object to deps; pass the three fields tools actually need, so tools are easy to fake in tests.

Write tool descriptions as if for a new colleague. State what the tool returns when nothing matches, what the units are, and which argument values are valid. A model that knows an empty list means "no such order" stops looping, which saves requests as well as tokens. Review tool docstrings with the same care as public API documentation, because for the model they are exactly that.

Log the conversation id everywhere. Pass conversation_id to the run, include it in application logs, and put it on the trace. When a user reports a bad answer, the id takes you from a support ticket to the full tool-by-tool story in one search. Without it you are reconstructing events from timestamps.

Review prompt changes like code. Because instructions apply to stored conversations immediately, a prompt edit changes the behaviour of live sessions on the next turn. Ship such edits through the same pull request, test and rollout path as code, and keep a small evaluation dataset in pydantic_evals so you notice regressions on a schedule rather than from users.

Decide the failure messages in advance. For each exception your wrapper maps (UsageLimitExceeded, UnexpectedModelBehavior, ContentFilterError, ModelHTTPError, a timeout), decide what the caller sees and which metric increments. Teams that skip this end up with one generic 500 and no way to tell a provider outage from a prompt bug.

Measure before you tune. Before changing a prompt, a model or a retry number, record three figures for a representative set of inputs: requests per run, tokens per run and the share of runs that needed a retry. Read them from result.usage in a test harness or from the run spans in your trace backend. A change that does not move those numbers, or the quality score from your evaluation dataset, is a change you cannot justify. Teams that skip this step tend to swap models in response to one bad answer, then discover a week later that the real cause was a vague tool docstring that every model misreads in the same way.

Prefer small, boring agents. An agent with six well-described tools and a typed output is easier to test, cheaper to run and simpler to trace than one with thirty tools and a long prompt. When the tool list grows, reach for on-demand capabilities or split the work into delegated agents with their own limits, rather than letting one prompt carry everything.

Rehearse the rollback. Because releases are frequent, the ability to pin back to the previous version in minutes is part of operating this library. Keep the previous lockfile in version control, keep staging on the same pin as production, and practise the rollback once while nothing is wrong.

A short checklist before you ship an agent

• Tests run with ALLOW_MODEL_REQUESTS = False and use override.
• Every tool has a docstring, typed parameters and a deliberate retry budget.
• Retryable and non-retryable failures use ModelRetry and ToolFailed correctly.
• UsageLimits are explicit and a provider spend cap sits behind them.
• The service wraps runs in a timeout and maps each exception to a response.
• History is stored on the server, keyed by tenant and conversation.
• Approval-required tools are also authorised inside the tool.
• MCP credentials come from per-run toolsets when users differ.
• Tracing is on, runs carry tenant metadata, and content capture matches your data policy.
• Versions are pinned, and the image has the CA certificates it needs.

A note on region and data handling

If your users or regulators sit in the Gulf or Egypt, decide early which provider regions the agent may call and where traces are stored. Prompts, tool arguments and tool results all cross the network to the model provider, and spans may cross it again to your trace backend. Choose provider endpoints and trace hosting that match your residency requirements, and test with content capture off before you enable it.

Where to go next

If your agents call remote tools, read the MCP guide. For trace storage and review, the Langfuse guide and the OpenTelemetry guide pair well with the instrumentation section. For serving, the FastAPI guide covers the web side. When these habits are routine, the senior level covers running Pydantic AI as a shared platform for several teams.

This is part three of three. It closes the series by taking every topic from Beginner and Mid one level further and adds the work you own once Pydantic AI is not your tool but your team's, and then other teams' foundation. Everything here is written against Pydantic AI 2.52, the current release as of 30 September 2026.

Up to this point the questions were about making an agent work: get a typed output, add a tool, stream a response. From here the questions change. What happens to a hundred concurrent runs when the provider starts returning 429s? Who can forge a tool approval? Which of your teams is still on a version whose security patches have an expiry date? What do you tell finance when a loop burns through a month of budget in an hour?

Where this picks up

I am starting with architecture, because every operational decision in this track follows from one fact: Pydantic AI is a library, and a library has no control plane.

Architecture and failure modes

Pydantic AI does not run anywhere on its own. It is code that executes inside your process, in a FastAPI route, a worker, a notebook or a scheduled job. There is no Pydantic AI server to scale, patch or fail over. That is a strength, because there is nothing extra to operate, and it is the first thing a platform owner has to absorb: every guarantee you want has to come from the code around the library or from something you attach to it.

An agent run is a small state machine. The user prompt is assembled, the model is called, the response is inspected for tool calls, the tools execute, results are appended, and the loop repeats until a valid output appears or something stops it. Under the hood this is a pydantic_graph graph, and agent.iter() lets you step through its nodes. Each of those steps is a place where something external can fail.

The failure modes fall into four families, and a platform needs an answer to each.

Provider failures are the most frequent: 429 rate limits, 5xx errors, timeouts, and content-filter blocks surfaced as ContentFilterError. They arrive as ModelHTTPError (with status_code, body, headers and retry_after) or as its parent ModelAPIError. They are transient by nature, so the right response is a bounded retry or a fallback model, and the dangerous mistake is stacking several retry layers without noticing. I treat that properly in the reliability section.

Runaway loops are the cost-shaped failures. The default UsageLimits(request_limit=50) caps a run at 50 model requests, which is a sensible backstop and also the reason a looping agent raises UsageLimitExceeded: The next request would exceed the request_limit of 50. Every other limit (tokens, tool calls, cost) is off by default.

Versioning, upgrades and the V2 migration

Pydantic AI reached 1.0 on 5 September 2025 with an API-stability commitment. V2 followed: 2.0.0b1 on 20 May 2026 and stable 2.0.0 on 23 June 2026, with no changes between the last beta and the release. The current release is 2.52.0, from 30 September 2026. Between 2.30 and 2.52 only about six weeks passed, so the cadence is a minor release every one to three days. The companion packages pydantic-ai-slim, clai, pydantic-evals and pydantic-graph are versioned in lockstep with it, while pydantic-ai-harness runs on its own line, currently 0.52.0.

The official version policy is what lets you live with that cadence. There are no intentional breaking changes in minor releases. Deprecated features are removed only in the next major, and V3 will not ship sooner than three months after V2.0. The V1 line receives security fixes only (1.107.7 at the time of writing) and is promised for at least six months after V2 stable, so plan for it to end no earlier than late December 2026.

First, OpenTelemetry span attribute changes are explicitly not considered breaking and can land in minors, including a change to the default instrumentation version. Second, new message parts, new stream events and new optional fields are also not breaking. A consumer that exhaustively matches on event or part types will meet new ones.

The upgrade path from V1 is documented and I recommend making it a hard requirement for every team: upgrade to the latest V1 (at least 1.100.0, now 1.107.x), fix every PydanticAIDeprecationWarning, and only then move to V2 and handle the behaviour changes that never produced a warning. V1-serialized message history still deserializes in V2 through ModelMessagesTypeAdapter, so stored conversations survive.

The behaviour changes that no warning covered are where platform teams lose time, so keep them in a checklist:

• The default install is slimmer. A bare pip install pydantic-ai installs pydantic-ai-slim with the openai, anthropic, google, cli, mcp, evals, web, retries and logfire extras. Bedrock, Groq, Mistral, Cohere, xAI, Hugging Face, Temporal, AG-UI, ui and spec are no longer included, and you get ImportError: Please install the ... package until you add them back. Prefer pydantic-ai-slim with named extras in your platform base image so the dependency set is explicit.
• end_strategy now defaults to 'graceful'. Function tools the model calls in the same response as a successful output tool now run, in emission order. A tool with a side effect that used to be skipped will now fire. Set end_strategy='early' to restore V1 behaviour.
• The bare openai: prefix means the Responses API. Use openai-chat: for Chat Completions. A team pointing at an OpenAI-compatible gateway that only implements Chat Completions will break silently on upgrade.
• Instrumentation moved to format version 5, and run spans now report gen_ai.aggregated_usage.* while model-request spans keep gen_ai.usage.*. Any dashboard that sums tokens from run spans needs updating.
• TLS verification uses the operating-system trust store since the move to httpx2 in 2.32 and 2.33. Slim images and corporate TLS-inspecting proxies fail with certificate errors until the CA is installed in the OS store.
• WebSearch() and WebFetch() are native-only and raise on unsupported models. MCP(url=...) now runs the server locally.

Capabilities and hooks: the platform surface

The central idea of V2 is that capabilities are the core extension primitive. A capability is a reusable bundle of tools, lifecycle hooks, instructions, model settings and model selection, passed as capabilities=[...]. Most behaviour that V1 configured through Agent(...) arguments moved there: history processing is ProcessHistory, instrumentation is Instrumentation, tool preparation is PrepareTools, and provider-side tools are NativeTool.

For a platform owner this is the most useful change in the library, because it gives you a unit of distribution. A capability is something you can version, test and ship to many teams as a package, and teams opt in by adding one line. It is the right shape for the things a platform wants every agent to have: standard instrumentation, a spend guard, a redaction hook, an audit trail, an approved web-search configuration.

You can build one declaratively with Capability(id=, description=, instructions=) and decorate tools onto it, or subclass AbstractCapability when you need the full hook set. The hook set follows a regular pattern: before_*, after_*, wrap_* and on_*_error for the run, node run, model request, tool validation, tool execution, output validation and output processing, plus on_event. The Hooks() capability gives you decorators for the common cases without a subclass.

import logging
from pydantic_ai import Agent
from pydantic_ai.capabilities import Hooks

audit = logging.getLogger("agent.audit")
hooks = Hooks()

@hooks.on.before_model_request
async def log_request(ctx, request_context):
    # Record the tenant and run before every provider call.
    audit.info("model request", extra={"tenant": ctx.deps.tenant_id, "run_id": ctx.run_id})
    return request_context

agent = Agent("openai:gpt-5.2", capabilities=[hooks])

The arguments differ by event, so copy the signature for each hook you use from the hooks page of the documentation, and give every hook a timeout= so a slow audit sink cannot stall every run. A hook that exceeds its timeout raises HookTimeoutError.

Three behaviours of hooks are worth knowing because they catch platform authors out. Raising ModelRetry from before_model_request is not retried; it propagates as an error, so use after_model_request or wrap_model_request when you want a retry. Hooks run in the request path, so anything they do adds latency to every run. And a hook is code that sees prompts, tool arguments and results, which makes it a place where sensitive data can leak into logs if you are careless.

Scaling: which component breaks first

Because the loop is asyncio, a single process handles many concurrent runs cheaply, since most of the time is spent waiting on the network. The documentation gives no framework throughput numbers, and I would distrust anyone who quotes them. What limits you in practice, in the order it usually bites, is as follows.

The provider's rate limits break first. Concurrent runs multiply requests, tool loops multiply them again, and a burst from one team exhausts a shared key for everyone. The framework gives you two throttles. Agent(max_concurrency=10) or an explicit ConcurrencyLimit(max_running=10, max_queued=100) limits concurrent runs of an agent, and when the queue is full it raises ConcurrencyLimitExceeded, which you should translate into a 429 or a retry-later response at your own edge. ConcurrencyLimitedModel(model, limiter=5) limits concurrent HTTP requests to the model, and a shared ConcurrencyLimiter(max_running=, name=) can span several models so that one provider key has one budget.

Retry multiplication breaks second. This is the trap that surprises experienced engineers. Retries exist at several layers and they multiply, they do not add. There can be a transport-level retry (AsyncHTTPX2TenacityTransport, from the retries extra), the provider SDK's own retries, and the model-request retries that your agent configuration implies. The documentation's worked example comes to 3 × 3 × 2 = 18 wire requests for one logical failure. UsageLimits.request_limit counts only model requests, so it does not bound that number. During a provider incident, every client retrying 18 times is how a brownout becomes an outage and a very large bill.

The discipline is to decide which layer owns retries and switch off the others. A common design is to set the SDK's max_retries=0 on the underlying clients, let FallbackModel handle provider failure immediately, and keep one transport retry with jittered backoff for 429s that honours retry_after.

Timeouts are not whole-run limits. ModelSettings(timeout=...) applies per attempt and only on model classes that forward it. Agent(tool_timeout=...) and the per-tool timeout= bound a tool, and on expiry the model receives the retry prompt 'Timed out after N seconds.'. MCP has its own init_timeout (5 seconds by default) and read_timeout (300 seconds by default). There is no built-in wall-clock limit for a whole run. Wrap the call in asyncio.timeout() or anyio.fail_after(), or cancel a CancellationToken from a timer. Without that, a run is bounded by request count and nothing else.

import asyncio
from decimal import Decimal
from pydantic_ai import Agent, UsageLimits

agent = Agent("openai:gpt-5.2", max_concurrency=20, tool_timeout=30)

async def run_bounded(prompt: str, deps):
    limits = UsageLimits(request_limit=15, total_tokens_limit=50_000,
                         cost_limit=Decimal("0.50"))
    async with asyncio.timeout(120):     # the whole-run deadline the library does not provide
        return await agent.run(prompt, deps=deps, usage_limits=limits)

Reliability: fallback, cancellation and durable execution

A platform's reliability story has three layers, and mixing them up is the commonest design error I see in reviews.

Fallback is about provider availability. FallbackModel(primary, secondary, fallback_on=...) moves to the next model on a ModelAPIError by default, and response handlers allow semantic fallback, for example when a response is technically successful but unusable. When every model fails you get FallbackExceptionGroup, whose .exceptions you should log. Remember the rule from the scaling section: disable SDK retries (max_retries=0) on the underlying clients, otherwise the fallback waits behind a stack of retries against a provider you already know is unhealthy. Also be honest about what fallback implies. A different model has different output quality, different tokenisation and different prices, and an evaluation set that passed on the primary says nothing about the secondary.

Durable execution is about surviving the death of the process. If a research agent runs for twenty minutes across forty tool calls and the pod is evicted at minute eighteen, fallback and cancellation do nothing for you. Pydantic AI integrates with Temporal, DBOS, Prefect, Restate, AWS Lambda, Kitaru and Airflow. Since 2.14 the Temporal, DBOS and Prefect integrations are capabilities, TemporalDurability, DBOSDurability and PrefectDurability, and the wrapper agents are deprecated for removal in V3.

from temporalio import workflow
from pydantic_ai import Agent
from pydantic_ai.durable_exec.temporal import PydanticAIPlugin, PydanticAIWorkflow, TemporalDurability

agent = Agent("openai:gpt-5.6-sol", name="geography", capabilities=[TemporalDurability()])

@workflow.defn
class GeographyWorkflow(PydanticAIWorkflow):
    __pydantic_ai_agents__ = [agent]

    @workflow.run
    async def run(self, prompt: str) -> str:
        return (await agent.run(prompt)).output

The rules that follow from that shape are the ones to put in your platform documentation. A run is durable only when called inside a workflow. The agent needs a stable name=, because the name identifies it across restarts. run_sync() inside a workflow raises UserError. With the DBOS and Prefect capabilities you must wrap the call in your own @DBOS.workflow or @flow. And everything that enters a workflow is persisted in workflow history, which means secrets must not be passed as workflow inputs, workspace env= included, unless you use a payload codec.

Security: the trust model

The most useful sentence in the official security guidance is that message_history is trusted server-side state. A conversation history is not just text. It contains ToolCallParts, tool returns and approvals, and the framework treats them as things your server wrote. Pydantic AI does not sign them. If a client is allowed to post the history, that client can forge a tool call the model never made, or forge a tool result, or claim an approval that nobody granted. The documentation calls this a design boundary and not a vulnerability, and that is the correct reading: the boundary is yours to enforce.

The defence has four parts, and a platform should provide all four as defaults.

1. Authenticate at the route, before any agent code runs, and derive the caller's identity from the credential rather than from the request body.
2. Scope toolsets to the caller. Build the toolset from deps per run with @agent.toolset, or filter the tools by role, so that a user's run cannot even see tools they are not allowed to use.
3. Re-check high-stakes actions inside the tool, against server-side state. A tool that refunds money should look up the order, the owner and the limit itself, and not trust that the model, the history or an approval flag implies permission.
4. Run sanitize_messages() over any client-supplied history. It lives in pydantic_ai.messages and strips client system prompts, non-HTTP file URLs and uploaded-file references, and resets workspace references.

The corollary is that tool approval is not an authorization boundary against an untrusted UI client. Deferred tools with requires_approval=True are an excellent human-in-the-loop workflow, but the approval comes back through the same channel that an attacker would use. Authorization means the server decides, from its own records, whether this caller may do this.

from pydantic_ai import Agent, ModelMessagesTypeAdapter
from pydantic_ai.messages import sanitize_messages

def load_client_history(raw_json: bytes):
    history = ModelMessagesTypeAdapter.validate_json(raw_json)
    return sanitize_messages(history)    # never pass client history straight through

# later: agent.run(prompt, message_history=load_client_history(body), deps=deps)

Other parts of the trust model deserve their own paragraphs.

MCP configuration is code. Files loaded through --mcp-config or load_mcp_toolsets spawn processes and expand environment variables, with ${VAR} and ${VAR:-default} syntax. Anyone who can write one can run commands as the service. Treat these files like deployment manifests, not like user data. The guide at /student-guides/mcp covers the protocol side.

LocalWorkspace is not a sandbox. It has the full authority of the host user, and it does not run on Windows at all. For isolation, the Harness offers sandbox capabilities (Modal, E2B, Sprites, Bubblewrap). Do not pass os.environ into an untrusted workspace.

Web fetching has a safety model of its own. web_fetch and FileUrl block private IP ranges and cloud metadata addresses by default, which defends against server-side request forgery, and the opt-ins are allow_local_urls and force_download='allow-local'. Downloads are capped at 50 MiB by default. The cap exists because of a memory-exhaustion fix in 1.107.2 and the 2.x line, and further web_fetch fixes landed in 2.44.0 (an SSRF blocklist bypass via an IPv6 zone identifier, among others) and 2.52.0 (GHSA-v36g-jcw9-x7cw, CPU and memory exhaustion from deeply nested HTML). The minimum safe version today is therefore 2.52.0 on the V2 line or 1.107.7 on V1.

Secrets, telemetry and the supply chain

Provider credentials arrive through environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, AWS_BEARER_TOKEN_BEDROCK or the AWS key pair, AZURE_OPENAI_API_KEY, and so on) or through Provider(api_key=...). For a platform the useful rules are simple. Inject keys from your secret manager at deploy time, as covered in /student-guides/vault, and never place a key in deps, because deps objects have a way of ending up in logs, traces and exception reports. Give each team or workload its own key, so that a leak is containable and spend is attributable.

MCP credentials are a special case, covered under multi-tenancy below. Durable-execution inputs are another: anything that goes into a Temporal, DBOS or Prefect workflow is persisted in its history.

Telemetry is a data-exfiltration path you built yourself. Instrumented runs emit prompts, completions and tool arguments as span attributes by default. InstrumentationSettings(include_content=False) removes them. Before 2.44.0 spans could still leak exceptions and instructions under include_content=False, a bug that was fixed, so combine the setting with a version floor. For the Gulf and Egyptian employers who read this guide, this is the setting that decides whether personal data in prompts ends up in a telemetry vendor's region. Decide it deliberately, and write down where traces are stored.

The supply chain has two layers. The package itself: pin exact versions, mirror the packages internally if your organisation requires it, and verify the installed set in the image. With a minor release every one to three days and an enormous extras list, reproducibility comes from a lockfile and not from a version range. Then the traffic: since the httpx2 change, requests verify TLS against the operating-system trust store and not certifi. In a minimal container with no ca-certificates, or behind a TLS-inspecting proxy such as a corporate gateway, calls fail with SSL: CERTIFICATE_VERIFY_FAILED or a self-signed-certificate-in-chain error. The correct fix is to install the CA into the OS store in your base image. The alternative is passing a configured httpx2.AsyncClient as http_client=. Disabling verification is never the fix. Groq and Cohere still use the legacy httpx with certifi, so two providers on the same host can fail differently.

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
    && rm -rf /var/lib/apt/lists/*
COPY corp-root-ca.crt /usr/local/share/ca-certificates/corp-root-ca.crt
RUN update-ca-certificates
RUN pip install --no-cache-dir "pydantic-ai-slim[openai,anthropic,logfire]==2.52.*"

Multi-tenancy

An agent object is shared and stateless, so tenant separation is something you add through three seams: deps, tool visibility, and history storage.

deps carries the tenant. Create the agent once at start-up and pass a per-request dataclass containing the tenant identifier, the authenticated user, and the clients scoped to that tenant. Tools read it through ctx.deps. Because deps is the only per-run channel, a tool that reaches for a global client has already crossed a tenant boundary.

MCP is where tenancy usually breaks. A shared MCPToolset is one session and one identity, for every concurrent run. If you build it once with one user's token, every other run uses that user's credentials, and you will not see an error, only wrong data and a security incident. The documented fix is a dynamic toolset built per run:

from pydantic_ai import Agent, RunContext
from pydantic_ai.mcp import MCPToolset

agent = Agent("openai:gpt-5.2", deps_type=TenantDeps)

@agent.toolset(per_run_step=False)
def tenant_tools(ctx: RunContext[TenantDeps]):
    return MCPToolset("https://mcp.internal.example/mcp", auth=ctx.deps.token)

Here per_run_step=False builds the toolset once per run and not on every step of the loop, which is what you want when opening a connection is not free. Measure the connection cost before deciding whether it is acceptable at your request rate.

History is keyed by tenant and conversation. Store messages with ModelMessagesTypeAdapter and to_json, append result.new_messages() per turn, and key the record by tenant and conversation_id. Every read must include the tenant in the key, and a test should prove that tenant A cannot load tenant B's conversation by guessing an identifier. Harness StepPersistence offers in-memory, file, SQLite and MongoDB options for resumable runs, and a platform should choose one store and make it the default.

Cost governance

Cost control for LLM agents is a layered system, and each layer catches what the previous one misses. The first layer is the design: an agent that loops fifty times is a design problem, so tool descriptions, output schemas and retry budgets are cost controls. The second layer is the set of limits in UsageLimits: request_limit (default 50), tool_calls_limit, token limits (input_tokens_limit, output_tokens_limit, total_tokens_limit, per_request_input_tokens_limit) and cost_limit. Set explicit values per agent class, and do not rely on the default alone, because 50 requests of a large-context model can be very expensive.

The cost_limit field deserves careful framing. It is calculated from genai-prices, whose price data is bundled at release time, so a model released or repriced after your installed version produces a CostNotFoundWarning or a stale number. You can refresh it with prices.update_in_background(). The documentation itself describes the limit as best-effort, and I would never present it to finance as a guarantee. Pair it with provider-side or gateway-side spend caps, which are the real ceilings. Pydantic AI Gateway offers project, user and key spend limits (daily, weekly and monthly) and either bring-your-own-key or managed billing, and a LiteLLM proxy (/student-guides/litellm) gives similar controls if you already run one.

Observability and evaluation

Pydantic AI emits OpenTelemetry spans that follow the GenAI semantic conventions, so the backend is your choice. The Logfire path is two lines (logfire.configure() and logfire.instrument_pydantic_ai()). Without Logfire, set up a TracerProvider and an OTLP exporter and call Agent.instrument_all(), or attach the Instrumentation(settings=InstrumentationSettings(...)) capability to specific agents. Langfuse (/student-guides/langfuse) and Phoenix both accept OTLP, and the general OpenTelemetry material is in /student-guides/opentelemetry.

Three platform concerns come up repeatedly.

Schema stability. The version policy says span attribute changes can ship in a minor release. So if an alert or a cost dashboard reads attribute names, pin InstrumentationSettings(version=5) and treat a bump as a tracked change. The V2 change from gen_ai.usage.* to gen_ai.aggregated_usage.* on run spans is the live example.

Traces tell you what happened. Evaluation tells you whether it was good, and for a platform it is the only protection against silent model drift. pydantic_evals provides Dataset(name=..., cases=[Case(...)]), evaluate_sync, LLMJudge evaluators and online evaluation, with results visible in Logfire. In V2 the dataset name is required and the arguments are keyword-only. Keep a golden dataset per agent class, run it in CI against the pinned model, and run it again whenever you change the model string, the Pydantic AI version or a fallback target. For deeper evaluation tooling, /student-guides/ragas covers a complementary approach.

Testing as a platform service

A platform that does not provide a test harness gets whatever each team improvises. The pieces already exist, and your job is to standardise them.

Set models.ALLOW_MODEL_REQUESTS = False globally in the shared test configuration, so that a test that accidentally reaches a real provider fails with RuntimeError: Model requests are not allowed, since ALLOW_MODEL_REQUESTS is False, and not with a bill. Use agent.override(model=TestModel()) for structural tests (the schema is valid, the tools are wired) and FunctionModel for scripted behaviour (the model calls this tool with these arguments, then the output validator rejects the first attempt). Use capture_run_messages() to assert on the conversation. The override accepts name, deps, model, toolsets, tools, native_tools, instructions, metadata, model_settings, retries, spec and workspace, so a test can replace almost anything.

import pytest
from pydantic_ai import models
from pydantic_ai.models.test import TestModel

models.ALLOW_MODEL_REQUESTS = False   # any real call in a test becomes a loud failure

@pytest.fixture
def offline(agent_under_test):
    with agent_under_test.override(model=TestModel(), native_tools=[]):
        yield agent_under_test

Governance and incident playbooks

Governance for an agent platform comes down to who may change what, with which review. The items that need an explicit owner are the model allow-list (which providers and regions), the mandatory capabilities, tool registration (a new tool with side effects is a security change, not a feature), instruction and spec changes, MCP server approval, and spending limits. I recommend a small set of written rules over a large approval process: a tool that writes data or spends money needs a security review and an idempotency story, a new MCP server needs an owner and a credential model, and any change to a model string needs a passing evaluation run.

Incidents are where a platform earns its keep. These are the playbooks I would write first.

Provider outage or rate-limit storm. Symptoms are rising ModelHTTPError with 429 or 5xx status, and latency growth. First, check retry multiplication: if clients are retrying at several layers, you are amplifying the incident, so reduce retries before doing anything else. Confirm that FallbackModel is activating (the rate of FallbackExceptionGroup tells you when it is not enough). Use retry_after from the error to back off. If you route through a gateway, shift traffic there. Afterwards, check what the fallback model did to answer quality.

Runaway cost. Symptoms are a sudden spend spike on one key or tenant. Find the tenant from run metadata and the request counts per run. Look for a loop (many requests per run, UsageLimitExceeded at the request limit, or worse, no limit configured), a changed tool description that confuses the model, or an Anthropic agent that picked up the larger default max_tokens. Revoke or cap the key at the provider first and investigate second.

Tool storm or duplicate side effects. Symptoms are duplicated emails, tickets or writes. Check whether the V2 end_strategy='graceful' default now runs tools alongside a successful output, whether a retry of a half-finished run repeated a non-idempotent tool, and whether sequential=True is being relied on as a global switch (in V2 it is a per-tool barrier). Fix with idempotency keys, and with agent.parallel_tool_call_execution_mode('sequential') if ordering matters across a whole run.

Suspected prompt injection or cross-tenant leak. Treat it as a security incident. Pull the run's traces by tenant, read all_messages() for the affected runs, identify which tool or document brought in the instruction, and disable that tool or toolset first. Then check whether any route accepted client-supplied history without sanitisation, and whether any toolset was shared across tenants.

For all of them, keep capture_run_messages() output and the run metadata, because the conversation transcript is the evidence. Mind your own data policy: the same transcript contains whatever the user typed.

Where Pydantic AI stops

Every tool has a boundary, and a platform owner should know where this one ends so that the first design review does not discover it.

It is a library, so there is no control plane. There is no central place to see all agents, to roll out a prompt change without deploying code, or to enforce policy from outside the process. Pydantic AI Gateway and Logfire cover part of that (keys, spend, observability), and agent specs help with externalising configuration. If you need a managed registry of agents with approval workflows and a UI for non-engineers, you are building it or buying it.

It does not store state or run workflows by itself. Conversation storage is yours, and durable execution requires Temporal, DBOS, Prefect or one of the other engines. For long-lived, branching, human-in-the-loop processes, an orchestrator is the main system and Pydantic AI is a step inside it.

It is not an authorization or sandboxing system. Approval flows are a workflow device, LocalWorkspace is not isolation, and guardrail capabilities are probabilistic. Strong boundaries come from your identity system, your network and a real sandbox.

It moves quickly. A minor release every one to three days is a feature for velocity and a cost for stability. If your organisation cannot staff a regular upgrade rhythm, a slower-moving dependency, or a thinner wrapper of your own over the provider SDKs, may serve you better. The official documentation compares Pydantic AI with LangChain and LangGraph, the Claude Agent SDK, the Vercel AI SDK, the OpenAI Agents SDK, Google ADK, Mastra, CrewAI and others, and the comparison pages are the right place to check current differences. Related guides in this catalogue include /student-guides/openai-agents-sdk, /student-guides/claude-agent-sdk, /student-guides/google-adk and /student-guides/langchain.

Choose Pydantic AI when type safety, validated structured output and a small, explicit, in-process abstraction are what you value, and when your team is comfortable owning the surrounding runtime. Choose something else when the missing control plane is the problem you are trying to solve.

Running Pydantic AI as a platform for other teams

Everything above converges on one operating model: a thin, opinionated layer between your teams and the library. The layer is mostly an internal package, and it has five jobs.

Provide the factory. One function builds agents with the approved model list, the mandatory capabilities (instrumentation, audit hook, spend guard, redaction), sensible UsageLimits, explicit timeouts and a retry policy that does not multiply. Teams pass a name, instructions, tools and an output type. They do not choose retry layers.

Provide the base image and the lockfile. Pinned Pydantic AI, explicit extras (the slim package plus named extras), the OS CA bundle, and a smoke test that runs Agent('test').run_sync('hi') as the keyless check. Publish the image on a schedule and tell teams when to move. A rhythm of, say, a monthly upgrade with a fast lane for security advisories works better than ad hoc bumps.

Provide the golden path for the hard things. Per-tenant MCP toolsets, durable execution with idempotent tools, streaming to a UI through VercelAIAdapter or AGUIAdapter.dispatch_request (which needs the ui or ag-ui extra and, since 2.44, a JSON Content-Type), history storage and sanitisation. Each of these is a pattern where the naive version is insecure or fragile, so a reviewed example saves every team from rediscovering the pitfall. If your serving layer is FastAPI, /student-guides/fastapi covers it.

The review checklist

Use this for design documents and pull requests that add or change an agent service. Each question maps to a section above.

Versions and dependencies

• Is Pydantic AI pinned exactly, with a lockfile, and are extras explicit?
• Does the build run with deprecation warnings as errors?
• Is the version at or above 2.52.0 (or 1.107.7 on V1)?
• Is the OS CA bundle installed in the image?

Runtime behaviour

• Does every run have UsageLimits set explicitly, plus a whole-run asyncio.timeout()?
• Is there exactly one owner for retries, with SDK retries disabled where FallbackModel is used?
• Are tool timeouts set, and are I/O tools async def?
• Is max_concurrency or a shared limiter in place, and is ConcurrencyLimitExceeded handled?
• Are side-effecting tools idempotent?

Security

• Is the route authenticated, and is the caller's identity taken from the credential?
• Is client-supplied history passed through sanitize_messages()?
• Are toolsets scoped per caller, and are high-stakes actions re-authorized inside the tool?
• Is anything exposing to_web() or clai web beyond a developer machine?
• Are MCP config files and agent specs under change control?
• Are keys per workload, and kept out of deps, logs and workflow inputs?

Tenancy and data

• Is the tenant in deps, in run metadata and in every history key?
• Are MCP toolsets built per run when auth is per user?
• Is include_content set deliberately, and do you know the region where traces live?

Quality and operations

• Is there an evaluation dataset for this agent, and does CI run it on any model or version change?
• Are dashboards pinned to an instrumentation version?
• Is there an alert on requests per run, UsageLimitExceeded rate and fallback activations?
• Is there a runbook, an owner and a rollback path?

The complete picture

The series started with a typed loop: a model, some tools and a validated output. At the senior level the same loop sits inside an organisation, and the picture is a set of layers, each with an owner.

The innermost layer is the agent: instructions, tools, output type, retry budget and limits. A team owns it. The next layer is capabilities: instrumentation, audit, guardrails, spend control. The platform owns them and ships them as a package. Around that is the runtime: concurrency limits, timeouts, fallback, cancellation, and for work that matters, a durable-execution engine. Around that is the trust boundary: authentication, per-caller toolsets, sanitised history, server-side authorization inside tools, sandboxes for code execution. Around that is the data boundary: tenant-keyed storage, per-user MCP credentials, content policy for telemetry, a model allow-list that encodes residency requirements. And outside everything is the lifecycle: pinned versions, a deliberate upgrade rhythm, evaluation gates, deprecation tracking and incident playbooks.

Three ideas hold the picture together. The first is that the library gives you primitives and not guarantees: durability, authorization, isolation and budgets are all things you attach. The second is that limits compound: retries multiply, concurrency queues, costs accumulate, so every layer's bound has to be chosen with the others in mind. The third is that change is constant: a release every one to three days, with attribute names, defaults and providers moving, means the platform's real product is a safe way to absorb change.

Where the series leaves you

You can now take a team's agent from an idea to a service with typed outputs, scoped tools, bounded cost, tested behaviour, traces and an upgrade path, and you can explain why each of those controls exists. You can reason about what breaks first under load, which retry layer to switch off, what a forged history can do, why an approval flag is not authorization, and how a shared MCP session leaks identity.

The next steps are practical. Build the factory and the base image for your own organisation, and have one team adopt them. Write the three incident runbooks you are least ready for and rehearse one. Put the evaluation suite in front of your next model change. Then read the adjacent guides: /student-guides/langfuse for tracing, /student-guides/mcp for the tool protocol, /student-guides/litellm for a gateway, and /student-guides/prefect or /student-guides/langgraph for the orchestration side.

Sources

• Pydantic AI overview
• Installation
• Migration guide, V1 to V2
• Troubleshooting
• Version policy
• Changelog
• Agents
• Dependencies
• Hooks
• Message history
• Retries
• Timeouts
• Capabilities overview
• Durable execution overview
• Durable execution with Temporal
• Toolsets
• Deferred tools
• MCP client
• Models overview
• Logfire integration
• UI integrations
• Testing
• Pydantic AI Gateway
• Pydantic AI Harness
• Pydantic AI releases and security advisories on GitHub

How senior interviews differ

At the senior level nobody asks you to write a tool decorator. The interviewer wants to know whether you can own an agent platform: design it for other teams, keep it up during a provider incident, defend its cost, and say plainly where it stops. The answers below are written the way a strong candidate would speak them, with the reasoning shown, because the reasoning is what is being scored. Everything is based on Pydantic AI 2.52 (30 September 2026). Where a detail changes quickly, say so in the interview; admitting that a release cadence of one minor every one to three days makes some facts perishable is itself a good senior signal.

The thirty-second answer

Pydantic AI is a Python library for building agents: a typed loop that sends instructions and history to a model, validates the tool calls the model asks for, runs them, and repeats until it gets a valid typed output. It runs inside your process, so it has no control plane. As a platform owner, I attach the rest myself: durable execution when runs must survive crashes, per-tenant toolsets and history, bounded retries and limits, traces, evaluations and a deliberate upgrade policy. Version 2, released in June 2026, made capabilities the extension primitive, and that is the unit I would ship to other teams.

System design prompts

Design an internal agent platform for fifteen product teams

Start by asking what the teams are building (customer chat, back-office automation, code assistants), what data they touch, and what the regulatory constraints are, because those decide the model allow-list and the isolation level. Then describe the layers.

The first deliverable is a factory: one internal function that constructs agents with the approved models, the mandatory capabilities (instrumentation, an audit hook, redaction, a spend guard), explicit UsageLimits, tool timeouts and a single retry policy. Teams supply instructions, tools and an output type. This turns standards into code. The second deliverable is a pinned base image with the slim package and named extras, the operating-system CA bundle, and a lockfile, plus a monthly upgrade rhythm with a fast lane for security advisories. The third is a test kit: global ALLOW_MODEL_REQUESTS = False, an offline fixture using TestModel and FunctionModel, contract tests proving every registered agent carries the mandatory capabilities, and a golden-dataset runner on pydantic_evals.

Then the cross-cutting concerns. Tenancy through deps, run metadata and tenant-keyed history. Observability through OpenTelemetry with the instrumentation version pinned. Cost through per-workload provider keys and a gateway with spend limits, with cost_limit treated as a tripwire. Durability through Temporal, DBOS or Prefect for the few workflows with side effects that matter. Finally, ownership: a named person who reads every release's compatibility notes, because "everyone reads them" means nobody does.

What a weak answer misses is the factory and the paper trail. It lists features of the library and does not say how fifteen teams are kept consistent.

Design a customer-facing chat service with tool use, for many tenants

Begin with the trust boundary, because that is where the failures are. The browser is untrusted. Authenticate at the route and derive tenant and user from the credential. If the client sends conversation history, it must pass through sanitize_messages() and must never be treated as truth, since message_history is trusted server-side state and Pydantic AI does not sign the tool calls and approvals inside it. Better still, store the history on the server, keyed by tenant and conversation, and let the client send only the new message.

Create the agent once at start-up. Pass a per-request deps dataclass with the tenant, the user and scoped clients. Build any MCP toolset per run through @agent.toolset, since a shared MCPToolset is one session and one identity. Filter tools by role before the model sees them. Re-authorize high-stakes tools inside the tool against server state, so that a forged approval or a manipulated model changes nothing.

For the transport, stream through VercelAIAdapter or AGUIAdapter with dispatch_request, remembering the extra and the JSON Content-Type requirement. Set concurrency limits and translate ConcurrencyLimitExceeded into a clean retry-later response. Set a whole-run deadline with asyncio.timeout(), since the library has none. Tag runs with tenant metadata, set include_content=False in production unless there is a documented reason, and keep a per-tenant spend counter.

A strong follow-up is the cancel path: when the user closes the tab, the run should be cancelled through a CancellationToken or by the adapter's cancel callback, and any sync tool the run already started will still finish, so such tools must be idempotent.

Design a long-running research or operations agent that calls many tools and takes actions

This is the durability question. A run that takes twenty minutes and makes dozens of tool calls will meet a deploy, an eviction or a provider outage. Fallback and retries do not help when the process dies, and saving the conversation does not help either, because durability is not storage.

I would run it under a durable-execution engine through the capability path: TemporalDurability, DBOSDurability or PrefectDurability, depending on what the organisation already operates. The wrapper agents (TemporalAgent, DBOSAgent, PrefectAgent) are deprecated for removal in V3, so a new design should not use them. The agent needs a stable name=, the call to agent.run() happens inside the workflow, and run_sync() inside a workflow raises UserError. Nothing secret goes into workflow inputs, because they are persisted in history.

Then the safety design. Side-effecting tools get idempotency keys so a replay does not repeat the action. Risky actions use approval-required deferred tools so a human decides, with the decision recorded server-side. Bound cost with UsageLimits per workflow step, and bound time at the workflow level. For code execution or file access, use a sandbox capability and not LocalWorkspace, which has the full authority of the host user.

If the interviewer pushes on whether this is over-engineered, answer with the cost of loss. If replaying costs two cents, skip it. If a replay sends emails or moves money, it is mandatory.

How would you roll out Pydantic AI V2 across an organisation still on V1

I would not do it in one step. The official path has three stages and I would enforce it as a gate: every service first moves to the latest V1 (1.107.x), fixes every PydanticAIDeprecationWarning, and only then moves to V2. Message history stored by V1 still deserializes under V2 through ModelMessagesTypeAdapter, which removes the largest data-migration risk.

The next point is the behaviour changes that gave no warning, which I would put in a checklist and test for. The default install is slimmer, so Bedrock, Groq, Mistral and several others need explicit extras. end_strategy now defaults to 'graceful', so tools called alongside a successful output now run. The bare openai: prefix means the Responses API, which matters for OpenAI-compatible gateways. Instrumentation moves to format version 5 and run spans use gen_ai.aggregated_usage.*, so cost dashboards need an update. TLS verification uses the operating-system trust store, which breaks images without the right CA. Roll out service by service, behind the evaluation suite, and keep a V1 branch for security fixes only, remembering that V1 patches are promised for at least six months after V2 stable.

Incident response questions

Your provider starts returning 429s and latency doubles. What do you do?

First I confirm what is failing from traces and error types: ModelHTTPError with status 429, with retry_after present or not. Then, before anything else, I check for retry amplification. Retries stack multiplicatively: a transport retry layer, the provider SDK's own retries and the model-request retries can turn one failure into many requests. The documentation's own example comes to eighteen wire requests. If every client is retrying at every layer, we are making the outage worse, so the first action is to cut retries and honour retry_after.

Second, check that fallback is working: is FallbackModel activating, and are SDK retries disabled so it moves on immediately? Third, shed load deliberately. Lower max_concurrency or the shared ConcurrencyLimiter and return clean retry-later responses, which is better than letting queues grow. Fourth, if there is a gateway, route around the failing provider. After the incident, review whether the fallback model's quality was acceptable, since the evaluation suite was probably run only on the primary.

Spend tripled overnight on one API key. How do you find the cause?

Contain first: cap or revoke the key at the provider or gateway, because cost_limit in the library is best-effort and not a billing control. Then attribute. If runs are tagged with tenant metadata, filter by tenant and workload; if they are not, that is the first finding of the postmortem. Look at requests per run: a loop shows up as a high mean, and a request limit that was never lowered from the default of fifty will let a loop run that far every time. Look at retries per tool, because a tool whose description degraded sends the model into repeated attempts. Look at the model: an Anthropic agent on 2.52 uses the model's maximum output as the default max_tokens, which raises worst-case cost per request. Look at prompt shape: dynamic data placed early in the instructions defeats provider caching.

The corrective actions are explicit limits per agent class, per-workload keys so attribution takes minutes, a spend alert, and a gateway cap.

Users report duplicate emails sent by an agent. Where do you look?

Four candidates, in order. First, V2's end_strategy='graceful' default: tools the model calls in the same response as a successful output tool now run, whereas V1 skipped them. Second, a retry of a half-finished run, either by the application or by a durable engine replaying activities, hitting a non-idempotent tool. Third, parallel tool execution, where two calls to the same tool were issued at once; in V2 sequential=True is a per-tool barrier and not a switch for the whole batch, so a team that relied on the old meaning will have changed behaviour. Fourth, a sync def tool that kept running after a timeout or cancellation, because threads cannot be stopped.

The fix is structural: an idempotency key derived from the run and tool call, enforced by the downstream system, and agent.parallel_tool_call_execution_mode('sequential') when ordering across a run truly matters.

After an image rebuild, some pods fail with certificate errors on provider calls. Why?

Since the move to httpx2 in 2.32 and 2.33, Pydantic AI's requests verify TLS against the operating-system trust store, not certifi. A slim image without ca-certificates, or a pod behind a TLS-inspecting proxy whose root CA is not installed, now fails with a certificate-verification error. I compare the CA bundles between the working and failing images, install the proxy's CA into the image and run update-ca-certificates, or pass a configured httpx2.AsyncClient as http_client=. I do not disable verification. I would also mention that Groq and Cohere still use the legacy client with certifi, so providers can fail differently on the same host, which is a good detail for explaining why only some calls broke.

A tester shows that one customer saw another customer's data in an agent reply. Walk me through it.

Treat it as a security incident and not as a model bug. Containment: disable the suspect tool or toolset, and the route if needed. Investigation: pull traces filtered by tenant metadata and read all_messages() for the affected runs, to see which tool returned the foreign data. The likeliest causes are architectural. A shared MCPToolset built once with one identity, used by concurrent runs, so one session serves everyone. A global client in a tool instead of one taken from ctx.deps. A history store keyed by conversation identifier without the tenant. A client-supplied history that was not sanitised, with a forged tool result in it.

Remediation is per-run toolsets through @agent.toolset, tenant in every key, sanitize_messages() on client history, server-side authorization inside tools, and a regression test that runs two tenants concurrently and asserts isolation. I would also review telemetry content, since prompts recorded in traces are another place where the data now lives.

A dashboard of token usage dropped by half after a routine minor upgrade. Did usage really fall?

Probably not. The version policy says OpenTelemetry attribute changes are not breaking and may land in minor releases, and V2 moved run spans to gen_ai.aggregated_usage.* while model-request spans keep gen_ai.usage.*. A query that sums the old attribute on run spans will undercount or double count depending on what changed. I verify against provider invoices or gateway metrics, fix the query, pin InstrumentationSettings(version=5), and add the instrumentation version to the upgrade checklist.

Trade-off questions and how to argue them to leadership

Build on Pydantic AI, or buy a managed agent platform?

The honest framing is control against operating burden. Pydantic AI is a library, so it is cheap to start, type-safe, and gives your engineers full control over tools, tenancy and data location. The price is that the control plane is yours: registry, approvals, durability, budgets, dashboards. A managed platform trades some control and data-location flexibility for those features. For leadership I would put it as a decision about where the differentiation is. If agents are core product logic with custom tools and strict data constraints, build on the library and invest in a thin platform layer. If the need is internal productivity with standard connectors, buy. I would also mention the middle path: use the library and buy the pieces around it, such as a gateway for keys and spend limits and a tracing backend.

Durable execution or retries and fallback?

They solve different failures. Retries and fallback handle a flaky provider. Durable execution handles a dying process and long workflows. The decision rule is the cost of replay. Short, side-effect-free requests do not justify a workflow engine and its operational weight. Long-running, side-effecting work does. In either case, idempotent tools are the foundation, and I would say so before naming any engine.

Pin tightly, or follow the latest?

The release cadence is a minor every one to three days, and the policy promises no intentional breaking changes in minors while explicitly excluding telemetry attributes, new message parts and new stream events. So tight pinning with a deliberate rhythm is the right default for a platform: exact versions in a lockfile, a scheduled upgrade through CI with deprecation warnings as errors, and a security fast lane. The argument to leadership is that the cost of upgrading monthly is small and predictable, while the cost of discovering a V3 removal list in a hurry is not. The V1 security-only window, promised for at least six months after V2 stable, is a concrete deadline to cite.

How do you justify spending on evaluation?

Models and providers change without a commit in your repository. An evaluation suite is the only thing that tells you whether a model-string change, a fallback target or a library upgrade made answers worse. Frame it to leadership as regression testing for a component whose behaviour you do not control. The cost is a dataset per agent class and some CI minutes. The benefit is that upgrades become routine and not frightening.

What a strong answer covers that a weak one misses

Across all of these, strong answers share five habits. They name the layer that owns each guarantee, saying that durability, authorization, isolation and budgets are attached to the library and not provided by it. They notice multiplication: retries across layers, concurrency queues, and per-run cost compounding into per-tenant cost. They separate workflow from authorization, recognising that an approval flag is a human-in-the-loop device and not a security control. They attribute everything, with tenant tags, per-workload keys and run metadata, because every incident starts with "who?". And they respect change: pinned versions, deprecation warnings as errors, evaluation gates, and the humility to say that a fact about a fast-moving library should be re-verified against the current documentation.

Weak answers recite features, propose a single retry knob, trust the client's history, or treat the cost limit as a guarantee.

Sixty-second self-test

Answer each aloud without notes.

1. Name three failure families of the agent loop and one control for each.
2. Why do retries multiply, and what do you disable when you add FallbackModel?
3. What is the default request_limit, and what bounds the whole run?
4. Why is a shared MCPToolset a multi-tenant bug, and what replaces it?
5. What can a client do if it can post message_history, and what are the four defences?
6. Why is tool approval not authorization?
7. Which three deprecated wrapper agents does the durability capability path replace?
8. Why can a token dashboard change after a minor upgrade without any real change in usage?
9. Name the V2 default changes that would affect a production service without a code change.
10. When would you choose something other than Pydantic AI?

If any answer takes more than twenty seconds, reread the matching section of the detailed guide.

How to use this page

The detailed guide explains why. This page is the short, sharp version you keep open during design reviews and incident calls: the traps that only appear when many teams run many agents at scale, the habits of someone who owns the platform, and checklists for pull requests and design documents. It is written for Pydantic AI 2.52 (30 September 2026). Facts about a library that releases every one to three days age quickly, so when a tip names a default, confirm it against the release notes for the version you actually run.

Production traps that only appear at scale

The retry layers multiply and nobody counted them

Symptom: during a provider incident, request volume rises far beyond your traffic, bills spike, and the outage lasts longer than the provider's. Cause: a transport retry layer, the provider SDK's retries and the agent's model-request retries multiply. The documentation's worked example is 3 × 3 × 2 = 18 wire requests. UsageLimits.request_limit counts only model requests, so it does not bound the wire total. Fix: pick one layer to own retries. Set max_retries=0 on the underlying SDK clients when you use FallbackModel, keep one transport retry with backoff that honours retry_after, and write the multiplication down in the service's design document.

The run has no deadline

Symptom: a handful of requests hang for many minutes, pinning connections and concurrency slots. Cause: ModelSettings(timeout=...) is per attempt, tool_timeout bounds a tool, and MCP has its own timeouts. There is no built-in limit for the whole run. Fix: wrap every run in asyncio.timeout() or anyio.fail_after(), or cancel a CancellationToken from a timer, and set the deadline from your user-facing latency budget.

The default request limit hides, then surprises

Symptom: a looping agent costs fifty requests every time before it stops with UsageLimitExceeded: The next request would exceed the request_limit of 50. Cause: fifty is the only limit on by default. Token, tool-call and cost limits are all off. Fix: set explicit UsageLimits per agent class, lower than fifty for most, and add token limits for agents with large contexts.

Cost limits treated as guarantees

Symptom: finance is told spend is capped, and then it is not. Cause: cost_limit is computed from bundled price data (genai-prices), is best-effort, and gives CostNotFoundWarning for a model it has no price for. Fix: treat it as a tripwire. The real ceiling is a provider or gateway spend cap per key, and refresh prices with prices.update_in_background() where you rely on the number.

The shared MCP session

Symptom: one user sees another user's data, with no error anywhere. Cause: a MCPToolset created once is one session and one identity, for all concurrent runs. Fix: build the toolset per run with @agent.toolset, passing the caller's credentials from ctx.deps. Add a concurrency test with two tenants that asserts each tool call sees only its own identity.

Client-supplied history

Symptom: a tool ran that the model never chose, or an approval appeared that no human gave. Cause: message_history is trusted server state, Pydantic AI does not sign the tool calls in it, and your route accepted it from the browser. Fix: store history server-side keyed by tenant and conversation, or pass whatever the client sends through sanitize_messages(), and re-check anything high-stakes inside the tool.

Approval mistaken for authorization

Symptom: a security review finds that anyone who can reach the approval endpoint can approve any pending action. Cause: deferred tools with requires_approval=True are a workflow feature. The approval travels back through the channel an attacker would use. Fix: the server decides from its own records whether this caller may do this, at the moment of execution.

The sync tool that outlives its run

Symptom: a run times out or is cancelled and the side effect still lands. Cause: a sync def tool runs in a thread, and a thread cannot be forcibly stopped. Fix: write I/O tools as async def, make side effects idempotent, and move unavoidable blocking work into asyncio.to_thread() with its own deadline. Remember too that run_sync() inside a sync tool raises a UserError to prevent deadlock, so delegation to a sub-agent belongs in an async def tool.

The event loop that closed

Symptom: RuntimeError: Event loop is closed from inside the HTTP client, usually under test frameworks or scheduled jobs. Cause: the agent's connection pool is bound to a loop that no longer exists. Fix: create the model and provider (or a fresh http_client) per loop in those contexts, and create agents once at application start in long-lived services.

Tools that fire alongside the answer

Symptom: after upgrading to V2, a side-effecting tool runs in responses that also contain a final output. Cause: end_strategy now defaults to 'graceful', so function tools called in the same response as a successful output tool run, in emission order. Fix: set end_strategy='early' to restore the V1 behaviour, or make the tool safe to run, and test both.

Telemetry that breaks without a code change

Symptom: token dashboards or alerts change after a routine upgrade. Cause: span attribute changes are not breaking under the version policy. Run spans report gen_ai.aggregated_usage.* and model-request spans report gen_ai.usage.*. Fix: pin InstrumentationSettings(version=5), read the compatibility notes, and validate dashboards against invoices after each bump.

The image that cannot verify TLS

Symptom: CERTIFICATE_VERIFY_FAILED or a self-signed-certificate error on provider calls, only in some environments. Cause: since 2.32 and 2.33 requests verify against the operating-system trust store, not certifi, and minimal images or TLS-inspecting proxies lack the CA. Fix: install ca-certificates and your proxy's root CA in the image, or pass a configured httpx2.AsyncClient as http_client=. Never turn verification off.

The slimmer install

Symptom: ImportError: Please install the ... package after moving to V2. Cause: a bare pip install pydantic-ai no longer includes bedrock, groq, mistral, cohere, xai, huggingface, temporal, ag-ui, ui or spec. Fix: add what you use, for example pip install "pydantic-ai[bedrock,groq]", and prefer pydantic-ai-slim with named extras in the base image.

Silent change of API flavour

Symptom: calls to an OpenAI-compatible gateway start failing with 404 after an upgrade. Cause: the bare openai: prefix now means the Responses API, and the gateway may implement only Chat Completions. Fix: use the openai-chat: prefix, or openai-responses: explicitly where you mean it.

Anthropic requests with a bigger worst case

Symptom: per-request cost and latency rise for Anthropic agents after 2.52. Cause: AnthropicModel's default max_tokens is now the model's maximum output, and such requests are streamed behind the scenes. Fix: set max_tokens in model_settings for each agent class according to the output you actually need.

Platform-ownership habits

Ship a factory, not a wiki. A function that builds agents with the mandatory capabilities enforces your standards. A document asks for them. Pair it with a contract test that fails the build for any registered agent missing the audit, instrumentation or spend capabilities.

Package behaviour as capabilities. Capabilities are the V2 extension primitive. Instrumentation, redaction, audit, spend guards and approved search configurations belong in capabilities you version and ship. Give each hook a timeout= so a slow sink cannot stall runs, and remember that raising ModelRetry from before_model_request is not retried.

Own the upgrade rhythm with one named person. Pin exact versions, lock the full tree, and upgrade on a schedule with a security fast lane. Read the "Compatibility Notes" of every release between your old and new pin. Run tests with -W error::pydantic_ai.exceptions.PydanticAIDeprecationWarning so V3 removals appear as red builds now. Keep a visible deprecation tracker for the ones that matter to you: TemporalAgent, DBOSAgent and PrefectAgent, the Exa common tools, the GitHub Models provider.

Treat a model string as a deployment. Provider aliases and fallback targets change behaviour without a commit in your repository. Prefer dated model identifiers where the provider offers them, and run the golden dataset in CI against any change to a model string, to the library version or to a fallback target.

Attribute everything. Per-workload provider keys, tenant metadata on every run through metadata=, and a key-to-owner register mean that "who spent this?" takes minutes. Measure that time. If it is an hour, the system is not working.

Decide the telemetry content policy deliberately. Choose per environment whether prompts and completions are recorded, using InstrumentationSettings(include_content=False) where they should not be, with version 2.44 or later so exceptions and instructions are redacted too. For employers in the Gulf and Egypt, record where traces are stored, because that location is a data-residency decision.

Make the safe path the easy path. Provide golden-path examples for the hard cases: per-tenant MCP toolsets, durable execution with idempotent tools, streaming through VercelAIAdapter or AGUIAdapter, history storage with sanitisation. Teams copy what exists. If the only example is insecure, that is what spreads.

Write incident runbooks before the incident. At minimum: provider storm, cost spike, duplicate side effects, certificate failures, suspected cross-tenant leak, upgrade regression. Each should begin with containment, then attribution, then cause. Rehearse the first two with a game day.

Keep the development UI on development machines. clai web and Agent.to_web() are for local use. The advisories GHSA-h4xc-3qfq-jf93 (fixed in 2.28.0) and GHSA-q2xc-rrxj-58x9 (fixed in 2.30.0) show that localhost is not a boundary, because the browser can reach it. Stay at 2.52.0 or above on V2, or 1.107.7 on V1, and use allowed_hosts where the UI is used at all.

Treat configuration files as code. MCP configuration spawns processes and expands environment variables. Agent specs change instructions and tools. Both deserve review and change control equal to application code.

Review checklist for pull requests

Use this when a change adds or modifies an agent.

Construction

• Is the agent created once at start-up, via the platform factory, and not inside a request handler?
• Is the model string explicit, with a provider prefix, and on the approved list for the data region?
• Are UsageLimits set explicitly, with a sensible request_limit and token limits where contexts are large?
• Are retries configured at one layer, with SDK retries off if FallbackModel is used?
• Is max_tokens set where the default is large?

Tools

• Does each tool have a clear docstring, typed arguments and a timeout?
• Are I/O tools async def, and are blocking calls moved to asyncio.to_thread()?
• Does every side-effecting tool take an idempotency key?
• Does a tool that touches money, data deletion or external communication re-check authorization against server state?
• Are approval-required tools backed by a server-side decision, with the approver recorded?

Data and tenancy

• Is tenant and caller identity in deps, taken from the authenticated credential?
• Is MCP auth per run, through @agent.toolset, when credentials are per user?
• Is stored history keyed by tenant and conversation, and is client history sanitised?
• Does a test prove tenant A cannot read tenant B's history or tools?

Operations

• Is there a whole-run asyncio.timeout(), and is ConcurrencyLimitExceeded handled at the edge?
• Is the run tagged with tenant metadata?
• Are traces configured with a deliberate include_content value?
• Does the change include or update an evaluation case?
• Does the offline test use TestModel or FunctionModel, with ALLOW_MODEL_REQUESTS off?

Review checklist for design documents

Scope and trust

• What untrusted text can reach the model, and what powerful capability does the agent hold? List each pair and say what removes or guards it.
• Which tools have side effects, and what happens if a run is retried, replayed or cancelled halfway?
• Who authenticates the caller, and where does authorization actually happen?

Reliability

• What are the provider failure modes, and what is the fallback plan, including the quality of the fallback model?
• What is the worst-case wire request count for one logical failure?
• Does the work need durable execution, and if so, which engine, with which stable agent name, and what goes into its history?
• What is the cancellation story, and which in-flight work cannot be cancelled?

Cost

• What is the expected and worst-case cost per run, and per tenant per month?
• What are the ceilings: library limits, per-key provider caps, a gateway?
• How does prompt structure support caching?

Lifecycle

• Which version is pinned, how does it get upgraded, and who owns that?
• What in the design depends on behaviour the version policy does not guarantee, such as span attribute names or the set of stream events?
• Which deprecated features does the design use, and when does it leave them?
• How is quality measured, and what gates a change to the model or library?

People

• Who is on call for this agent, and where is the runbook?
• Who approves a new tool, a new MCP server or a change to instructions?

Practice cards

Each card is a ten-minute exercise you can do alone or in a team session.

Card 1: count your wire requests. Pick a production agent. Write down its transport retries, SDK retries and agent retries, multiply them, and compare the result with the request limit and your provider's rate limit. Change one setting so the product falls.

Card 2: break your own history route. Craft a request with a forged tool call and approval in the history and send it to a staging chat endpoint. If anything runs, add authentication, server-side history and sanitize_messages(), and write the regression test.

Card 3: the two-tenant test. Write a test that runs two concurrent agent runs with different tenants through your MCP and history layers. Assert that neither run ever sees the other's identity or data.

Card 4: simulate the outage. In staging, block the primary provider's hostname. Watch fallback activation, retry counts, queue depth and alerts, and time how long it takes to find the cause from dashboards alone.

Card 5: the replay audit. List every tool that has a side effect. For each, write what happens if the run is replayed from the top. Fix the ones that duplicate.

Card 6: the cost drill. Revoke a staging key and measure how long it takes to name the owning team and workload. Then add whatever metadata would have made it faster.

Card 7: the upgrade rehearsal. Create a branch bumping Pydantic AI by a few minors. Run the suite with deprecation warnings as errors, run the evaluation set, and compare dashboards with the old version. Write what surprised you.

Operational recipes

Keep these commands and snippets in your runbook, and adapt them to your service.

Confirm the version that actually runs in the deployed image:

python -c "import pydantic_ai; print(pydantic_ai.__version__)"

Run the test suite so that V3 removals fail the build:

python -W error::pydantic_ai.exceptions.PydanticAIDeprecationWarning -m pytest

Install a deliberately minimal dependency set in a base image:

pip install "pydantic-ai-slim[openai,anthropic,logfire]==2.52.*"

Add back extras that V2 no longer installs by default:

pip install "pydantic-ai[bedrock,groq,temporal]"

Bound a run in code with limits and a deadline:

import asyncio
from pydantic_ai import UsageLimits

async def bounded(agent, prompt, deps):
    limits = UsageLimits(request_limit=15, total_tokens_limit=50_000)
    async with asyncio.timeout(120):
        return await agent.run(prompt, deps=deps, usage_limits=limits)

Make every test offline by default:

from pydantic_ai import models

models.ALLOW_MODEL_REQUESTS = False

Inspect what happened in a failing run:

from pydantic_ai import capture_run_messages

with capture_run_messages() as messages:
    try:
        agent.run_sync("reproduce the failure")
    except Exception:
        for m in messages:
            print(m)

Keep prices current where you depend on cost_limit:

from pydantic_ai import prices

updater = prices.update_in_background()

Reading errors quickly

A short decoder for the messages that show up in incident channels.

Related reading

Related reading in this catalogue: /student-guides/langfuse for tracing, /student-guides/mcp for the tool protocol, /student-guides/litellm for gateway controls, and /student-guides/opentelemetry for the telemetry layer under all of it.