The Complete AutoGen Guide

This is part one of three. It covers everything you need to build working multi-agent programs with Microsoft AutoGen, starting from a machine that has never had it installed. By the end you can create an agent backed by a language model, give it Python functions to call, put two or three agents in a team that stops when it should, stream the conversation to your terminal, save and restore a run, and read the error messages AutoGen produces when something is wrong.

One thing has to be said before the first command, because it changes how you should read the rest: AutoGen is in maintenance mode. The README on github.com/microsoft/autogen states that it "will not receive new features or enhancements and is community managed going forward", and that "new users should start with Microsoft Agent Framework". AutoGen 0.7.5 is still a working, widely deployed framework, and the ideas in it — agents, teams, termination conditions, tool loops, handoffs — are the ideas the successor uses too. So learning it is not wasted. But you should know from the first page that this is a tool you learn to maintain and migrate, not one you bet a brand-new three-year project on. The second section covers that in detail, and every later section points out where the successor differs.

Each section ends with a Try it task. Do them as you go. Multi-agent code is deceptively easy to read and surprisingly easy to get wrong, and the only reliable cure is to watch your own run stop for the wrong reason once.

What AutoGen is, and the problem it solves

AutoGen is a Python framework for building applications in which one or more agents — programs that wrap a language model, a set of tools and a bit of memory — work on a task, and where some of them talk to each other to get it done. You describe the agents and how they take turns; AutoGen runs the loop, routes the messages, calls the tools, and tells you why it stopped.

To see what it buys you, write the thing it replaces. Suppose you want a model to answer questions about a city's weather. Without a framework you call the chat completions endpoint, read the response, notice that the model asked to call a function, find that function in a dictionary of your own making, call it, append the result to the message list in exactly the shape the provider expects, call the endpoint again, and check whether the model is done. That is forty or fifty lines of plumbing, none of it your idea. Now add a second model that reviews the first one's answer: you also own a turn-taking loop, a rule for when the conversation is finished, a cap so a disagreement cannot run forever, and a way to see what happened when it goes wrong at three in the morning.

Every line in that paragraph is a thing AutoGen owns for you. The tool-calling round trip becomes tools=[get_weather]. The turn-taking loop becomes RoundRobinGroupChat([writer, critic]). The rule for finishing becomes TextMentionTermination("APPROVE") | MaxMessageTermination(10). The visibility becomes await Console(team.run_stream(task=...)).

That diagram is the whole shape of an AgentChat program, and almost everything in this guide is an elaboration of one of those five boxes.

It is worth knowing what came before, because you will meet the ancestor in search results. AutoGen started in 2023 as a research project whose central object was ConversableAgent and whose central verb was initiate_chat. That API — usually called v0.2 — was quick to demo and hard to reason about: agents replied through registered reply functions, the control flow lived inside the library, and configuration arrived as an llm_config dictionary. Microsoft rewrote it in early 2025. The v0.4 line, which 0.7.5 belongs to, split the framework into an event-driven runtime and a high-level task API, made every agent explicitly stateful, made asynchrony first class, and made the stopping rule a real object you can combine with | and &.

Status first: maintenance mode, and what it means for your code

Treat this as part of the installation, not as a footnote. A framework's support status changes which habits are correct.

The facts, as of this guide's research date. The maintenance-mode banner went on the repository README on 2 October 2025 and was refreshed in April 2026. The last release published to PyPI is 0.7.5, dated 30 September 2025; commits after it are bug fixes, security fixes and documentation. The repository is not archived, so issues and pull requests still move, but no new capability is coming. The named successor is Microsoft Agent Framework, built by the AutoGen and Semantic Kernel teams together, which is at 1.x (the Python agent-framework package was at 1.19.0 in September 2026). Microsoft publishes an official AutoGen-to-Agent-Framework migration guide on Microsoft Learn.

Three practical consequences follow, and they shape the advice in every later section.

Pin everything, exactly. A maintained library absorbs the churn of its dependencies for you. A library in maintenance mode does not. The clearest live example is the Model Context Protocol extra: autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound, so a fresh install today pulls mcp 2.x, and the first import fails with ImportError: cannot import name 'RequestContext' from 'mcp.shared.context'. Nothing is wrong with your code; a dependency moved and nobody updated the bound. You fix it by pinning mcp<2 yourself. Assume more of these over time, and use a lock file from your very first project.

Read the release notes before any upgrade, even a minor one. AutoGen's 0.x minors carried real breaking changes — 0.4 to 0.5 to 0.6 to 0.7 each moved something. Pin autogen-agentchat, autogen-core and autogen-ext to the same exact version, which is what their own metadata expects: autogen-agentchat pins autogen-core to an exact match.

Learn the concepts with an eye on the mapping. The migration guide gives a direct table, and it is short enough to internalise now. AssistantAgent becomes Agent. OpenAIChatCompletionClient keeps its name. FunctionTool becomes a @tool decorator. AgentTool(agent) becomes agent.as_tool(). RoundRobinGroupChat becomes SequentialBuilder. MagenticOneGroupChat becomes MagenticBuilder. GraphFlow becomes WorkflowBuilder. Model context and state become AgentSession. One behavioural difference is worth memorising because it will bite you in both directions: AutoGen's AssistantAgent is effectively single-turn unless you raise max_tool_iterations, whereas the successor's Agent is multi-turn by default.

Three packages and one very confusing name

AutoGen ships as three Python packages, and understanding the split tells you where to look for anything.

Beginners should live almost entirely in AgentChat, reaching into ext for a model client and into core for a couple of helper types. Core's own API — runtimes, topics, subscriptions, routed agents — is a different and lower-level way to write the same kinds of programs, and it belongs at mid and senior level. The three-layer split exists so that AgentChat can stay opinionated while Core stays general.

Alongside the libraries are three developer tools you will see mentioned, none of them needed to learn the framework. AutoGen Studio is a low-code browser UI for assembling teams by clicking; its own documentation says it is "not meant to be a production-ready app", and the stable release 0.4.2.2 requires the autogen-* packages below 0.6, so it must live in a separate virtual environment from your 0.7.x code. Magentic-One is a prebuilt generalist team plus an m1 command line, installed from magentic-one-cli. AgBench is a benchmarking harness.

Now the name problem, which costs beginners more time than any technical issue in this guide.

There is a third name in the mess. pyautogen was the original distribution, and the migration guide states that Microsoft "no longer [has] admin access to the pyautogen PyPI package, and the releases from that package are no longer from Microsoft since version 0.2.34". Today pyautogen 0.10.0 is a thin proxy depending on autogen-agentchat, so installing it is harmless but pointless. The honest rule is the import rule: if the code says import autogen, it is not the framework you installed.

The mental model: four nouns

Everything in AgentChat is built from four nouns. Learn them in this order, because each one depends on the ones before it.

The model client is the thing that talks to a language model. Its interface is ChatCompletionClient, defined in autogen_core.models, and the concrete implementations live in autogen_ext.models.* — one for OpenAI and Azure OpenAI, one for Anthropic, one for Ollama, one for llama.cpp, one for Azure AI, an adapter for Semantic Kernel, and a ReplayChatCompletionClient that returns canned strings and is perfect for tests. A client exposes create() and create_stream() for calls, count_tokens() and remaining_tokens() for budgeting, actual_usage() and total_usage() for cost, a model_info property describing what the model can do, and close(). You create one client and share it between agents.

The agent has a name that must be a valid Python identifier, a description that teams use to decide who should speak, and the behaviour you configure: a system message, tools, memory, a view of the conversation history. The important property, and the one beginners most often get wrong, is that agents are stateful. An agent remembers the conversation across calls. So when you call it again you pass only the new message, never the whole transcript — passing the transcript duplicates everything in its context and doubles your token bill for no benefit.

The team is a group of agents sharing one message thread, driven by a group-chat manager. Four presets matter at this level, and RoundRobinGroupChat is the only one you need today. It gives agents a fixed turn order. SelectorGroupChat asks a model to pick the next speaker each turn. Swarm lets agents hand off to each other explicitly. MagenticOneGroupChat wraps an orchestrator that plans, tracks whether progress has stalled, and re-plans. A fifth, GraphFlow, lets you draw the control flow as a directed graph and is still documented as experimental.

The termination condition decides when a run stops. It is a stateful callable, checked after each agent response against only the new messages, which returns a StopMessage or None. You combine conditions with | for "or" and & for "and", and each one resets itself after a run finishes.

Two return types carry results. Response, returned by an agent's low-level on_messages, has chat_message (the final message) and inner_messages (the events that led to it). TaskResult, returned by run() and yielded last by run_stream(), has messages and — the field you will read most often in your life — stop_reason, a human-readable string saying exactly why the loop ended.

Messages come in two families, and knowing which is which stops a lot of confusion when you start streaming. Chat messages travel between agents: TextMessage, MultiModalMessage, StopMessage, HandoffMessage, ToolCallSummaryMessage and the generic StructuredMessage[T]. Events are internal to one agent and exist so you can watch it think: ToolCallRequestEvent, ToolCallExecutionEvent, MemoryQueryEvent, UserInputRequestedEvent, ModelClientStreamingChunkEvent, ThoughtEvent and SelectSpeakerEvent. Every message carries source, models_usage, metadata, id and created_at, and offers to_text().

Installing AutoGen and checking it works

Use a virtual environment. AutoGen has a wide dependency surface and version pinning matters more here than in a maintained library, so a per-project environment is not optional advice.

python3 -m venv .venv
source .venv/bin/activate              # Linux and macOS
# Windows cmd:        .venv\Scripts\activate.bat
# Windows PowerShell: .venv\Scripts\Activate.ps1

If you prefer conda, conda create -n autogen python=3.12 && conda activate autogen is equivalent. Python 3.10 is the minimum the packages declare; 3.12 is a safe choice.

Then install. The first line is the official README's; the second is what you should commit.

pip install -U "autogen-agentchat" "autogen-ext[openai]"

# For reproducibility, pin the exact versions instead:
pip install "autogen-agentchat==0.7.5" "autogen-ext[openai]==0.7.5"

autogen-core arrives automatically, because autogen-agentchat pins it to the same exact version. The square brackets are pip extras: autogen-ext is a bag of optional integrations and you install only the ones you need. The ones worth knowing early are openai, anthropic, ollama, gemini, azure and llama-cpp for models; mcp, docker and jupyter-executor for tools and code execution; redis, chromadb, mem0 and diskcache for memory and caching; magentic-one, web-surfer and file-surfer for the prebuilt agents; and grpc and rich for the distributed runtime and prettier terminal output. In PowerShell, keep the quotes around the whole argument or the shell eats the brackets.

Set your key as an environment variable. OpenAIChatCompletionClient reads OPENAI_API_KEY by itself, and the Anthropic client reads ANTHROPIC_API_KEY.

export OPENAI_API_KEY=sk-...           # PowerShell: $env:OPENAI_API_KEY="sk-..."

Now verify, in two steps. First that the packages are importable and the versions agree:

python -c "import autogen_agentchat, autogen_core; print(autogen_agentchat.__version__, autogen_core.__version__)"
# -> 0.7.5 0.7.5
pip show autogen-agentchat autogen-core autogen-ext

Second — and do this before you spend a single token — run the framework end to end with no model at all. ReplayChatCompletionClient is a real client that returns the strings you give it in order. It proves your installation, your async setup and your mental model are all correct, and it costs nothing.

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.replay import ReplayChatCompletionClient

async def main():
    agent = AssistantAgent("assistant", model_client=ReplayChatCompletionClient(["Hello World!"]))
    result = await agent.run(task="Say hello")
    print(result.messages[-1].content)   # Hello World!

asyncio.run(main())

Three per-platform notes save a lot of grief. On macOS and Linux use python3; Docker (Docker Desktop or Colima on macOS, Docker Engine on Linux) is needed for the Docker code executor, and autogen-ext[web-surfer] also needs playwright install. On Windows, the local code executor needs the Proactor event loop, and the source warns "The current event loop policy is not WindowsProactorEventLoopPolicy…"; fix it with asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy()). In Jupyter or IPython do not call asyncio.run(...) — the notebook already runs an event loop, and you will get RuntimeError: asyncio.run() cannot be called from a running event loop. Use top-level await.

Your first real agent

Now the same program against a real model. This is the official hello world, and it is worth reading line by line because every later example is a variation of it.

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main() -> None:
    model_client = OpenAIChatCompletionClient(model="gpt-4.1")
    agent = AssistantAgent("assistant", model_client=model_client)
    print(await agent.run(task="Say 'Hello World!'"))
    await model_client.close()

asyncio.run(main())

Four things are happening. The client is created with a model name and no key, because it falls back to OPENAI_API_KEY. The agent is created with a name — "assistant" is a valid Python identifier, which is required — and the client. run(task=...) sends one message and returns a TaskResult, which is why printing it gives you the whole structure rather than just the text. And close() releases the client's HTTP connections; forget it and Python will complain about unclosed sessions at exit.

AssistantAgent has more knobs than any other class you will touch, so here are the ones that matter at this level, with their real defaults.

Note that default system message. The stock AssistantAgent has been told to say TERMINATE when it is finished, which is why so many examples pair it with TextMentionTermination("TERMINATE"). If you replace the system message — and you should, for any agent with a real job — you also take responsibility for whatever word your termination condition is looking for. A termination condition watching for a word your agent was never told to say produces a run that goes to its message cap every time, and the only clue is stop_reason.

There are three ways to run things, and they differ only in how much you see:

• result = await agent.run(task="...") gives you the TaskResult at the end and nothing in between.
• async for message in agent.run_stream(task="..."): ... yields each message and event as it happens, with the TaskResult last.
• await Console(agent.run_stream(task="...")) prints that stream to your terminal in a readable form. Console comes from autogen_agentchat.ui, and Console(..., output_stats=True) adds token counts.

task can be a plain string, a BaseChatMessage, or a list of them. Both run and run_stream also take cancellation_token= to abort and output_task_messages= (default True) to control whether your own task is echoed back in the result.

Giving an agent tools

A tool is how an agent does something other than produce text. In AutoGen, a tool is any Python function — synchronous or asynchronous — with type hints on its parameters and a docstring. You pass the function itself and AutoGen wraps it in a FunctionTool for you. The type hints become the schema the model sees, and the docstring becomes the description the model uses to decide whether to call it, so both are load-bearing rather than decorative.

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def get_weather(city: str) -> str:
    """Get the weather for a given city."""
    return f"The weather in {city} is 73 degrees and Sunny."

async def main() -> None:
    model_client = OpenAIChatCompletionClient(model="gpt-4.1")
    agent = AssistantAgent(
        "weather_agent",
        model_client=model_client,
        tools=[get_weather],
        reflect_on_tool_use=True,
    )
    await Console(agent.run_stream(task="What is the weather in Dubai?"))
    await model_client.close()

asyncio.run(main())

Run it with Console and read the stream carefully, because it shows the tool protocol in full: a ToolCallRequestEvent where the model asks for get_weather(city="Dubai"), a ToolCallExecutionEvent carrying what your function returned, and then a final message.

What that final message is depends on reflect_on_tool_use. With it set to True, the tool result goes back to the model and the agent's final message is the model's prose. With it left off, the agent's final message is a ToolCallSummaryMessage containing the raw tool result formatted with tool_call_summary_format, which defaults to "{result}". Neither is wrong. Reflection reads better and costs an extra model call; the summary is cheaper, faster and exactly what you want when the tool's output is already the answer. The default is subtle: reflect_on_tool_use=None resolves to True when output_content_type is set and False otherwise.

Two rules about tools are worth learning now rather than discovering through an exception. First, an agent takes tools= or workbench=, never both; passing both raises ValueError: Tools cannot be used with a workbench. A workbench is a set of tools that share state and resources — StaticWorkbench for a fixed list, McpWorkbench for a Model Context Protocol server — and it is the mid-level topic. Second, the model must support function calling. If model_info says function_calling: False, passing tools= raises ValueError: The model does not support function calling. straight away.

Tools are also where security enters. A tool runs with your program's privileges, and its arguments come from a model that may be acting on text a stranger wrote. Validate them inside the function as you would an HTTP request body: check ranges, restrict paths, never interpolate a model-supplied string into a shell command or SQL statement. The framework does not do this for you, and the same caution applies with more force to MCP servers — the README's warning is "Only connect to trusted MCP servers." See the MCP guide.

Two agents in a team, and knowing when to stop

A team is a group of agents sharing one message thread. The simplest and most predictable is RoundRobinGroupChat, which gives them a fixed turn order. The classic shape is a producer and a reviewer.

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main() -> None:
    model_client = OpenAIChatCompletionClient(model="gpt-4.1")

    primary = AssistantAgent(
        "primary",
        model_client=model_client,
        description="Writes the first draft.",
        system_message="You are a helpful writer. Write short, concrete prose.",
    )
    critic = AssistantAgent(
        "critic",
        model_client=model_client,
        description="Reviews drafts and approves them.",
        system_message=(
            "Review the draft and give specific, actionable feedback. "
            "When the draft needs no further change, reply with the single word APPROVE."
        ),
    )

    termination = TextMentionTermination("APPROVE") | MaxMessageTermination(10)
    team = RoundRobinGroupChat([primary, critic], termination_condition=termination)

    result = await Console(team.run_stream(task="Write a short poem about the Red Sea in autumn."))
    print(result.stop_reason)

    await team.reset()
    await model_client.close()

asyncio.run(main())

Read the termination condition as English: stop when someone says APPROVE, or when ten messages have gone by. The second half is not optional pessimism, it is the safety net. A team with no termination condition and no max_turns runs until your budget or the provider stops it, and two polite agents can disagree about a poem for a remarkably long time. Make the pairing a habit: one condition that expresses what you actually want, one that bounds the damage.

Here are the built-in conditions, all from autogen_agentchat.conditions:

One counting detail trips up nearly everybody: MaxMessageTermination counts the task message. With MaxMessageTermination(3) you get your task plus two agent replies, and the stop reason reads "Maximum number of messages 3 reached, current message count: 3". If you want N replies, ask for N+1.

The other behaviour to internalise is run and resume. Calling team.run() again — with no task, or with a new one — continues from where the previous run stopped; the agents still remember everything. await team.reset() is what clears the state and gives you a clean slate. This is a feature, and it is also the reason a long-running script slowly gets more expensive if you never reset: the context keeps growing.

You will meet the other presets at mid level, but a one-line map helps now. SelectorGroupChat spends an extra model call each turn asking which agent should speak next, needs at least two participants, and by default will not let the same agent speak twice in a row. Swarm routes by explicit handoffs, where the next speaker is whoever the most recent HandoffMessage named. MagenticOneGroupChat runs an orchestrator with a task and progress ledger, defaults to twenty turns, and gives up after three stalls. Since 0.7.1 a team can itself be a participant of the first two, which is how you nest a sub-team.

Seeing what happened: streaming, events and state

Debugging agents is mostly a visibility problem, so spend ten minutes getting good at this and you will save hours later.

run_stream yields every chat message and every event in order, with the TaskResult last. Wrapping it in Console is the quick option; iterating it yourself is how you build your own UI or log.

from autogen_agentchat.base import TaskResult

async for item in team.run_stream(task="Write a haiku about Cairo traffic."):
    if isinstance(item, TaskResult):
        print("STOPPED:", item.stop_reason)
    else:
        print(f"[{item.source}] {type(item).__name__}: {item.to_text()[:120]}")

Setting model_client_stream=True on an AssistantAgent adds ModelClientStreamingChunkEvents, so you see tokens as they arrive rather than complete messages. That is what you want behind a chat interface, and it is noise in a log.

Token usage is attached to the messages themselves. Every message carries models_usage, a RequestUsage with prompt_tokens and completion_tokens, and the client keeps running totals you can read with total_usage() and actual_usage(). Console(..., output_stats=True) prints them for you. Look at these numbers on your very first team run; the shape of the bill is rarely what people guess, because each agent re-reads the whole shared thread on its turn.

State is the other half of visibility. save_state() on a team or an agent returns a JSON-serialisable dictionary, and load_state() restores it.

import json

state = await team.save_state()
with open("state.json", "w") as f:
    json.dump(state, f)

# later, in another process
with open("state.json") as f:
    await new_team.load_state(json.load(f))

For an AssistantAgent, that state is essentially its model context — the conversation it remembers. This is how you build a web application on top of AgentChat: one team per user session, state written to Postgres or Redis between requests, loaded again on the next one. You cannot load_state while a team is running, and trying raises RuntimeError: The team cannot be loaded while it is running.

Two related constraints surprise people building their first service. A team instance cannot run twice concurrently: a second overlapping run() raises ValueError: The team is already running, it cannot run again until it is stopped. Never share one team object across HTTP requests. And for controlled stopping, prefer ExternalTermination().set(), which lets the current agent finish its turn, over a CancellationToken, which aborts immediately and can leave state inconsistent.

Finally, plain Python logging works: autogen_core exposes TRACE_LOGGER_NAME for human-readable debug output and EVENT_LOGGER_NAME for structured events. The runtime is also instrumented with OpenTelemetry, emitting GenAI spans named create_agent, invoke_agent and execute_tool, so any OTLP backend can show you a trace of a run — that is how you would wire it to Langfuse. AUTOGEN_DISABLE_RUNTIME_TRACING=true switches the runtime spans off.

Putting a human in the loop

Sometimes the right next speaker is a person. AutoGen gives you two ways to arrange that, and choosing wrongly is a classic first-project mistake.

The direct way is UserProxyAgent, an agent whose turn is taken by a human. Its signature is UserProxyAgent(name, *, description="A human user", input_func=None) and input_func defaults to Python's input().

from autogen_agentchat.agents import AssistantAgent, UserProxyAgent
from autogen_agentchat.conditions import TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat

assistant = AssistantAgent("assistant", model_client=model_client)
user = UserProxyAgent("user_proxy")

team = RoundRobinGroupChat(
    [assistant, user],
    termination_condition=TextMentionTermination("APPROVE", sources=["user_proxy"]),
)
await Console(team.run_stream(task="Draft a two-sentence summary of our data residency policy."))

This is perfect for a terminal script and wrong for a web application, for a reason worth understanding. While UserProxyAgent waits for input it blocks the running team, and a running team cannot be saved. So a web request that is waiting for a human is a request holding a live team object with unsaveable state, which is exactly the thing you cannot do at scale.

The pattern that does work is to make "we need the human" a reason to stop. End the run, return to your application, and call run() again with the person's reply when it arrives. HandoffTermination(target="user") and a plain TextMentionTermination both do this, and because resuming a team is normal, the second run() picks up precisely where the first stopped. You can watch for UserInputRequestedEvent in the stream to know a request for input is coming.

Context, memory and keeping the bill down

By default an agent sends its entire conversation history to the model on every turn. That is the UnboundedChatCompletionContext, and for a short task it is exactly right. For a long one it is a slow-motion cost problem: every turn is longer than the last, and eventually you hit the model's context limit and the run fails.

A model context is the configurable answer to "which part of history does the model actually see". There are four, all in autogen_core.model_context:

HeadAndTailChatCompletionContext encodes a real insight: the beginning of a conversation holds the task and the constraints, the end holds the current state, and the middle is the most droppable part.

Memory is the complementary idea. A memory store is queried before each model call and its results are injected into the context, which lets an agent carry facts across conversations rather than within one. The interface is autogen_core.memory.Memory, and the simplest implementation is ListMemory.

from autogen_core.memory import ListMemory, MemoryContent, MemoryMimeType
from autogen_core.model_context import BufferedChatCompletionContext

mem = ListMemory()
await mem.add(MemoryContent(content="User prefers metric units", mime_type=MemoryMimeType.TEXT))

agent = AssistantAgent(
    "assistant",
    model_client=model_client,
    model_context=BufferedChatCompletionContext(buffer_size=10),
    memory=[mem],
)

When memory fires you will see a MemoryQueryEvent in the stream, which is how you confirm it is doing anything. Beyond ListMemory, autogen_ext offers vector and service-backed stores — ChromaDBVectorMemory, RedisMemory, Mem0Memory and a canvas memory — each behind its own pip extra.

While we are on cost, collect the levers in one place, because this is the question your manager will ask. Cap the conversation with MaxMessageTermination or max_turns. Cap the spend directly with TokenUsageTermination. Cap the context with a buffered or token-limited model context. Avoid SelectorGroupChat when routing is deterministic, since it pays for an extra model call every turn just to choose a speaker. Leave reflect_on_tool_use off when the tool's raw output is already the answer, since reflection is another model call. And cache repeated calls: model-response caching has been off by default since v0.4, and you turn it on by wrapping your client.

from autogen_ext.models.cache import ChatCompletionCache, CHAT_CACHE_VALUE_TYPE
from autogen_ext.cache_store.diskcache import DiskCacheStore
from diskcache import Cache

cached_client = ChatCompletionCache(
    model_client,
    DiskCacheStore[CHAT_CACHE_VALUE_TYPE](Cache("/tmp/autogen-cache")),
)

That needs pip install "autogen-ext[diskcache]", and there is a Redis equivalent in autogen_ext.cache_store.redis. While you are developing and running the same prompt twenty times, a cache is the single biggest saving available to you.

Configuration, and the errors you will actually see

Two configuration topics matter at this level: telling AutoGen about a model it does not recognise, and saving a setup as JSON.

The first is model_info, a small dictionary describing what a model can do. The OpenAI client knows OpenAI's own models, so for gpt-4.1 you pass nothing. The moment you point it at something else — a local model through Ollama's OpenAI-compatible endpoint, a vLLM or LM Studio server, a LiteLLM proxy, Gemini's OpenAI-compatible endpoint — it has no idea, and you must say.

from autogen_ext.models.openai import OpenAIChatCompletionClient

client = OpenAIChatCompletionClient(
    model="llama3.2",
    base_url="http://localhost:11434/v1",
    api_key="placeholder",
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": False,
        "family": "unknown",
        "structured_output": True,
    },
)

Omit it and you get ValueError: model_info is required when model name is not a valid OpenAI model. Omit just the structured_output key and you get UserWarning: Missing required field 'structured_output' in ModelInfo. This field will be required in a future version of AutoGen. — a warning today, an error later, so fill it in. Lie about function_calling and your tool calls fail at the provider instead of in AutoGen, which is a much worse place to debug. family takes a value from ModelFamily or the string "unknown". Note also that model_capabilities= is the deprecated predecessor of model_info and the two are mutually exclusive; if you see it in a tutorial, rename it.

A direct Ollama client also exists at autogen_ext.models.ollama.OllamaChatCompletionClient, taking a host key, which is tidier than the OpenAI-compatible endpoint. The Ollama guide covers the server side.

The second topic is component configuration. Every AutoGen component can serialise itself to JSON with dump_component() and be rebuilt with load_component().

cfg = team.dump_component()
json_str = cfg.model_dump_json()

team2 = RoundRobinGroupChat.load_component(cfg)

This is how AutoGen Studio stores teams, and it keeps configuration out of code. Two warnings come with it. Callables are not serialisable — a selector_func, a GraphFlow lambda condition and a FunctionTool cannot survive the round trip. And dump_component() writes an explicitly passed api_key into the JSON, so rely on environment variables and never commit a dumped config. There is a sharper point too: in released 0.7.5, load_component() imports whatever module path the provider string names, so loading component JSON from an untrusted source is remote code execution. A namespace restriction exists on main but is in no PyPI release. Treat component JSON as code.

Now the errors. AutoGen's messages are unusually good — most of them name the fix — so learning to read them is quick.

Four non-error pitfalls round out the list, and each one produces behaviour rather than a traceback, which makes them harder. A team with no termination condition and no max_turns runs until something external stops it. Passing the full history into agent.run() each time duplicates context, because agents are already stateful. Forgetting await model_client.close() leaves HTTP connections open and produces warnings at exit. And max_tool_iterations=1 means one tool round, which looks like an agent that gives up.

Putting it all together

One script that uses everything above: a tool-using researcher, a reviewer, a termination pair, streaming output, token stats and saved state. Treat it as the template for your own first project.

import asyncio
import json

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_core.model_context import BufferedChatCompletionContext
from autogen_ext.models.openai import OpenAIChatCompletionClient

# A stand-in for a real data source: a dict keyed by city.
CITY_DATA = {
    "cairo": {"population_m": 22.2, "timezone": "EET"},
    "riyadh": {"population_m": 7.7, "timezone": "AST"},
    "dubai": {"population_m": 3.7, "timezone": "GST"},
}

async def lookup_city(city: str) -> str:
    """Look up the population in millions and the timezone for a city."""
    record = CITY_DATA.get(city.strip().lower())
    if record is None:
        return f"No data for {city}. Known cities: {', '.join(sorted(CITY_DATA))}."
    return json.dumps(record)

async def main() -> None:
    model_client = OpenAIChatCompletionClient(model="gpt-4.1")

    researcher = AssistantAgent(
        "researcher",
        model_client=model_client,
        description="Looks up city facts with the lookup_city tool and drafts the answer.",
        system_message=(
            "You answer questions about cities. Use the lookup_city tool for every number "
            "you report, and never guess a figure. Address the reviewer's feedback when given."
        ),
        tools=[lookup_city],
        reflect_on_tool_use=True,
        max_tool_iterations=5,
        model_context=BufferedChatCompletionContext(buffer_size=12),
    )

    reviewer = AssistantAgent(
        "reviewer",
        model_client=model_client,
        description="Checks that every figure came from the tool and the answer is complete.",
        system_message=(
            "Check the draft. Every number must be traceable to a tool result, and the answer "
            "must cover every city asked about. Give one concrete correction if it does not. "
            "If the draft is correct and complete, reply with the single word APPROVE."
        ),
        model_context=BufferedChatCompletionContext(buffer_size=12),
    )

    termination = TextMentionTermination("APPROVE", sources=["reviewer"]) | MaxMessageTermination(12)
    team = RoundRobinGroupChat([researcher, reviewer], termination_condition=termination)

    result = await Console(
        team.run_stream(task="Compare Cairo, Riyadh and Dubai by population and timezone."),
        output_stats=True,
    )
    print("\nstop_reason:", result.stop_reason)
    print("usage:", model_client.total_usage())

    with open("team_state.json", "w") as f:
        json.dump(await team.save_state(), f)

    await model_client.close()

asyncio.run(main())

Walk through the decisions, because each one is a thing you now know rather than a thing copied from a sample.

The tool returns JSON and, when it fails, returns a useful sentence rather than raising. That is deliberate: a model reads the string, so an error message listing the known cities lets it recover on its own turn. max_tool_iterations=5 is there because three cities need three lookups, and the default of one would have stopped after the first. reflect_on_tool_use=True is there because the answer should be prose comparing three cities, not three raw JSON blobs.

The termination condition has the sources=["reviewer"] restriction so the researcher cannot end the run by quoting APPROVE in a draft — a small, real failure mode. MaxMessageTermination(12) is the bound, and it counts the task message. Both agents get a buffered context so a long disagreement cannot grow unboundedly, and both description fields are written properly, which becomes necessary the moment you switch to SelectorGroupChat.

What you can now do, and what comes next

You can set up a pinned AutoGen environment and verify it without spending a token. You can create a model client for a hosted or a local model and describe an unrecognised model with model_info. You can build an agent with a system message and tools, and you know why max_tool_iterations and reflect_on_tool_use change its behaviour so much. You can put agents in a RoundRobinGroupChat, express "done" as a combined termination condition, and read stop_reason to find out what really happened. You can stream a run, count its tokens, bound its context, give it memory, save its state and resume it in another process. And you can read AutoGen's error messages and recognise a v0.2 or AG2 tutorial on sight.

What you have not touched is the rest of the framework. The next things to learn, roughly in order of how often they come up at work:

• SelectorGroupChat and Swarm, for routing that is not a fixed order — including selector_func to skip the selection model call when the choice is deterministic, and handoffs= with HandoffTermination for explicit agent-to-agent transfer.
• Workbenches and MCP, which is how an agent reaches real external systems. Remember "mcp<2" and the "only connect to trusted MCP servers" warning.
• Code execution with CodeExecutorAgent and DockerCommandLineCodeExecutor, which became the default executor in 0.7.5 for good reason. Pass approval_func= or you will get a warning telling you code is running with no human oversight. The Docker guide covers the container side.
• Structured output with output_content_type=MyPydanticModel, which turns an agent's reply into a validated object instead of prose you have to parse.
• Agents as tools — AgentTool and TeamTool — remembering that they must not be called in parallel, so the parent's client needs parallel_tool_calls=False.
• GraphFlow and DiGraphBuilder, for fan-out, fan-in and loops, still marked experimental.
• The Core API: runtimes, AgentId, topics and subscriptions, and RoutedAgent with @message_handler. This is where multi-tenancy and event-driven designs live, and it is the senior guide's territory.
• Observability and deployment: OpenTelemetry spans through an OTLP backend, one team per session behind FastAPI, state in a database.

For context on where AutoGen sits, read two or three neighbours. CrewAI takes a role-and-task view of the same problem. LangGraph makes the state machine explicit from the start, which is closer to GraphFlow than to RoundRobinGroupChat. Semantic Kernel is the other parent of the successor framework, so time spent there is directly useful. The OpenAI Agents SDK is the lighter-weight single-vendor comparison. And once anything you build runs for someone other than you, Langfuse is how you find out what it actually did.

A closing word on judgement. Learn AutoGen for what it teaches: the vocabulary of agents, teams, tools, handoffs and termination is now shared across the field, and the concepts transfer almost one-for-one to Microsoft Agent Framework. If you own AutoGen code at work, pin it carefully, keep up with security fixes yourself, and plan a migration with the official guide in hand rather than under time pressure. If you are starting something new, start it on the successor — and this guide taught you most of what you need for that too.

Next in this series: the Mid-level guide, which picks up at SelectorGroupChat, workbenches, structured output, state in a real service and debugging beyond Console.

Sources

• AutoGen repository and README (maintenance-mode notice)
• AutoGen releases
• python-v0.7.5 release notes
• AutoGen documentation (stable)
• Installation
• Quickstart
• AgentChat tutorial
• Models
• Agents
• Teams
• Termination
• Human in the loop
• Managing state
• Memory
• Serializing components
• Migration guide (v0.2 to v0.4+)
• Core concepts: architecture
• Model clients
• Tools
• Command line code executors
• Extensions installation and extras
• AutoGen Studio
• API reference
• autogen-agentchat on PyPI
• autogen-ext on PyPI
• Migrating from AutoGen to Microsoft Agent Framework
• Microsoft Agent Framework

This is the file to read on the morning of an interview. It assumes you have worked through the detailed guide, and it compresses what matters into answers you can say out loud. AutoGen questions at junior level are rarely about obscure APIs; they are about whether you understand what a multi-agent framework is for, whether you know which AutoGen you are talking about, and whether you have ever watched one of these loops fail.

The thirty-second answer

"AutoGen is a Microsoft Python framework for building applications where one or more language-model agents work on a task, including ones where several agents take turns and hand work to each other. You describe the agents, the turn-taking and the stopping rule, and the framework runs the loop, routes the messages, executes the tool calls and tells you why it stopped."

If you have ten seconds more, add the thing that marks you as current: "It is in maintenance mode as of October 2025 — bug and security fixes only — and Microsoft points new projects at Microsoft Agent Framework, which the AutoGen and Semantic Kernel teams built together. The concepts carry over almost one for one, which is why it is still worth knowing."

That second sentence is worth a lot in an interview. It tells a hiring manager you read release notes rather than tutorials, and it reframes the conversation from "can you use this" to "can you judge this", which is the more senior question and a free opportunity.

The model, in four nouns

Say them in this order. Each depends on the one before.

Model client. The thing that talks to the LLM. The interface is ChatCompletionClient from autogen_core.models; the implementations live in autogen_ext.models.* — OpenAI and Azure OpenAI, Anthropic, Ollama, Azure AI, llama.cpp, a Semantic Kernel adapter, and ReplayChatCompletionClient for tests. You create one and share it between agents, and you call await client.close() when you are done.

Agent. A name that must be a valid Python identifier, a description that teams use for routing, a system message, and optionally tools, memory and a model context. The headline fact: agents are stateful. They remember the conversation, so you pass only new messages, never the whole transcript.

Team. A group of agents sharing one message thread, driven by a group-chat manager. RoundRobinGroupChat for a fixed order, SelectorGroupChat when a model picks the next speaker, Swarm when agents hand off explicitly, MagenticOneGroupChat for an orchestrator that plans and re-plans, GraphFlow for a directed graph — the last one still experimental.

Termination condition. A stateful callable checked after every agent response against only the new messages. It returns a StopMessage or None, and you compose conditions with | and &.

Then the two results: Response (from an agent's on_messages, with chat_message and inner_messages) and TaskResult (from run/run_stream, with messages and stop_reason). If you can only remember one field name in the whole framework, remember stop_reason — it is the first thing you read when a run misbehaves, and saying so in an interview signals real use.

The packages, and the name trap

Expect to be asked, or to be given a code sample and asked what is wrong with it. There are three Microsoft packages: autogen-core (the event-driven runtime and the base interfaces), autogen-agentchat (the high-level task API — presets, teams, conditions, Console) and autogen-ext (everything touching the outside world, installed through pip extras like [openai], [anthropic], [mcp], [docker], [chromadb]).

The trap is the name. PyPI autogen and ag2 belong to AG2, the community fork at ag2ai/ag2 with a v0.2-descended API. pyautogen was the original distribution; Microsoft has said it no longer has admin access to it and that releases from it have not been Microsoft's since 0.2.34, and today it is just a proxy depending on autogen-agentchat.

The one-line test to give an interviewer: Microsoft AutoGen imports use underscores. from autogen_agentchat.agents import AssistantAgent is current. Any bare import autogen, any ConversableAgent, initiate_chat, register_reply, llm_config or config_list, is v0.2 or AG2 code. Install autogen-agentchat plus autogen-ext[openai], pinned to the same exact version, on Python 3.10 or newer.

The commands and calls to have ready

Nobody will ask you to recite an API, but fumbling the basics looks bad on a whiteboard. These are the ones worth having cold.

python3 -m venv .venv && source .venv/bin/activate
pip install "autogen-agentchat==0.7.5" "autogen-ext[openai]==0.7.5"
export OPENAI_API_KEY=sk-...
python -c "import autogen_agentchat; print(autogen_agentchat.__version__)"

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

client = OpenAIChatCompletionClient(model="gpt-4.1")
agent = AssistantAgent("assistant", model_client=client, tools=[my_func], reflect_on_tool_use=True)

result = await agent.run(task="...")                  # TaskResult
await Console(agent.run_stream(task="..."))           # streamed to the terminal

team = RoundRobinGroupChat(
    [writer, critic],
    termination_condition=TextMentionTermination("APPROVE") | MaxMessageTermination(10),
)
result = await team.run(task="...")
print(result.stop_reason)
await team.reset()

state = await team.save_state()                       # JSON-serialisable
await client.close()

Three details in that block are worth being able to explain, because they are the ones interviewers probe. run versus run_stream: the first returns the result at the end, the second yields each message and event as it happens with the TaskResult last. Calling team.run() again continues the conversation; reset() is what clears it. And a tool is just a Python function with type hints and a docstring — the hints become the schema the model sees, the docstring becomes the description it uses to choose.

The classic traps, and why they happen

These are the questions that separate someone who has read about AutoGen from someone who has run it.

max_tool_iterations defaults to 1. One model-then-tool round per turn. If a task needs two chained tool calls, the agent stops after the first and looks like it gave up. The parameter arrived in 0.6.2, so older tutorials neither mention it nor need to — their authors worked around the limit by wrapping the agent in a team. Say the default out loud; it is the single most impressive small fact you can produce on this topic.

MaxMessageTermination counts the task message. MaxMessageTermination(3) gives you your task plus two replies, and the stop reason reads "Maximum number of messages 3 reached, current message count: 3". Want N replies, ask for N+1.

A termination condition watching for a word the agent was never told to say. The stock AssistantAgent system message ends "Reply with TERMINATE when the task has been completed", which is why so many samples pair it with TextMentionTermination("TERMINATE"). Replace the system message and you own the keyword too. The symptom is a run that always hits its message cap, and the only clue is stop_reason.

No bound at all. A team with no termination condition and no max_turns runs until your budget or the provider stops it. Always combine a semantic condition with MaxMessageTermination or TimeoutTermination.

Passing the whole history back in. Agents are stateful. Feeding the transcript into run() each time duplicates everything in the context and doubles the bill for nothing.

Sharing one team across requests. A second overlapping run() raises ValueError: The team is already running, it cannot run again until it is stopped. One team instance per session.

UserProxyAgent in a web app. It blocks the running team while it waits for input, and a running team cannot be saved. For an asynchronous human, terminate the run (HandoffTermination(target="user") or a text mention), persist the state, and call run() again with the reply.

reflect_on_tool_use left off when you wanted prose. Without reflection the agent's final message is a ToolCallSummaryMessage containing the raw tool output formatted with tool_call_summary_format (default "{result}"). With it, the tool result goes back to the model and you get prose — at the price of one more model call.

asyncio.run() in a notebook. It raises RuntimeError: asyncio.run() cannot be called from a running event loop. Use top-level await; asyncio.run is for scripts.

A model AutoGen does not recognise. Point the OpenAI client at Ollama, vLLM, LM Studio or a LiteLLM proxy and you must pass model_info with vision, function_calling, json_output, family and structured_output, or you get ValueError: model_info is required when model name is not a valid OpenAI model. Note too that model_capabilities= is the deprecated predecessor.

Common interview questions, with model answers

"What problem does AutoGen solve that calling the API directly does not?" It owns the loops. The tool-calling round trip — read the response, find the function, call it, append the result in the provider's exact shape, call again — becomes tools=[...]. The turn-taking between several models becomes a team. The "when are we done" rule becomes a composable termination condition instead of a hand-written while. And it gives you a stream of typed messages and events, so you can see what happened. The cost is a framework's worth of concepts and a dependency you have to maintain.

"Why is AutoGen async?" Because an agent run is almost entirely waiting — on model APIs, tools, HTTP calls, code execution. The v0.4 rewrite made asynchrony first class so a team can overlap that waiting instead of serialising it, and so the runtime can deliver messages as events. The practical consequence is that every example is inside async def and you either call asyncio.run(main()) in a script or use top-level await in a notebook.

"When would you use RoundRobinGroupChat rather than SelectorGroupChat?" Whenever the order is known. Round robin is deterministic, free and easy to debug. SelectorGroupChat spends an extra model call every single turn asking which agent should speak, which costs money and adds a source of nondeterminism, so it earns its place only when the routing genuinely depends on the content. If routing is mostly deterministic with a few exceptions, selector_func lets you decide in code and fall back to the model by returning None.

"How do you stop a multi-agent conversation reliably?" Two conditions, combined with |. One expresses intent — a reviewer saying APPROVE, restricted with sources=["reviewer"] so the writer cannot end the run by quoting the word — and one bounds the damage, MaxMessageTermination or TimeoutTermination or TokenUsageTermination. Then read stop_reason to find out which one fired. Belt and braces, because the semantic condition depends on a model behaving and the bound does not.

"How would you put this behind an HTTP endpoint?" One team per user session, never shared. Save await team.save_state() — a JSON-serialisable dict — to Postgres or Redis between requests, and load_state it on the next one. Do not use UserProxyAgent, because it blocks a running team and an unsaveable team is useless to a web app; instead terminate when you need the human and resume with their reply. Bound every run, and be aware you cannot load_state or reset while a run is in progress.

"How do you control cost?" Five levers, roughly in order of effect. Cap the conversation with max_turns or MaxMessageTermination; cap the spend with TokenUsageTermination; cap the context with BufferedChatCompletionContext or TokenLimitedChatCompletionContext instead of the unbounded default; avoid paying for a selector model call when routing is deterministic; and cache with ChatCompletionCache over diskcache or Redis, since caching has been off by default since v0.4. Measure first — every message carries models_usage, the client exposes total_usage(), and Console(..., output_stats=True) prints it.

"What is the difference between a chat message and an event?" Chat messages travel between agents and are part of the shared thread: TextMessage, MultiModalMessage, StopMessage, HandoffMessage, ToolCallSummaryMessage, StructuredMessage[T]. Events are internal to one agent and exist so you can watch it work: ToolCallRequestEvent, ToolCallExecutionEvent, MemoryQueryEvent, UserInputRequestedEvent, ModelClientStreamingChunkEvent, ThoughtEvent, SelectSpeakerEvent. Both come out of run_stream, and the distinction matters when you build a UI: you render the chat messages and usually log the events. Every message carries source, models_usage, metadata, id and created_at.

"What is the difference between a model context and memory?" A model context decides which slice of the existing conversation is sent to the model — unbounded by default, or buffered to the last N messages, or head-and-tail, or limited by a token budget. Memory is a store queried before each model call whose results are injected into the context, so it carries facts across conversations rather than within one. ListMemory is the simple built-in; autogen_ext adds ChromaDB, Redis and Mem0 behind their own extras. You see memory firing as a MemoryQueryEvent in the stream, which is how you confirm it is doing anything at all.

"How would you test code built on AutoGen without spending money?" ReplayChatCompletionClient from autogen_ext.models.replay takes a list of strings and returns them in order, so you can assert on your own orchestration — the turn order, the termination, the state round trip — with no network and no key. I use it for the install smoke test too, which proves the environment and the async setup before any token is spent. For tool functions, test them as ordinary Python functions, because that is all they are.

"What are the security considerations?" Three. Tools run with your process's privileges on arguments a model produced from text you may not control, so validate them like request bodies. Model-generated code goes in DockerCommandLineCodeExecutor — the default since 0.7.x — with an approval_func, never the local executor on untrusted input; without an approval function you get a UserWarning saying code will run with no human oversight. And component JSON is code: in released 0.7.5 load_component() imports whatever module the provider string names, so never load it from an untrusted source, and never commit a dumped config, because dump_component() serialises an explicitly passed api_key into the JSON.

"Would you start a new project on AutoGen?" No, and say why clearly. Microsoft's own README says new users should start with Microsoft Agent Framework. If I inherited AutoGen I would pin all three packages to the same exact version with a lock file, own the security patching myself, and plan a migration against the official Microsoft Learn guide rather than under pressure. If I were choosing today I would use the successor, or compare CrewAI, LangGraph and the OpenAI Agents SDK depending on the shape of the problem. The concepts are the same either way, which is why learning AutoGen is not wasted effort.

Sixty-second self-test

Answer out loud. Anything you hesitate on, go back to that section of the detailed guide.

1. Name the four nouns of AgentChat, and say which one is stateful.
2. Which three packages do you install, and what does each hold?
3. How do you tell at a glance that a tutorial is for v0.2 or AG2 rather than for what you installed?
4. What is the default of max_tool_iterations, and what does the wrong value look like from outside?
5. With MaxMessageTermination(3), how many agent replies do you get?
6. Why should a termination condition almost always be two conditions joined with |?
7. What does calling team.run() a second time do, and what clears the state instead?
8. Which field of TaskResult do you read first when a run went wrong?
9. Why is UserProxyAgent the wrong choice behind an HTTP endpoint, and what replaces it?
10. What five keys go in model_info, and when do you have to supply it?
11. Name three cost levers and the thing you measure them with.
12. What is AutoGen's support status, what is the successor, and what does that change about how you use it?

Twelve for twelve means you can hold a junior AutoGen conversation without bluffing. If you want one more thing to say, mention that you keep Console(..., output_stats=True) in every scratch script, because debugging an agent you cannot see is guesswork — interviewers recognise that habit as the mark of someone who has actually shipped one of these.

These are the mistakes beginners actually make with AutoGen, collected from the shape of the framework rather than from a list of best practices. Each one has a symptom you will recognise, the cause underneath it, and the fix. Then the habits worth adopting on day one, and a checklist to run before you call anything finished.

A note on versions before anything else: everything here is for Microsoft AutoGen 0.7.5 — autogen-agentchat, autogen-core and autogen-ext, imported with underscores. If a fix below does not match what you are looking at, check first whether you are in v0.2 or AG2 code, because that accounts for more confusion than every other item on this page put together.

The mistakes

You followed a tutorial and nothing imports. Symptom: ModuleNotFoundError: No module named 'autogen', or an AttributeError on ConversableAgent, or a sample built around initiate_chat and llm_config that cannot possibly work. Cause: the tutorial is for AutoGen v0.2 or for AG2, the community fork whose PyPI names are autogen and ag2. The v0.4 rewrite changed everything, and the fork kept the old shape. Fix: close the page. Microsoft AutoGen imports contain underscores — autogen_agentchat, autogen_core, autogen_ext — and the docs live at microsoft.github.io/autogen/stable/. Triage any page by searching it for ConversableAgent, initiate_chat, register_reply, llm_config, config_list or cache_seed; a hit on any of them means it is the wrong AutoGen.

Your agent stops halfway through a multi-step task. Symptom: it calls one tool, reports a partial answer, and ends as though satisfied. Cause: AssistantAgent.max_tool_iterations defaults to 1, so one model-then-tool round per turn. Fix: raise it, e.g. max_tool_iterations=10. This is the highest-value single fact in this file. It arrived in 0.6.2, so a great deal of still-popular example code predates it.

Your team always runs to the message cap. Symptom: stop_reason always reads "Maximum number of messages N reached", never the thing you wanted. Cause: the termination condition watches for a word the agent was never instructed to say. The stock AssistantAgent system message ends "Reply with TERMINATE when the task has been completed", so the moment you replace it you also own the keyword. Fix: make the system message and the condition agree, word for word. Then add sources=["reviewer"] to TextMentionTermination so only the agent you mean can end the run — otherwise a writer quoting "I'll wait for APPROVE" terminates it by accident.

A run never ends, or ends with a surprising bill. Symptom: a conversation going round in circles, or a cost spike with nothing to show. Cause: no termination condition and no max_turns. Two polite agents can disagree indefinitely. Fix: always two conditions joined with | — one semantic, one bound. TextMentionTermination("APPROVE") | MaxMessageTermination(10). Add TimeoutTermination or TokenUsageTermination when wall-clock or spend is the thing you care about.

You asked for three replies and got two. Symptom: an off-by-one in message counts. Cause: MaxMessageTermination counts the task message. With MaxMessageTermination(3) you get the task plus two replies, and the stop reason confirms it: "current message count: 3". Fix: ask for N+1.

Every turn gets slower and more expensive. Symptom: latency and token counts climbing through a long conversation, eventually a context-length error. Cause: the default model context is unbounded, so the whole history goes to the model on every turn — once per agent, per turn. Fix: give agents BufferedChatCompletionContext(buffer_size=N), HeadAndTailChatCompletionContext or TokenLimitedChatCompletionContext. Note that this is a real trade-off: you are deciding what the model is allowed to forget, and sometimes the answer gets worse.

You passed the transcript back in and the bill doubled. Symptom: token usage far higher than the conversation length suggests, and agents repeating themselves. Cause: agents are stateful. They already remember. Feeding the full history into run() duplicates everything in their context. Fix: pass only the new message. If you want a clean slate, await team.reset().

Your web app breaks under two users. Symptom: ValueError: The team is already running, it cannot run again until it is stopped. Cause: one team object shared across requests. Fix: one team instance per session, state persisted between requests with save_state() and load_state(). You also cannot load_state or reset while a run is in progress — RuntimeError: The team cannot be loaded while it is running.

Your human-in-the-loop design cannot be deployed. Symptom: it works beautifully at the terminal and is impossible to put behind an endpoint. Cause: UserProxyAgent blocks the running team while it waits for input(), and a running team cannot be saved. Fix: for a synchronous human at a terminal, UserProxyAgent is fine. For anyone else, make "we need the human" a reason to stop: HandoffTermination(target="user") or a text mention, persist state, then call run() again with the reply. Resuming is normal, so the conversation continues exactly where it left off.

You get a ToolCallSummaryMessage instead of an answer. Symptom: the agent's final message is raw JSON or a bare tool return value. Cause: reflect_on_tool_use is off, so the final message is the tool result formatted with tool_call_summary_format (default "{result}"). The default is None, which resolves to True only when output_content_type is set. Fix: set reflect_on_tool_use=True when you want prose, and leave it off when the raw result is the answer. Reflection costs one extra model call.

A local or proxied model refuses to work. Symptom: ValueError: model_info is required when model name is not a valid OpenAI model, or UserWarning: Missing required field 'structured_output' in ModelInfo, or ValueError: The model does not support function calling. Cause: the OpenAI client only knows OpenAI's models. Point it at Ollama, vLLM, LM Studio, LiteLLM or Gemini's compatible endpoint and you must describe the model yourself. Fix: pass a complete model_info with all five keys — vision, function_calling, json_output, family, structured_output. Fill structured_output even though it is only a warning today; the message says it will be required. Do not use model_capabilities=, which is the deprecated predecessor and mutually exclusive with model_info. Be honest in the dict: claiming function_calling: True for a model that lacks it moves the failure from AutoGen to the provider, where it is much harder to read.

An import fails on a fresh install of the MCP extra. Symptom: ImportError: cannot import name 'RequestContext' from 'mcp.shared.context'. Cause: autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound, and a fresh install now pulls mcp 2.x. Nothing is wrong with your code; maintenance mode means the bound is not being updated. Fix: pip install "autogen-ext[mcp]==0.7.5" "mcp<2". Assume more of these over time, which is the whole argument for a lock file.

ModuleNotFoundError for anthropic, ollama, chromadb or playwright. Cause: autogen-ext is a bag of optional integrations behind pip extras. Fix: install the extra — "autogen-ext[anthropic]", [ollama], [chromadb], [web-surfer] (which then also needs playwright install). In PowerShell keep the quotes or the shell eats the brackets.

Notebook code raises an asyncio error. Symptom: RuntimeError: asyncio.run() cannot be called from a running event loop. Cause: Jupyter already runs an event loop. Fix: top-level await in notebooks, asyncio.run(main()) in scripts. On Windows, the local code executor additionally needs asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy()).

Warnings about unclosed connections at exit. Cause: you forgot await model_client.close(). Fix: close it, ideally in a finally block or a context manager of your own, so it happens even when the run raises.

Small naming errors that stop a program dead. ValueError: The agent name must be a valid Python identifier. means you wrote "my agent" or "my-agent"; use my_agent. ValueError: The participant names must be unique. means two agents in one team share a name. ValueError: Tools cannot be used with a workbench. means you passed both tools= and workbench=; pick one. ValueError: At least two participants are required for SelectorGroupChat. means you should be using RoundRobinGroupChat.

You committed a secret without noticing. Symptom: an API key in a JSON file in git. Cause: dump_component() serialises an explicitly passed api_key into its output. Fix: never pass keys in code — let the client read OPENAI_API_KEY or ANTHROPIC_API_KEY from the environment — and never commit a dumped config. For Azure, prefer azure_ad_token_provider with azure.identity over a static key, which also keeps you out of trouble with employers who require Entra-backed identities and will ask where the key lives.

You picked the wrong team preset and paid for it. Symptom: token usage much higher than the number of agent replies explains, or a conversation whose speaker order changes between identical runs. Cause: SelectorGroupChat spends an extra model call every turn asking which agent should speak next. It also refuses to let the same agent speak twice in a row by default (allow_repeated_speaker=False), retries a bad selection up to three times, and requires at least two participants. Fix: use RoundRobinGroupChat whenever the order is known — it is deterministic, free and far easier to debug. Reach for a selector only when routing genuinely depends on the content, and when it mostly does not, pass a selector_func that decides in code and returns None to fall back to the model.

Two agents hand off at once and one handoff is silently dropped. Symptom: UserWarning: Multiple handoffs detected. Only the first is executed. Cause: the model emitted parallel tool calls, and handoffs are implemented as tool calls. Fix: set parallel_tool_calls=False on the model client. The same requirement applies to AgentTool and TeamTool, which the 0.7.2 notes document as not safe to call in parallel.

A connection error that is not your code's fault. Symptom: openai.APIConnectionError: Connection error. against a URL that works in a browser. Cause: a wrong base_url, a server that is down, or TLS interception by a corporate proxy or a cloud access gateway — common on managed laptops. Fix: check the URL first, then the CA bundle. Pointing SSL_CERT_FILE at a bundle that includes your organisation's root certificate fixes the interception case; disabling verification is not a fix, it is a different problem.

You cannot tell whether memory is doing anything. Symptom: you added a ListMemory entry and the agent appears to ignore it. Cause: no visibility. Memory is queried before each model call and injected into the context, and the only direct evidence is an event. Fix: stream the run and look for MemoryQueryEvent. If it is absent, the memory is not attached; if it is present and the answer still ignores the content, the problem is the system message, not the store.

Habits worth adopting on day one

Pin all three packages to the same exact version, and keep a lock file. autogen-agentchat, autogen-core and autogen-ext move together — agentchat pins core exactly. In a framework whose dependency bounds are no longer maintained, pip freeze > requirements.txt on day one is the difference between an environment you can rebuild next month and one you cannot. A proper lock through uv, pip-tools or poetry is better.

Smoke-test with ReplayChatCompletionClient before spending anything. It returns the strings you give it, in order, with no network and no key. Use it to prove your install, to prove your async setup, and then permanently in your tests to assert on your own orchestration — turn order, termination, state round trips — rather than on a model's prose.

Keep Console(..., output_stats=True) in every scratch script. Debugging an agent you cannot see is guesswork. The stream shows you the tool call, the tool result, the speaker selection and the token counts, and the stats line is the only honest answer to "what does this cost".

Read stop_reason every single time. It is the first thing to look at when a run surprises you, and it usually contains the whole answer.

Write a real description on every agent, even in a two-agent round robin. It costs one line, it documents the design, and it becomes load-bearing the moment you switch to SelectorGroupChat, where the selector reads it to choose a speaker.

Make tools fail usefully instead of raising. A tool returns a string to a language model, so return f"No data for {city}. Known cities: {', '.join(known)}." lets the agent recover on its own next turn. Raising gives it nothing to work with. This is the cheapest reliability improvement available to you.

Validate tool arguments like request bodies. They come from a model that may be acting on text a stranger wrote. Check ranges, restrict paths, never interpolate a model-supplied string into a shell command or a SQL statement.

Turn on caching while you develop. Model-response caching has been off by default since v0.4. Wrapping your client in ChatCompletionCache over diskcache is a few lines and saves most of the cost of running the same prompt twenty times.

Save state early. await team.save_state() returns a plain dict. Write it to a file once and open it: seeing that a team is just a conversation, a few counters and your configuration is the moment the framework stops feeling like magic.

Keep a browser tab on the release notes and the migration guide. Maintenance mode means the interesting information is in the repository and in Microsoft's AutoGen-to-Agent-Framework mapping, not in blog posts.

Prefer graceful stops to hard cancellation. ExternalTermination().set() ends a run after the current agent's turn, leaving state you can save. A CancellationToken aborts immediately and can leave the team inconsistent, so keep it for genuine emergencies and user-pressed-stop, not for ordinary control flow.

Decide early where your state lives. A team's save_state() output is a plain JSON-serialisable dict, which means it fits anywhere: a file while you prototype, Postgres or Redis in a service. Deciding this before you write the endpoint saves rewriting it, and it forces the one-team-per-session design that you need anyway. If you work for an employer with data-residency rules — common across the Gulf and Egypt — remember that this state is the conversation itself, so the store holding it falls under the same rules as any other user data, and so does your choice of model region.

The checklist

Run through this before you call a beginner AutoGen project finished.

• Virtual environment, Python 3.10 or newer, all three packages pinned to the same exact version, lock file committed.
• No import autogen anywhere, and no ConversableAgent, initiate_chat or llm_config left over from a tutorial.
• Every agent name is a valid Python identifier, and names are unique within each team.
• Every agent has a system message you wrote and a description that explains when it should speak.
• max_tool_iterations raised wherever a task needs more than one tool round.
• reflect_on_tool_use set deliberately, not by accident.
• A termination condition that is two conditions joined with |: one semantic, one bound.
• The semantic condition's keyword appears verbatim in the relevant agent's system message, and sources= restricts who can trigger it.
• A model context chosen on purpose for anything that runs more than a few turns.
• model_info complete, with all five keys, for every model that is not an OpenAI model the client recognises.
• await model_client.close() in a finally block or equivalent.
• One team per session; no team object shared across requests; state saved and loaded around the edges.
• No keys in code, none in any committed JSON; environment variables or an Azure token provider instead.
• Any model-generated code runs in Docker with an approval_func.
• Tools validate their arguments and return a helpful sentence on failure.
• You have read stop_reason and the token stats for at least one full run and can explain both.
• You know this is a maintenance-mode framework, and you have skimmed the migration guide to Microsoft Agent Framework.

Tick all of those and you have something a reviewer can read without flinching — which, on a framework nobody is adding features to any more, is exactly the standard worth holding yourself to.

This is part two of three. Beginner gave you an AssistantAgent, a model client, a RoundRobinGroupChat and a termination condition. This level explains the machinery under those four nouns precisely enough that you can predict what a team will do before you run it, then puts that machinery into a service that survives real traffic, real money and real incidents.

Everything here is verified against AutoGen 0.7.5 (autogen-agentchat, autogen-core and autogen-ext, all released 2025-09-30, Python 3.10+).

Where this picks up

Each section states the problem first, then the mechanism, then a Try it you can do on a real project in a few minutes.

The reply loop, exactly

Beginner treated AssistantAgent as a box that answers. To predict cost, latency and tool behaviour you need the loop inside it, because almost every surprising bill or truncated answer at this level comes from one of its defaults.

When a team or your own code hands an agent new messages, the agent does this:

1. Appends the incoming messages to its model context — its own view of history, not the team's.
2. Queries each configured Memory and injects the results into the context as a MemoryQueryEvent.
3. Calls model_client.create(...) with the context, the system message and the tool schemas.
4. If the response is text, that becomes the reply and the loop ends.
5. If the response is tool calls, it emits a ToolCallRequestEvent, executes the calls, emits a ToolCallExecutionEvent, and then decides what to do with the results.

Step 5 is where the defaults bite. max_tool_iterations defaults to 1, which means exactly one model-to-tool round per invocation. If the model needs to call search, read the result, and then call fetch, it cannot: the agent stops after the first batch. This default was introduced in 0.6.2 and a great many tutorials written before it describe behaviour you will not reproduce. For any agent that genuinely chains tools — a web agent, an MCP-backed agent, anything research-shaped — set it explicitly.

The second default is reflect_on_tool_use, which decides what the agent says after a tool runs. Its default is None, resolved at construction time: True when output_content_type is set, False otherwise. With reflection off, the agent's final message is a ToolCallSummaryMessage containing the raw tool result rendered through tool_call_summary_format, whose default is "{result}". That is why a weather agent sometimes replies with a bare JSON blob instead of a sentence: nothing asked the model to write prose about the result. With reflection on, the agent makes a second model call to turn tool output into an answer — better prose, roughly double the tokens for that turn.

from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def get_weather(city: str) -> str:
    """Get the weather for a given city."""
    return f"The weather in {city} is 73 degrees and Sunny."

client = OpenAIChatCompletionClient(model="gpt-4.1")

researcher = AssistantAgent(
    "researcher",
    model_client=client,
    tools=[get_weather],
    description="Looks up live conditions for a city.",   # teams route on this
    system_message="Answer in one sentence. Say TERMINATE when done.",
    max_tool_iterations=6,      # default is 1: one model->tool round only
    reflect_on_tool_use=True,   # otherwise the reply is the raw tool result
)

Two more things about that constructor matter in production. The description is not documentation: SelectorGroupChat and MagenticOneGroupChat put it in the prompt they use to pick the next speaker, so a vague description produces bad routing and you will blame the model. And the name must be a valid Python identifier; "my agent" or "my-agent" raises ValueError: The agent name must be a valid Python identifier.

Finally, agents are stateful. Their model context accumulates. Pass only the new messages into run(); replaying the whole transcript each turn duplicates every message in the context, which inflates tokens quadratically and confuses the model about what it already said.

Model clients as a contract

The model client is the only component that talks to a provider, and model_info is the contract AutoGen uses to decide what it may ask of it. Getting it wrong produces errors that look like framework bugs.

ModelInfo is a TypedDict with the keys vision, function_calling, json_output, family and structured_output, plus an optional multiple_system_messages. For recognised OpenAI model names it is inferred. For anything else — a local Ollama model, a vLLM server, LiteLLM in front of a gateway, an OpenAI-compatible endpoint — you must supply it, or you get ValueError: model_info is required when model name is not a valid OpenAI model. Omit structured_output and you get UserWarning: Missing required field 'structured_output' in ModelInfo. This field will be required in a future version of AutoGen.

from autogen_ext.models.openai import OpenAIChatCompletionClient, AzureOpenAIChatCompletionClient

# A local or gateway-fronted model over the OpenAI-compatible protocol
local = OpenAIChatCompletionClient(
    model="llama3.2",
    base_url="http://localhost:11434/v1",
    api_key="placeholder",
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": False,
        "family": "unknown",
        "structured_output": True,
    },
    timeout=60.0,
    max_retries=3,
)

# Azure OpenAI with Entra ID instead of a static key
azure = AzureOpenAIChatCompletionClient(
    azure_deployment="gpt-4o",
    model="gpt-4o",
    api_version="2024-06-01",
    azure_endpoint="https://my-resource.openai.azure.com/",
    # azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), scope)
)

The declared capability is enforced before any network call, which is a feature. tools= on a client whose model_info says function_calling: False raises ValueError: The model does not support function calling. The handoff variant is explicit about why: ValueError: The model does not support function calling, which is needed for handoffs. If you lie in the other direction — claim function_calling: True for a model that lacks it — AutoGen will happily send schemas and you will debug the provider's error instead of a clear one.

family comes from ModelFamily (GPT_5, GPT_41, GPT_4O, O3, R1, GEMINI_2_5_PRO, CLAUDE_4_SONNET, LLAMA_4_SCOUT, MISTRAL, UNKNOWN) or the literal string "unknown". It is how AutoGen adapts prompt handling per model generation, so set it honestly rather than defaulting everything to unknown.

Timeouts and retries belong on the client, not on your own wrapper: timeout and max_retries are first-class config keys alongside temperature, top_p, max_tokens, seed, stop, n, frequency_penalty, presence_penalty, logit_bias, response_format, stream_options, parallel_tool_calls, reasoning_effort, add_name_prefixes and include_name_in_message. Two of those are worth singling out. reasoning_effort landed in 0.7.5 for OpenAI GPT-5 models. And parallel_tool_calls=False is a correctness setting, not a tuning knob — several patterns below break without it.

Other clients live in autogen_ext.models.*: OllamaChatCompletionClient (which takes host, and became load_component()-able in 0.7.5), AnthropicChatCompletionClient and AnthropicBedrockChatCompletionClient, AzureAIChatCompletionClient for Foundry, SKChatCompletionAdapter for the Semantic Kernel providers, LlamaCppChatCompletionClient, and ReplayChatCompletionClient for tests.

Two deprecations to avoid in new code: model_capabilities= is replaced by model_info and warns if used (the two are mutually exclusive), and client.capabilities is replaced by client.model_info. Also response_format=<PydanticModel> in create args is deprecated in favour of json_output=<PydanticModel>.

Choosing a team shape

Beginner used RoundRobinGroupChat because it has no moving parts. Real work needs routing, and AutoGen offers four other presets. They differ in exactly one respect: who decides who speaks next. Pick on that axis and the rest follows.

SelectorGroupChat calls the model each turn with a selector_prompt containing {roles}, {participants} and {history}, and asks it for the next speaker. Defaults worth knowing: allow_repeated_speaker=False, max_selector_attempts=3, and at least two participants or you get ValueError: At least two participants are required for SelectorGroupChat. The cost is one extra LLM call per turn, which on a ten-turn conversation is a 10% to 50% surcharge depending on history size. When part of your routing is deterministic, short-circuit it:

from autogen_agentchat.teams import SelectorGroupChat

def route(messages) -> str | None:
    # Deterministic rules first; return None to fall back to the LLM.
    if messages[-1].source == "planner":
        return "searcher"
    if "ERROR" in messages[-1].to_text():
        return "fixer"
    return None

team = SelectorGroupChat(
    [planner, searcher, analyst],
    model_client=client,
    termination_condition=termination,
    selector_func=route,
    allow_repeated_speaker=False,
    max_selector_attempts=3,
)

candidate_func is the narrower cousin: it restricts the candidate set and lets the LLM choose within it. Note that neither is serialisable — a callable cannot be dumped to a component JSON, which matters for the declarative config pattern later.

Swarm moves the decision into the agents. You declare handoffs=["flights_refunder", "user"] on an agent; AutoGen turns each into a tool call that emits a HandoffMessage, and the next speaker is whatever that message targets. All agents share the same context. The first participant must be able to hand off or you get ValueError: The first participant must be able to produce a handoff messages. Handing off to "user" plus HandoffTermination(target="user") is the clean way to pause for a human.

from autogen_agentchat.teams import Swarm
from autogen_agentchat.conditions import HandoffTermination, TextMentionTermination
from autogen_agentchat.messages import HandoffMessage

travel = AssistantAgent("travel_agent", model_client=client,
                        handoffs=["flights_refunder", "user"])
refunder = AssistantAgent("flights_refunder", model_client=client,
                          handoffs=["travel_agent", "user"])

team = Swarm([travel, refunder],
             termination_condition=HandoffTermination(target="user")
                                   | TextMentionTermination("TERMINATE"))

result = await team.run(task="Refund my flight to Dubai.")
# ... collect the human's reply out of band, then resume:
last = result.messages[-1]
await team.run(task=HandoffMessage(source="user", target=last.source, content=reply))

Handoffs rely on tool calling, so a model that emits two handoff calls at once gives you UserWarning: Multiple handoffs detected. Only the first is executed: [...]. Disable parallel tool calls in the model client to avoid this warning. Set parallel_tool_calls=False on the client used by handoff agents and the ambiguity disappears.

MagenticOneGroupChat is the generalist: an orchestrator maintains a task ledger and a progress ledger, plans, notices when progress has stalled, and re-plans. Defaults are max_turns=20 and max_stalls=3. Since 0.7.2 it defaults to DockerCommandLineCodeExecutor for code. It is the most expensive preset by a wide margin — every turn involves orchestrator reasoning on top of the worker's call — so treat it as a research or exploration tool rather than the engine of a high-volume endpoint.

GraphFlow is the one to reach for when the workflow is a known shape rather than a conversation. It is still documented as experimental, which here means the API may move between minors; the behaviour in 0.7.5 is stable enough to use with pinned versions.

from autogen_agentchat.teams import DiGraphBuilder, GraphFlow

b = DiGraphBuilder()
b.add_node(writer).add_node(editor_style).add_node(editor_facts).add_node(publisher)
b.add_edge(writer, editor_style)                      # fan-out
b.add_edge(writer, editor_facts)
b.add_edge(editor_style, publisher)                   # fan-in: activation_condition="all"
b.add_edge(editor_facts, publisher)
b.set_entry_point(writer)

flow = GraphFlow(participants=b.get_participants(), graph=b.build())

add_edge(source, target, condition=None, activation_group=None, activation_condition=None) carries the control flow. condition is a string match or a callable on the message; activation_condition is "all" (the default, true fan-in) or "any" (first one through wins). Loops are allowed but ValueError: A termination condition is required for cyclic graphs without a maximum turn limit. keeps you honest. add_conditional_edges is deprecated — use add_edge(..., condition=...). Callable conditions are not serialisable.

One more GraphFlow tool deserves a mention because it solves a problem the other presets cannot: MessageFilterAgent wraps a node and limits what history it sees.

from autogen_agentchat.agents import MessageFilterAgent, MessageFilterConfig, PerSourceFilter

tight_editor = MessageFilterAgent(
    name="editor_facts",
    wrapped_agent=editor_facts,
    filter=MessageFilterConfig(
        per_source=[PerSourceFilter(source="writer", position="last", count=1)]
    ),
)

That node now sees only the writer's last message, not the whole thread — which both cuts tokens and removes a prompt-injection surface, since whatever another agent said cannot reach it.

Finally, nested teams: since 0.7.1 RoundRobinGroupChat and SelectorGroupChat accept List[ChatAgent | Team], so a sub-team can be a participant. Swarm, MagenticOneGroupChat and GraphFlow still take agents only. Nesting is how you keep a noisy sub-conversation out of the parent's context: the parent sees the sub-team's result, not its deliberation.

Termination as a safety system

A team with no termination condition and no max_turns runs until the model stops cooperating or your budget does. At this level termination stops being a tidy-up detail and becomes the main guard rail, so its semantics are worth knowing exactly.

A termination condition is a stateful callable checked after each agent response, on the delta of new messages — not on the whole transcript. It returns a StopMessage or None, composes with | (OR) and & (AND), and auto-resets after each run. The built-ins in autogen_agentchat.conditions are MaxMessageTermination(max_messages, include_agent_event=False), TextMentionTermination(text, sources=None), StopMessageTermination(), TokenUsageTermination(max_total_token=, max_prompt_token=, max_completion_token=), HandoffTermination(target), TimeoutTermination(timeout_seconds), ExternalTermination(), SourceMatchTermination(sources), TextMessageTermination(source=None), FunctionCallTermination(function_name) and FunctionalTermination(func).

The off-by-one that catches everyone: MaxMessageTermination counts the task message. With MaxMessageTermination(3) you get the task plus two replies, and the stop reason reads "Maximum number of messages 3 reached, current message count: 3". Budget accordingly.

The production shape is always a semantic condition OR'd with at least two hard stops — one on count, one on either time or tokens:

from autogen_agentchat.conditions import (
    TextMentionTermination, MaxMessageTermination,
    TokenUsageTermination, TimeoutTermination,
)

termination = (
    TextMentionTermination("APPROVE")       # the outcome you want
    | MaxMessageTermination(25)             # the conversation went in circles
    | TokenUsageTermination(max_total_token=120_000)   # the money ran out
    | TimeoutTermination(timeout_seconds=180)          # the request must answer
)

TokenUsageTermination only works because every message carries models_usage (a RequestUsage with prompt_tokens and completion_tokens) — the same field your cost dashboard reads later.

For an operator-facing stop, use ExternalTermination: hold the object, call .set(), and the team finishes the current agent's turn and stops cleanly with state you can save. That is almost always what you want for a "cancel" button. CancellationToken is the other option, and it is different in kind: .cancel() aborts the in-flight model call or code execution immediately, and the team's state may be left inconsistent. Use the token for abandoning work, external termination for stopping it.

Tools, workbenches and MCP

A tool at this level is rarely a Python function you own. It is an HTTP API, a database, a browser, or an MCP server maintained by somebody else — which changes both the plumbing and the threat model.

Plain functions still work and are still the best default: pass a sync or async function with type hints and a docstring to tools=[...] and AutoGen wraps it in a FunctionTool. The docstring becomes the description the model reads, so write it for the model.

A workbench is the next abstraction up: a set of tools sharing state and resources, with lifecycle. StaticWorkbench wraps a fixed list; McpWorkbench fronts an MCP server. An agent accepts tools= or workbench=, never both — ValueError: Tools cannot be used with a workbench. If you need both, wrap your functions in a StaticWorkbench.

from autogen_ext.tools.mcp import McpWorkbench, StdioServerParams

params = StdioServerParams(
    command="npx",
    args=["@playwright/mcp@latest", "--headless"],
    read_timeout_seconds=60,
)

async with McpWorkbench(params) as mcp:
    browser = AssistantAgent(
        "web",
        model_client=client,
        workbench=mcp,
        model_client_stream=True,
        max_tool_iterations=10,      # browsing is inherently multi-step
    )
    await Console(browser.run_stream(task="Find the latest AutoGen release notes."))

StdioServerParams takes command, args, env, cwd, encoding, encoding_error_handler and read_timeout_seconds. For a remote server use StreamableHttpServerParams(url, headers, timeout, sse_read_timeout, terminate_on_close) or the older SseServerParams(url, headers, timeout, sse_read_timeout). The workbench must be started — outside async with you get RuntimeError: McpWorkbench is not started. Call start() first. Passing a model_client to the workbench enables MCP sampling, where the server can ask your model for a completion. Since 0.6.4, tool_overrides= with ToolOverride lets you rename or re-describe a server's tools, which is how you fix a third-party server whose tool names collide with yours or whose descriptions mislead your model.

The threat model is the part people skip. An MCP server runs with your process's privileges. The README is blunt: "Only connect to trusted MCP servers." Pass credentials through env on the server params, never in prompts where they end up in transcripts and traces. And remember that tool output flows straight into your model context, so a hostile server — or a web page an honest server fetched — can attempt prompt injection. AutoGen does not mitigate that for you; MessageFilterAgent, allowlists and approval functions are your tools for narrowing it.

Agents as tools is the other composition pattern, and the one with a sharp edge:

from autogen_agentchat.tools import AgentTool, TeamTool

translator_tool = AgentTool(translator_agent, return_value_as_last_message=False)
research_tool = TeamTool(research_team, name="research",
                         description="Runs a multi-agent literature search.")

supervisor = AssistantAgent(
    "supervisor",
    model_client=OpenAIChatCompletionClient(model="gpt-4.1", parallel_tool_calls=False),
    tools=[translator_tool, research_tool],
    max_tool_iterations=5,
)

parallel_tool_calls=False on the parent's client is mandatory, not advisory: the 0.7.2 notes document that AgentTool and TeamTool must not be called in parallel. Two concurrent invocations of the same wrapped team hit the same team instance, which raises ValueError: The team is already running, it cannot run again until it is stopped.

Structured output instead of string parsing

Any agent whose reply feeds another system should return a typed object, not prose you regex. AutoGen gives you two levels of this.

At the agent level, output_content_type= takes a Pydantic model and the agent produces a StructuredMessage[T] whose .content is an instance:

from pydantic import BaseModel
from autogen_agentchat.agents import AssistantAgent

class Triage(BaseModel):
    severity: str
    service: str
    needs_human: bool
    summary: str

triage_agent = AssistantAgent(
    "triage",
    model_client=client,
    output_content_type=Triage,
    system_message="Classify the incident report.",
)

result = await triage_agent.run(task=report_text)
decision = result.messages[-1].content      # a Triage instance
if decision.needs_human:
    await page_oncall(decision)

At the client level, await client.create(messages, json_output=MyModel) does the same for one call. Both require a model whose model_info has structured_output: True; otherwise you get ValueError: Model does not support structured output. or Specified model_client doesn't support structured output mode. Note the interaction with the reply loop: setting output_content_type flips reflect_on_tool_use to True, because the agent needs a model call to shape the tool result into your schema.

The operational payoff is that a schema violation becomes a Pydantic error at the boundary, where you can retry or route to a human, instead of a silent misparse three services downstream.

Context, memory and the token bill

The default model context is UnboundedChatCompletionContext: every message ever exchanged goes to the model on every call. For a ten-turn conversation that is fine. For a support session that runs all day it is a quadratic bill and, eventually, a context-window error.

Four context implementations ship in autogen_core.model_context, and choosing between them is a real design decision:

HeadAndTailChatCompletionContext is the under-used one. In a long agent run the original task and constraints are in the first few messages; a plain buffer throws them away and the agent drifts off the brief. Keeping the head fixes that at almost no cost.

Memory is the complement. A Memory is queried before each model call and its results are injected into the context, which is how facts outlive the window:

from autogen_core.memory import ListMemory, MemoryContent, MemoryMimeType
from autogen_core.model_context import HeadAndTailChatCompletionContext

prefs = ListMemory()
await prefs.add(MemoryContent(content="User prefers metric units and Arabic dates.",
                              mime_type=MemoryMimeType.TEXT))

agent = AssistantAgent(
    "support",
    model_client=client,
    memory=[prefs],
    model_context=HeadAndTailChatCompletionContext(head_size=3, tail_size=12),
)

ListMemory lives in core; autogen-ext adds ChromaDBVectorMemory (with PersistentChromaDBVectorMemoryConfig), RedisMemory (which needs autogen-ext[redisvl] — otherwise ImportError: To use Redis Memory RedisVL must be installed.), Mem0Memory, and the Canvas and task-centric memory variants. Each query appears in the transcript as a MemoryQueryEvent, so you can see exactly what was injected when an agent says something you did not expect.

Caching is the third lever and the cheapest win in CI. Model response caching has been off by default since v0.4; you opt in by wrapping the client:

from autogen_ext.models.cache import ChatCompletionCache, CHAT_CACHE_VALUE_TYPE
from autogen_ext.cache_store.diskcache import DiskCacheStore
from diskcache import Cache

cached_client = ChatCompletionCache(
    client,
    DiskCacheStore[CHAT_CACHE_VALUE_TYPE](Cache("/tmp/autogen-cache")),
)

Swap DiskCacheStore for RedisStore from autogen_ext.cache_store.redis when several workers should share the cache. The honest use for caching is development and test loops, where you re-run the same scenario twenty times; in production, identical prompts are rarer than you hope, and a shared cache across tenants is a data-leak question you must answer deliberately.

Put the levers in order of leverage and the cost picture is simple: cap turns and tokens with termination, compact the context, skip the selector LLM call where routing is deterministic, turn reflection off where raw tool output is a fine answer, and cache in CI. Console(..., output_stats=True) prints the token stats per run, and client.total_usage() and actual_usage() give you the aggregate to export.

Code execution and the approval boundary

If any agent writes and runs code, that is the highest-risk component in your system, and 0.7.x changed its defaults specifically to make the safe path the default one.

Since 0.7.5, DockerCommandLineCodeExecutor is the default wherever a default executor is created; create_default_code_executor() in autogen_ext.code_executors prefers Docker and falls back to Local with a warning. MagenticOneGroupChat and the MagenticOne helper default to Docker since 0.7.2.

from autogen_ext.code_executors.docker import DockerCommandLineCodeExecutor
from autogen_agentchat.agents import CodeExecutorAgent, ApprovalRequest, ApprovalResponse

def approve(req: ApprovalRequest) -> ApprovalResponse:
    if "rm -rf" in req.code or "requests.post" in req.code:
        return ApprovalResponse(approved=False, reason="blocked pattern")
    return ApprovalResponse(approved=True, reason="ok")

async with DockerCommandLineCodeExecutor(
    work_dir="coding", image="python:3-slim", timeout=60
) as ex:
    coder = CodeExecutorAgent(
        "executor",
        code_executor=ex,
        model_client=client,
        approval_func=approve,
        max_retries_on_error=2,
        supported_languages=["python", "sh"],
    )

The defaults for the Docker executor are image="python:3-slim", timeout=60, auto_remove=True and stop_container=True. Use it as an async context manager; otherwise ValueError: Container is not running. Must first be started with either start or a context manager. If the daemon is not up you get RuntimeError: Failed to connect to Docker. Please ensure Docker is installed and running. — on macOS that means Docker Desktop or Colima. Leaving work_dir as the current directory emits DeprecationWarning: Using the current directory as work_dir is deprecated., and you should heed it: the executor writes files there.

approval_func arrived in 0.7.2, and without it you get UserWarning: No approval function set for CodeExecutorAgent. This means code will be executed automatically without human oversight. Treat that warning as a production blocker. The function receives an ApprovalRequest and returns an ApprovalResponse, so it can be a human prompt, a static policy as above, or a call to your own policy service. max_retries_on_error=2 lets the agent see the traceback and try again, and requires a model client — ValueError: model_client.model_info must be provided when max_retries_on_error > 0.

LocalCommandLineCodeExecutor always emits UserWarning: Using LocalCommandLineCodeExecutor may execute code on the local machine which can be unsafe. It is for a laptop and a trusted prompt, never for input a user influenced. The other options are JupyterCodeExecutor, DockerJupyterCodeExecutor for stateful notebook-style sessions, and ACADynamicSessionsCodeExecutor, which runs the code in Azure Container Apps dynamic sessions — the right choice when you want the sandbox outside your cluster, and a genuinely useful one for Gulf and Egyptian teams already on Azure in UAE North or Qatar Central, because the execution stays in-region.

A sandbox bounds the blast radius of code; it does nothing about what the code is for. Egress from the container, the secrets mounted into it, and the data it can read are yours to restrict. Deny outbound network by default unless a task needs it.

State, sessions and a real web service

A team in a notebook is a variable. A team behind an HTTP endpoint is a session that must survive across requests, workers and deploys. The mechanics are small, and three errors tell you when you have got them wrong.

The persistence API is two methods: save_state() returns a JSON-serialisable dict for the whole team, and load_state() restores it. An AssistantAgent's state is essentially its model context.

import json
from autogen_agentchat.teams import RoundRobinGroupChat

# after a run
state = await team.save_state()
await store.put(session_id, json.dumps(state))     # Postgres, Redis, Cosmos DB

# on the next request, in any worker
team = build_team()                                 # fresh objects, same shape
await team.load_state(json.loads(await store.get(session_id)))
result = await team.run(task=new_user_message)      # continues where it stopped

Three rules follow from that.

One team instance per session. Sharing a team across concurrent requests gives you ValueError: The team is already running, it cannot run again until it is stopped. Build the objects per request and rehydrate from the store.

Never touch state mid-run. load_state while running raises RuntimeError: The team cannot be loaded while it is running., and reset() mid-run raises RuntimeError: The group chat is currently running. It must be stopped before it can be reset. Stop or terminate first.

Resume is the default, not a special mode. Calling team.run() again with no task — or with a new one — continues from where it stopped. team.reset() is what clears the thread. Since 0.6.4, GraphFlow keeps its state after a termination fires and can be resumed like the others, and the stop reason lives in TaskResult.stop_reason.

The human-in-the-loop consequence is the one teams get wrong most often. UserProxyAgent(name, description="A human user", input_func=None) defaults to input(), blocks the running team, and makes it unsaveable while it waits. That is fine in a terminal and wrong in a web app, where the human may answer in ten minutes from another device. The correct pattern is to end the run and start a new one: terminate on HandoffTermination(target="user") or a text mention, persist the state, return the question to the client, and resume with the answer as a new task — a HandoffMessage for Swarm, a plain TextMessage otherwise.

For horizontal scale, keep the API workers stateless and put the state in the store. The gRPC distributed runtime exists but is explicitly experimental, with agent paging and load balancing listed as not implemented; load-state-per-request in a stateless worker is the boring arrangement that actually scales. Official samples for the FastAPI, Chainlit and Streamlit shapes live under python/samples/ in the repository.

Declarative configuration

Hard-coding team topology means a prompt change is a code deploy. AutoGen's component config turns any component into JSON and back, which is both an operational tool and the mechanism AutoGen Studio is built on.

cfg = team.dump_component()                   # a ComponentModel
open("team.json", "w").write(cfg.model_dump_json(indent=2))

# later, possibly in another service
from autogen_agentchat.teams import BaseGroupChat
team = BaseGroupChat.load_component(json.load(open("team.json")))

The dumped JSON carries provider, component_type, version, component_version, description, label and config. Model clients work the same way: ChatCompletionClient.load_component({"provider": "OpenAIChatCompletionClient", "config": {"model": "gpt-4o"}}).

Two limits decide whether this fits your design. First, callables are not serialisable — selector_func, candidate_func, GraphFlow lambda conditions and FunctionTool all fall out. A team whose routing is a Python function cannot be fully expressed as config, so pick one approach per team rather than discovering the gap at export time.

Second, and more serious: dump_component() serialises api_key into the JSON if you passed it explicitly. Rely on environment variables so the field is absent, scan exported configs before committing them, and treat any dumped config as a secret until proven otherwise.

AutoGen Studio is the GUI over this format, and its version skew is a practical trap: 0.4.2.2 requires autogen-core/agentchat/ext >=0.4.9.2,<0.6, so it cannot share a virtual environment with your 0.7.x code. Install it separately, bind it to 127.0.0.1, and believe the docs when they say it is "not meant to be a production-ready app" — it has no auth or permission model.

python3 -m venv .venv-studio && source .venv-studio/bin/activate
pip install -U autogenstudio
autogenstudio ui --host 127.0.0.1 --port 8081 --appdir ./my-app
autogenstudio serve --team team.json --port 8084     # a team JSON as an API endpoint

Testing and CI

Multi-agent systems are usually described as untestable, which is mostly a failure to separate the parts that are deterministic from the part that is not. AutoGen ships the tool that makes the split possible.

ReplayChatCompletionClient returns a scripted list of responses with no network and no key. Point it at your real team and everything except the model becomes deterministic: routing, termination, tool dispatch, state round-trips.

import pytest
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination, MaxMessageTermination
from autogen_ext.models.replay import ReplayChatCompletionClient

@pytest.mark.asyncio
async def test_team_stops_on_approval():
    client = ReplayChatCompletionClient(
        ["Here is a draft.", "Looks good. APPROVE"]
    )
    writer = AssistantAgent("writer", model_client=client)
    critic = AssistantAgent("critic", model_client=client)
    team = RoundRobinGroupChat(
        [writer, critic],
        termination_condition=TextMentionTermination("APPROVE") | MaxMessageTermination(10),
    )

    result = await team.run(task="Write one sentence about Cairo.")

    assert "APPROVE" in result.messages[-1].to_text()
    assert "APPROVE" in result.stop_reason

Four layers of testing earn their keep, in this order:

Unit tests on tools. A FunctionTool is a Python function. Test it as one, including the error paths, because a tool that raises inside an agent turns into a confusing model conversation rather than a stack trace.

Replay tests on topology. Script the responses and assert on result.stop_reason, the sequence of message.source values, and which ToolCallRequestEvents fired. These catch the regressions that actually happen: a termination condition that stopped firing, a selector that routes to the wrong agent after a description edit, a state round-trip that drops the context.

Cached live tests. A small suite against the real model with ChatCompletionCache over diskcache, committed or cached in CI. The first run costs money; later runs are free until the prompts change, which also makes the diff visible when they do.

Evaluation, not assertion, for quality. Whether the answer is good is a measurement, not a test. Build a dataset of tasks with expected properties and score runs over time — this is where a tracing and eval platform like Langfuse or an eval library such as RAGAS belongs, rather than in a pytest assertion that fails on a synonym.

Two CI practicalities. Pin all three packages to the same exact version — autogen-agentchat pins autogen-core==0.7.5, and autogen-ext must match — and keep a lock file, because of the unbounded mcp dependency described earlier. And if any test touches a Docker executor, the runner needs a Docker daemon; on Windows runners LocalCommandLineCodeExecutor additionally needs asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy()).

name: test
on: [push, pull_request]
jobs:
  pytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.lock
      - run: pytest -q          # replay-client tests need no API key

Observability and debugging beyond printing

Console(team.run_stream(...)) is the beginner's debugger and still the fastest one. Beyond it, AutoGen gives you structured logs and real OpenTelemetry spans, which is what you need when the failure happened yesterday in production.

The logging layer uses three named loggers from autogen_core: TRACE_LOGGER_NAME for human-readable debug output, EVENT_LOGGER_NAME for structured events, and ROOT_LOGGER_NAME. Use child loggers so you can tune one module at a time:

import logging
from autogen_core import TRACE_LOGGER_NAME, EVENT_LOGGER_NAME

logging.basicConfig(level=logging.WARNING)
logging.getLogger(TRACE_LOGGER_NAME).setLevel(logging.DEBUG)
logging.getLogger(EVENT_LOGGER_NAME).setLevel(logging.INFO)
logging.getLogger(f"{TRACE_LOGGER_NAME}.my_app.routing").setLevel(logging.DEBUG)

The tracing layer is more useful at this level. The runtime is instrumented with GenAI semantic-convention spans — create_agent, invoke_agent and execute_tool, added in 0.6.2 — so a run becomes a trace you can read as a tree: which agent spoke, which tool it called, how long each model call took, and where the retry loop went round twice.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from autogen_core import SingleThreadedAgentRuntime

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
OpenAIInstrumentor().instrument()          # adds prompt/completion spans

runtime = SingleThreadedAgentRuntime(tracer_provider=provider)

Install the pieces with pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc opentelemetry-instrumentation-openai, and point the exporter at anything that speaks OTLP: Jaeger locally (jaegertracing/all-in-one, UI on 16686, OTLP on 4317 and 4318), or your existing backend — Grafana and Tempo, or an LLM-aware platform such as Langfuse or Arize Phoenix, which understand the GenAI conventions and will show you prompts and token counts per span. Pass tracer_provider= explicitly or set the global provider; to turn the runtime's own spans off, use a NoOpTracerProvider or set AUTOGEN_DISABLE_RUNTIME_TRACING=true.

For cost and quality telemetry you do not need OTel at all, because the data is in the messages. Every message carries models_usage; sum it per run, tag it with the session and tenant, and you have per-conversation cost without a vendor. Console(..., output_stats=True) prints the same thing interactively.

The debugging questions that come up most, and where to look:

• Why did that agent speak? SelectSpeakerEvent in the transcript, and the description fields the selector read.
• Why did it stop? TaskResult.stop_reason, verbatim.
• Why did the tool not run twice? max_tool_iterations, which defaults to 1.
• Why is the answer raw JSON? reflect_on_tool_use is False, so you are seeing ToolCallSummaryMessage.
• Why did context explode? UnboundedChatCompletionContext, plus full history being passed back into run().
• Why does it work in a notebook and not in a script? asyncio.run() versus top-level await; RuntimeError: asyncio.run() cannot be called from a running event loop is the notebook form.
• Why does this tutorial's code not import? import autogen is AG2 or the legacy v0.2 API. Microsoft AutoGen imports are autogen_agentchat, autogen_core and autogen_ext, with underscores.

Deployment, and the migration question

Deploying AutoGen is deploying a Python application: there is no server, no scheduler and no control plane to operate. That makes the packaging boring, which is the good news, and it means every operational property is one you must supply.

The shape that works is a stateless container behind your usual ingress. Build it with Docker, expose the team through FastAPI, keep session state in Postgres or Redis, and run it on Kubernetes with the ordinary health checks and resource limits. Because the workers hold no session state, scaling is horizontal and restarts are safe.

A deployment checklist that maps onto what this guide covered:

1. Pin everythingExact versions of autogen-agentchat, autogen-core and autogen-ext, plus a lock file that holds mcp<2. Maintenance mode means dependency bounds will not be fixed for you.
2. Bound every runA semantic termination condition OR'd with message, token and time limits, and max_turns as a backstop.
3. Externalise session statesave_state() to a store keyed by session; one team instance per request; no UserProxyAgent in the request path.
4. Sandbox and approve codeDocker or ACA dynamic sessions, an approval_func, no outbound network unless required.
5. Export telemetryOTLP spans plus per-run models_usage tagged with tenant and session.
6. Guard the config pathComponent JSON only from a store you control, and never a dumped config containing api_key.
7. Own the security patchesNo upstream feature work means you watch the repository yourself, including fixes that land on main and never ship.

That last point leads to the decision you cannot avoid. Microsoft's Agent Framework was built by the AutoGen and Semantic Kernel teams and is at 1.x, and Microsoft publishes an official AutoGen-to-MAF migration guide on Microsoft Learn. The mapping is close enough to be reassuring:

Two differences the guide calls out are worth internalising even if you never migrate. AssistantAgent "is single-turn unless you increase max_tool_iterations", whereas MAF's Agent is multi-turn by default — the same default that surprises everyone, acknowledged and reversed. And AutoGen has "embedded and experimental distributed runtimes" while MAF "focuses on single-process composition today; distributed execution is planned", which confirms that the stateless-workers pattern was the right one all along. Note also that the Anthropic and Ollama clients and ChatCompletionCache are listed as planned in MAF, so a migration today may cost you a model provider.

A pragmatic rule: a working AutoGen 0.7.5 service with pinned dependencies does not need to be rewritten this quarter. A new system you expect to run for three years should start on the successor, or on another framework in this catalogue — LangGraph if the workflow is a graph you want to own explicitly, CrewAI for role-based crews, the OpenAI Agents SDK or the Claude Agent SDK if you are committed to one provider. What transfers is everything in this guide: bounded runs, externalised state, sandboxed execution, structured output, and telemetry you can read after the fact.

Putting it all together

Here is the mid-level shape in one file: a bounded, sandboxed, instrumented team with a compacted context, structured output, external state, and no default left at a value that would surprise you.

import asyncio, json, logging
from pydantic import BaseModel
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import SelectorGroupChat
from autogen_agentchat.conditions import (
    TextMentionTermination, MaxMessageTermination,
    TokenUsageTermination, TimeoutTermination,
)
from autogen_core import TRACE_LOGGER_NAME
from autogen_core.model_context import HeadAndTailChatCompletionContext
from autogen_ext.models.openai import OpenAIChatCompletionClient

logging.basicConfig(level=logging.WARNING)
logging.getLogger(TRACE_LOGGER_NAME).setLevel(logging.INFO)

class Verdict(BaseModel):
    decision: str
    rationale: str

def build_team(client) -> SelectorGroupChat:
    ctx = lambda: HeadAndTailChatCompletionContext(head_size=3, tail_size=10)

    async def lookup_policy(topic: str) -> str:
        """Return the company policy text for a topic."""
        return POLICIES.get(topic, "No policy found.")

    researcher = AssistantAgent(
        "researcher", model_client=client, tools=[lookup_policy],
        description="Finds the relevant internal policy for a question.",
        model_context=ctx(), max_tool_iterations=4, reflect_on_tool_use=True,
    )
    reviewer = AssistantAgent(
        "reviewer", model_client=client,
        description="Decides whether the researcher's answer is compliant.",
        model_context=ctx(), output_content_type=Verdict,
        system_message="Return a decision of APPROVE or REJECT with a rationale.",
    )

    def route(messages):
        return "reviewer" if messages[-1].source == "researcher" else None

    return SelectorGroupChat(
        [researcher, reviewer],
        model_client=client,
        selector_func=route,                 # skips the selector LLM call
        termination_condition=(
            TextMentionTermination("APPROVE")
            | MaxMessageTermination(20)
            | TokenUsageTermination(max_total_token=100_000)
            | TimeoutTermination(timeout_seconds=120)
        ),
        max_turns=12,
    )

async def handle_request(session_id: str, message: str, store) -> dict:
    client = OpenAIChatCompletionClient(
        model="gpt-4.1", parallel_tool_calls=False, timeout=60.0, max_retries=3,
    )
    try:
        team = build_team(client)
        saved = await store.get(session_id)
        if saved:
            await team.load_state(json.loads(saved))

        result = await team.run(task=message)

        await store.put(session_id, json.dumps(await team.save_state()))
        tokens = sum(
            m.models_usage.prompt_tokens + m.models_usage.completion_tokens
            for m in result.messages if m.models_usage
        )
        logging.info("session=%s tokens=%s stop=%s",
                     session_id, tokens, result.stop_reason)
        return {"reply": result.messages[-1].to_text(),
                "stop_reason": result.stop_reason, "tokens": tokens}
    finally:
        await client.close()

Every line of that is a decision from this guide. parallel_tool_calls=False because handoffs and agent-tools require it. timeout and max_retries on the client rather than a wrapper. max_tool_iterations=4 because the default of 1 would truncate the lookup. reflect_on_tool_use=True because the raw policy text is not an answer. description written for the selector, not for humans. selector_func to skip a model call where routing is deterministic. HeadAndTailChatCompletionContext so the brief survives a long session. output_content_type so the reviewer's verdict is a type. Four termination conditions plus max_turns. State loaded and saved per request so any worker can serve any session. Usage summed and logged with the stop reason. The client closed in a finally.

Where you are now

You can predict what an AutoGen team will do before running it, because you know who chooses the speaker in each preset, what the reply loop does with tool results, and which defaults — max_tool_iterations=1, reflection off, unbounded context, caching off, no approval function — differ from what you would have assumed. You can put a team behind an HTTP endpoint that survives concurrency, restarts and deploys, and you know the three errors that tell you when state handling is wrong. You can bound a run on meaning, messages, tokens and time at once; execute model-written code behind a policy function in a container; export both OTel traces and per-run token costs; test the whole topology in CI with no API key; and move prompts and topology into configuration with the two caveats about callables and serialised keys firmly in mind.

You also know where AutoGen sits: a capable, frozen 0.7.5 with an official successor and a documented migration path, which is a fact to plan around rather than a reason to panic.

What the Senior guide adds is ownership. The Core API underneath AgentChat — the actor runtime, AgentId, topics and subscriptions, intervention handlers — and what it buys you in multi-tenancy and custom routing. Which component breaks first under load, and the architecture that follows from that. The full trust model: prompt injection across handoffs and tool boundaries, the component-loading risk, supply chain and secrets. Multi-tenant isolation, cost attribution, upgrade and migration strategy for a fleet of teams, incident playbooks, and running AutoGen as a platform other teams build on — including the review checklist you apply to their pull requests and the honest answer to where AutoGen stops.

Sources

• AutoGen documentation (stable)
• microsoft/autogen on GitHub (README and maintenance-mode notice)
• AutoGen release notes and python-v0.7.5
• AgentChat tutorial: agents
• AgentChat tutorial: teams
• AgentChat tutorial: termination
• AgentChat tutorial: models
• AgentChat tutorial: state
• AgentChat tutorial: human in the loop
• SelectorGroupChat
• Swarm
• GraphFlow
• Magentic-One
• Memory
• Serializing components
• Tracing and logging
• Workbench
• Command-line code executors
• Component config
• Azure OpenAI with AAD auth
• Extensions installation and extras
• AutoGen Studio user guide
• Migration guide: v0.2 to v0.4+
• Microsoft Learn: migrating from AutoGen to Agent Framework
• autogen-agentchat on PyPI

Mid-level AutoGen interviews are not vocabulary tests. The interviewer assumes you can build an AssistantAgent and wants to know whether you have run one in production: whether you know which defaults will cost money, where state lives, what happens when the model writes code, and whether you have an opinion about a framework Microsoft has frozen. The questions below are the ones that actually get asked, with the reasoning a strong answer shows.

Everything here is current for AutoGen 0.7.5.

The thirty-second framing

If you are asked to situate AutoGen, lead with the three layers and the status, in that order.

"AutoGen is Microsoft's multi-agent framework in three layers. autogen-core is an event-driven actor runtime: agents, message passing, topics and subscriptions. autogen-agentchat is the high-level task-driven API on top of it, with preset agents and team shapes. autogen-ext holds the implementations — model clients, code executors, memory, MCP tools — behind pip extras. Most applications only touch AgentChat. Worth saying up front: it is in maintenance mode, the README points new users at Microsoft Agent Framework, and 0.7.5 from September 2025 is the last PyPI release. I would run existing AutoGen systems with pinned dependencies and start greenfield work on the successor."

Volunteering the maintenance-mode fact is a signal: a candidate who omits it either has not looked at the repository recently or is hiding a problem.

Scenario: "Our agent costs five times what we budgeted. What do you check first?"

Go in order of leverage, and name the mechanism behind each item rather than listing knobs.

First, the turn count. A team with no termination condition and no max_turns runs until the model stops cooperating. Check result.stop_reason: if it reads "Maximum number of messages ... reached" every time, the conversation is not converging and the limit is the only thing ending it. Note the off-by-one — MaxMessageTermination counts the task message.

Second, the context. The default is UnboundedChatCompletionContext, so every call resends the whole history and cost grows quadratically in turns. Swap in BufferedChatCompletionContext, HeadAndTailChatCompletionContext or TokenLimitedChatCompletionContext — head-and-tail, because a plain buffer discards the original task and the agent drifts off the brief, producing more turns and more cost.

Third, the team shape. SelectorGroupChat spends an extra model call per turn choosing a speaker, with history in the prompt; a selector_func short-circuits the deterministic hops and returning None falls back to the LLM. MagenticOneGroupChat is the most expensive preset by a distance, because the orchestrator reasons on top of every worker turn.

Fourth, reflection. reflect_on_tool_use=True adds a second model call after every tool round. Sometimes that makes the answer readable; sometimes the raw result was fine and you paid double for prose nobody reads.

Fifth, measure instead of guessing. Every message carries models_usage, and the client exposes total_usage() and actual_usage(). Sum per run, tag with session and tenant, and the dominant cost becomes obvious. Then add limits so it cannot recur: TokenUsageTermination(max_total_token=...) as a ceiling, and ChatCompletionCache over diskcache or Redis in development and CI.

A weak answer says "use a smaller model". It is the one lever that trades quality for cost, and the last one to pull.

Scenario: "The same conversation gives different results depending on which pod handles the request."

This is a state question: an AutoGen team is an in-process object with in-memory state, so unless you externalise it, each pod holds its own version of the conversation.

The fix is save_state() and load_state(). save_state() returns a JSON-serialisable dict for the whole team; persist it keyed by session in Postgres, Redis or Cosmos DB, and on each request build fresh team objects of the same shape, load_state(), run, and save again. Workers stay stateless, so any pod serves any session and a deploy does not drop a conversation.

Three constraints go with it, and naming them shows you have hit them. Never share a team instance across concurrent requests, or you get ValueError: The team is already running, it cannot run again until it is stopped. Never touch state mid-run: load_state raises RuntimeError: The team cannot be loaded while it is running. and reset() raises RuntimeError: The group chat is currently running. It must be stopped before it can be reset. And remember that resume is the default — calling run() again continues the thread, while reset() is what clears it.

If the interviewer pushes on the distributed runtime, be straight: the gRPC worker runtime is explicitly experimental, agent paging and load balancing are documented as not implemented, and breaking changes are expected. Stateless workers loading state per request are the arrangement that scales, and the Agent Framework guide confirms the direction by stating that MAF "focuses on single-process composition today".

Scenario: "An agent calls one tool and stops, when it clearly needs two."

This is the single most common AutoGen bug report and the answer is one line: AssistantAgent.max_tool_iterations defaults to 1, which allows exactly one model-to-tool round per invocation. Raise it.

What makes the answer strong is the surrounding detail. The parameter was added in 0.6.2, so tutorials written before it describe behaviour you cannot reproduce. Anything inherently multi-step needs it set explicitly. The related default is reflect_on_tool_use, which is None and resolves to True only when output_content_type is set; with it off the final message is a ToolCallSummaryMessage containing the raw tool result formatted with tool_call_summary_format (default "{result}"), which is why some agents reply with bare JSON.

If you want to score the point fully, mention that the Agent Framework migration guide calls this out explicitly: AssistantAgent "is single-turn unless you increase max_tool_iterations", whereas MAF's Agent is multi-turn by default. Knowing that the successor reversed the default proves you read the primary sources.

Scenario: "An agent needs to hand off to a human who may answer tomorrow."

The trap is UserProxyAgent. It defaults to input(), blocks the running team, and the team cannot be saved while it waits — correct in a terminal, wrong in any web application.

The pattern that works is terminate-and-resume: end the run on HandoffTermination(target="user") in a Swarm or a TextMentionTermination elsewhere, save_state(), return the question to the client, then rebuild the team, load_state() and run() with the reply. For a Swarm the reply goes back as HandoffMessage(source="user", target=last_message.source, content=reply). Volunteer the follow-up: set parallel_tool_calls=False, because handoffs are tool calls and a model emitting two at once produces UserWarning: Multiple handoffs detected. Only the first is executed.

Trade-off: "RoundRobin, Selector, Swarm, Magentic-One or GraphFlow?"

Answer on one axis — who decides who speaks next — and the rest follows. Interviewers are listening for a decision procedure, not five feature lists.

• RoundRobinGroupChat: you decide, as a fixed order. Zero routing cost, fully predictable. Right for writer-critic loops and fixed pipelines.
• SelectorGroupChat: an LLM decides from the agents' description fields. Flexible, costs one model call per turn, and routing quality depends entirely on how well those descriptions are written. Defaults: allow_repeated_speaker=False, max_selector_attempts=3, minimum two participants.
• Swarm: the agents decide, by emitting a HandoffMessage. Right when the domain already has explicit transfers — triage to specialist, L1 to L2. Requires function calling, and the first participant must be able to hand off.
• MagenticOneGroupChat: an orchestrator with a task and progress ledger decides, and re-plans on a stall (max_turns=20, max_stalls=3). Expensive; better as a research tool than a high-volume endpoint.
• GraphFlow: you decide, as a directed graph with conditions, fan-out, fan-in and loops. Right when the workflow has a known shape. Still experimental, so pin versions.

Add the composition note: since 0.7.1, RoundRobinGroupChat and SelectorGroupChat accept a Team as a participant, so the parent sees a sub-team's result rather than its deliberation; Swarm, MagenticOneGroupChat and GraphFlow take agents only. What a weak answer misses is that description is not documentation — the selector reads it, and vague descriptions produce bad routing that candidates blame on the model.

Trade-off: "AgentChat or the Core API?"

Default to AgentChat: it is the task-driven layer with the presets, state persistence and termination, and most applications never need anything else. Reach for autogen-core when you need something the conversation abstraction does not express — event-driven workflows rather than turn-taking, routing by message type, agents in separate processes, or real multi-tenant isolation. Core's model is an actor system where an AgentId is a (type, key) pair, the type is bound to a factory rather than a class, and instances are created lazily on first message; TypeSubscription(topic_type, agent_type) maps a topic's source to an agent key, so publishing to TopicId(type="tasks", source="tenant-42") reaches that tenant's own instance. That is the clean multi-tenancy story and the main reason to drop a level. Say what it costs: you write message types, handlers, registration and runtime lifecycle yourself, and lose the presets. Picking Core for a writer-critic loop is complexity bought for nothing.

Scenario: "An agent writes and runs Python. Convince me it is safe."

Start by narrowing the claim: a sandbox bounds the blast radius of code, it does not decide what the code is for. Then give the layers.

Execution goes in a container. DockerCommandLineCodeExecutor has been the default wherever a default executor is created since 0.7.5, and create_default_code_executor() prefers Docker and falls back to Local with a warning. Defaults are image="python:3-slim", timeout=60, auto_remove=True, stop_container=True, and it must be used as an async context manager or you get ValueError: Container is not running. LocalCommandLineCodeExecutor always warns that it "may execute code on the local machine which can be unsafe" and has no place in a service.

Every execution passes an approval function. CodeExecutorAgent(approval_func=...) arrived in 0.7.2, and without it you get UserWarning: No approval function set for CodeExecutorAgent. This means code will be executed automatically without human oversight. The function takes an ApprovalRequest and returns an ApprovalResponse, so it can be a human prompt, a static deny-pattern policy, or a call to your own policy service.

Then the parts the framework does not do. Container egress, mounted secrets and readable data are yours to restrict. ACADynamicSessionsCodeExecutor moves execution into Azure Container Apps dynamic sessions, which keeps it off your cluster and, for teams on Azure in UAE North or Qatar Central, keeps it in-region. Prompt injection through tools and handoffs is not mitigated at all; allowlists, intervention handlers and MessageFilterAgent are your tools there.

Raise the risk nobody asks about: in released 0.7.5, load_component() imports whatever module path the provider string names, so untrusted component JSON is an arbitrary-import primitive. The namespace restriction and AUTOGEN_ALLOWED_PROVIDER_NAMESPACES exist on main and in no PyPI release.

Scenario: "How do you test this? It is a multi-agent system."

Separate the deterministic parts from the model, then test the deterministic parts properly.

ReplayChatCompletionClient from autogen_ext.models.replay returns a scripted list of responses with no network and no API key. Point it at your real team and routing, termination, tool dispatch and state round-trips all become deterministic. You can assert on result.stop_reason, on the ordered list of message.source values, and on which ToolCallRequestEvents fired — which catches the regressions that actually happen, like a description edit that silently changes selector routing.

Four layers, in order: unit tests on tools, since a FunctionTool is just a function; replay tests on topology; a small cached live suite using ChatCompletionCache; and evaluation rather than assertion for answer quality, scored over a dataset instead of a pytest assertion that fails on a synonym.

Mention the CI practicalities and you sound like someone who has done it: pin all three packages to the same exact version, because autogen-agentchat pins autogen-core==0.7.5 and autogen-ext must match; keep a lock file, because autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound and a fresh install breaks with ImportError: cannot import name 'RequestContext' from 'mcp.shared.context' until you pin mcp<2; and if any test uses a Docker executor, the runner needs a daemon.

Scenario: "Production says an agent gave a wrong answer yesterday. Debug it."

The answer should be about stored telemetry, not reproduction, because reruns of a nondeterministic system prove nothing.

AutoGen's runtime is instrumented with GenAI semantic-convention OpenTelemetry spans — create_agent, invoke_agent and execute_tool, added in 0.6.2 — so a run is a trace tree: which agent spoke, which tool it called, how long each model call took, where a retry went round twice. Pass tracer_provider= to the runtime or set the global provider, and export over OTLP to Jaeger, Tempo or an LLM-aware backend that understands the conventions. AUTOGEN_DISABLE_RUNTIME_TRACING=true turns the runtime spans off when they are noise. Alongside that, the three named loggers from autogen_core — TRACE_LOGGER_NAME, EVENT_LOGGER_NAME and ROOT_LOGGER_NAME — take child loggers, so you can raise verbosity for one module instead of the whole framework.

Then name the specific places to look: TaskResult.stop_reason says verbatim why the run ended, SelectSpeakerEvent says why that agent spoke, MemoryQueryEvent says what was injected into the context, and the tool-call events carry the arguments and results. Persist the transcript with the state and the whole chain is readable after the fact.

Scenario: "Would you start a new project on AutoGen today?"

This is a judgement question and the interviewer wants a defensible position, not enthusiasm in either direction.

The honest answer is no for a system expected to run for years, and that is not a criticism of the code. AutoGen is in maintenance mode: no new features, community managed, last PyPI release 0.7.5 in September 2025, and security fixes landing on main that never ship. The unbounded mcp dependency that now breaks on install is exactly what maintenance mode looks like.

Microsoft Agent Framework is the successor, built by the AutoGen and Semantic Kernel teams, at 1.x, with an official migration guide on Microsoft Learn. The mapping is close: AssistantAgent to Agent, FunctionTool to the @tool decorator, AgentTool(agent) to agent.as_tool(), RoundRobinGroupChat to SequentialBuilder, MagenticOneGroupChat to MagenticBuilder, GraphFlow to WorkflowBuilder, and model context and state to AgentSession. Note the gap: the Anthropic and Ollama clients and ChatCompletionCache are listed as planned, so migrating today may cost you a provider.

For an existing service the answer is different: a working 0.7.5 deployment with pinned dependencies does not need rewriting this quarter. Put migration on the roadmap and spend the time on what transfers anyway.

What separates a strong answer from a weak one

A strong answer names the default. "It stopped after one tool call because max_tool_iterations defaults to 1" beats "I would check the loop." Mid-level AutoGen is largely a knowledge of defaults that differ from intuition: one tool round, reflection off, unbounded context, caching off since v0.4, no approval function, MaxMessageTermination counting the task.

A strong answer quotes the error. Saying ValueError: The team is already running, it cannot run again until it is stopped. proves you hit it in production. Describing it vaguely proves you read about it.

A strong answer separates the framework's job from yours. AutoGen gives you a sandbox; you give it egress rules. An approval hook; you write the policy. OTel spans; you run the collector. Assuming the framework handles prompt injection or tenant isolation loses credibility fast.

A strong answer measures. Token costs come from models_usage on every message and client.total_usage(), not from an estimate. Interviewers notice the difference between "Selector is more expensive" and "Selector cost us about 30% more prompt tokens on a ten-turn task, so we added a selector_func for the deterministic hop."

A strong answer knows the ecosystem confusion. On PyPI, autogen and ag2 belong to AG2, the community fork, with a v0.2-style API (ConversableAgent, initiate_chat); pyautogen is now a proxy package depending on autogen-agentchat, and Microsoft has not had admin access to it since 0.2.34. Microsoft AutoGen imports use underscores, so any import autogen in a snippet means it is not this framework.

A strong answer volunteers the status. Maintenance mode, the successor, and the migration table. A candidate who presents AutoGen as the obvious choice for a new system in 2026 has not checked, and everything else they say gets discounted.

Sixty-second self-test

Answer out loud, then check yourself above. What is the default max_tool_iterations and what breaks because of it? What does the final message contain when reflect_on_tool_use is false? Who chooses the next speaker in each of the five presets? How many messages does MaxMessageTermination(3) allow? Which two methods persist a team, and which three errors mean you are misusing them? Why is UserProxyAgent wrong in a web application? Which setting must be false for handoffs and TeamTool? What has been the default code executor since 0.7.5? Which field carries token usage? What are the three OpenTelemetry span names? Why must you pin mcp<2? And what is the one-sentence status of the project?

If you can answer those without hedging, you can hold a mid-level AutoGen conversation with anyone who has run it in production.

These are the things that go wrong once you are past the tutorials and running AutoGen 0.7.5 for real: the errors that waste an afternoon, the practices that pay back fastest, and the recipes worth keeping in a snippet file.

The errors that cost the most time

The agent calls one tool and stops. AssistantAgent.max_tool_iterations defaults to 1: one model-to-tool round per invocation. Any agent that must read a tool result and then call another tool needs it raised. Symptom: a research or browsing agent that answers from the first search hit and ignores the follow-up. Fix: set it explicitly, something like max_tool_iterations=6 for MCP-backed agents. The parameter arrived in 0.6.2, so older tutorials cannot have mentioned it.

The reply is raw JSON instead of a sentence. reflect_on_tool_use defaults to None, which resolves to True only when output_content_type is set and False otherwise. With it off, the final message is a ToolCallSummaryMessage rendered through tool_call_summary_format (default "{result}"). Fix: set reflect_on_tool_use=True if you want prose, and accept the extra model call, or set a tool_call_summary_format that reads acceptably.

ValueError: The team is already running, it cannot run again until it is stopped. You shared one team instance across concurrent requests, which is the normal mistake when a team is created at module import time in a web app. Fix: one team instance per session, built per request and rehydrated with load_state().

RuntimeError: The team cannot be loaded while it is running. or RuntimeError: The group chat is currently running. It must be stopped before it can be reset. State operations are not allowed mid-run. Terminate or cancel first, then load or reset.

ValueError: model_info is required when model name is not a valid OpenAI model. You pointed OpenAIChatCompletionClient at a local model, vLLM, LM Studio, LiteLLM or an Ollama OpenAI endpoint. Fix: pass a full model_info dict with vision, function_calling, json_output, family and structured_output. Omitting the last gives UserWarning: Missing required field 'structured_output' in ModelInfo.

ValueError: The model does not support function calling. Your model_info says function_calling: False and you passed tools=. The handoff variant reads ...which is needed for handoffs. Fix the capability declaration or the model — and note AutoGen checks this before any request leaves the process, which is a feature.

ValueError: The agent name must be a valid Python identifier. "my agent" and "my-agent" are rejected. Use my_agent. Related: ValueError: The participant names must be unique.

ValueError: Tools cannot be used with a workbench. An agent takes tools= or workbench=, never both. Wrap your plain functions in a StaticWorkbench if you need them alongside an McpWorkbench.

ImportError: cannot import name 'RequestContext' from 'mcp.shared.context'. autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound, so a fresh install pulls mcp 2.x. Fix: pip install "autogen-ext[mcp]==0.7.5" "mcp<2". This is maintenance mode in practice and the strongest argument for a lock file.

RuntimeError: McpWorkbench is not started. Call start() first. Use the workbench inside async with, or call start() yourself.

RuntimeError: Failed to connect to Docker. Please ensure Docker is installed and running. The daemon is down. On macOS that means Docker Desktop or Colima. Related: ValueError: Container is not running. Must first be started with either start or a context manager. means you skipped async with.

UserWarning: No approval function set for CodeExecutorAgent. Treat this as a production blocker rather than noise: code is executing with no policy in front of it.

UserWarning: Multiple handoffs detected. Only the first is executed. The model emitted two handoff tool calls at once. Set parallel_tool_calls=False on the client used by handoff agents.

ValueError: A termination condition is required for cyclic graphs without a maximum turn limit. Your GraphFlow has a loop. Add termination_condition= or max_turns=.

ModuleNotFoundError: No module named 'anthropic' / 'ollama' / 'chromadb' / 'grpc' / 'playwright'. A missing extra. Install autogen-ext[anthropic], [ollama], [chromadb], [grpc], [web-surfer] and so on. RedisMemory has its own: ImportError: To use Redis Memory RedisVL must be installed.

ModuleNotFoundError: No module named 'autogen' or AttributeError on ConversableAgent. You are following AG2 or legacy v0.2 code. Microsoft AutoGen imports are autogen_agentchat, autogen_core and autogen_ext, with underscores.

RuntimeError: asyncio.run() cannot be called from a running event loop. You ran a script's asyncio.run(...) in a notebook. Use top-level await there, and asyncio.run(...) only in scripts.

DeprecationWarning: model_capabilities is deprecated, use model_info instead. Rename it; the two are mutually exclusive. Same family: add_conditional_edges on DiGraphBuilder (use add_edge(..., condition=...)), response_format=<PydanticModel> in create args (use json_output=), and work_dir left as the current directory on an executor.

The team answers, then the next request forgets everything. You saved state but rebuilt the team with a different shape. load_state() restores into the structure you give it, so the participants, their names and their order must match what was saved. Keep a single build_team() factory and call it from both the save and the load path; a renamed agent is a silently broken session.

Two agents, same answer, twice the bill. SelectorGroupChat defaults to allow_repeated_speaker=False, but with only two agents the alternation can turn into an echo where each restates the other. The cure is not a setting but a prompt: give each agent a description and a system_message that make its job disjoint from the other's, and add a TextMentionTermination on a word only the deciding agent is told to say.

A tool raises and the conversation gets strange instead of failing. A FunctionTool that throws does not crash the run; the error becomes text the model reads and reasons about. That is usually what you want, but it means a broken database connection can look like a model quality problem. Catch exceptions inside the tool and return an explicit, short, machine-readable failure string, and log the real traceback yourself.

The practices that pay off, in order of leverage

1. Bound every run four ways. A semantic condition for the outcome you want, OR'd with a message cap, a token cap and a timeout, plus max_turns on the team as a backstop. This is the single highest-leverage habit in AutoGen because it converts an open-ended loop into a bounded one, and it protects cost, latency and your on-call rota at once. Remember that MaxMessageTermination counts the task message.

2. Set the defaults you do not want. Write a factory function that constructs your agents, and set max_tool_iterations, reflect_on_tool_use, model_context and description explicitly in it, every time. The defaults are reasonable for a tutorial and wrong for most production agents, and a factory means you decide once rather than per file.

3. Externalise state from day one. save_state() to a store keyed by session, load_state() per request, stateless workers. Retrofitting this is a rewrite of your request handling; doing it at the start costs twenty lines.

4. Write description for the selector, not for humans. SelectorGroupChat and MagenticOneGroupChat put it in the routing prompt. "Finds the relevant internal policy for a question and returns its text" routes well; "the researcher" does not. This is the cheapest quality improvement available in a multi-agent team.

5. Compact the context deliberately. Pick BufferedChatCompletionContext, HeadAndTailChatCompletionContext or TokenLimitedChatCompletionContext based on what the agent needs to remember. Head-and-tail is the most under-used: it keeps the original brief, which a plain buffer throws away, causing drift and therefore more turns.

6. Pin all three packages and keep a lock file. autogen-agentchat pins autogen-core==0.7.5; autogen-ext must match. Maintenance mode means dependency bounds are not being corrected, so the lock file is the only thing standing between you and the mcp 2.x break.

7. Put every tool behind a test and every topology behind a replay test. ReplayChatCompletionClient makes routing, termination and state deterministic with no API key. A description edit that breaks routing then fails in CI rather than in production.

8. Export telemetry before you need it. OTLP spans from the instrumented runtime, plus a per-run sum of models_usage tagged with session and tenant. Incidents are investigated from stored traces, because rerunning a nondeterministic system proves nothing.

9. Never let load_component() see untrusted JSON. In 0.7.5 it imports whatever module path provider names. The namespace restriction exists on main and in no release.

10. Sandbox and approve code execution. Docker executor, an approval_func, no container egress unless the task needs it. The warning about a missing approval function is telling you something true.

Practice cards

Each card is a self-contained exercise with a stated outcome. They take between ten minutes and an hour and each one removes a class of production surprise.

Operational recipes

Cut the cost of a chatty team. In order: add TokenUsageTermination and a tighter MaxMessageTermination; swap the context for head-and-tail; add a selector_func for deterministic hops; turn off reflect_on_tool_use where the raw tool result is an acceptable answer; and wrap the client in ChatCompletionCache for development and CI. Measure with models_usage after each change rather than applying them all at once.

Stop a run cleanly from outside. Hold an ExternalTermination object and call .set(). The team finishes the current agent's turn and stops with state you can save, which is what a cancel button needs. Keep CancellationToken.cancel() for abandoning work — it aborts the in-flight call immediately and may leave the team's state inconsistent.

Add an MCP server safely. Pin mcp<2. Use StdioServerParams for a local process or StreamableHttpServerParams for a remote one, pass credentials through env rather than prompts, use the workbench inside async with, set max_tool_iterations high enough for multi-step work, and rename misleading tools with tool_overrides=. Remember the server runs with your privileges and its output lands in your model context, so treat it as an injection surface.

Narrow what an agent can see. Wrap a GraphFlow node in MessageFilterAgent with a MessageFilterConfig of PerSourceFilters. It cuts tokens and removes a prompt-injection path at the same time, because whatever another agent said never reaches that node.

Move prompts and topology into config. team.dump_component(), write the JSON, load it with BaseGroupChat.load_component() in the service. Two caveats: callables (selector_func, candidate_func, GraphFlow lambdas, FunctionTool) are not serialisable, and dump_component() writes api_key into the JSON if you passed it explicitly — rely on environment variables and scan exported configs before committing them.

Keep one agent out of another's context. Three mechanisms, in increasing strength: MessageFilterAgent for a GraphFlow node, a nested team so the parent sees only a result, and AgentTool/TeamTool so the sub-agent is invoked as a function with its own context entirely. Pick the weakest one that solves the problem, because each step up costs you visibility in the transcript.

Work out what a run cost without a vendor. Sum prompt_tokens and completion_tokens from models_usage over result.messages, multiply by your per-token rates, and log it with the session id, tenant and stop_reason. That one log line answers most cost questions for the next six months, and it is four lines of code. Console(..., output_stats=True) gives you the same figures interactively while you develop.

Decide the caching question deliberately. ChatCompletionCache is an unambiguous win in development and CI, where you re-run the same scenario repeatedly. In production, identical prompts are rarer than you hope, and a cache shared across tenants is a data-exposure question you have to answer before you turn it on. Scope the cache key or the store per tenant, or leave it off.

Run AutoGen Studio without breaking your project. It is version-skewed: 0.4.2.2 requires the autogen-* packages >=0.4.9.2,<0.6, so give it a separate virtual environment. Bind it to 127.0.0.1, because it has no auth or permission model and the docs say it is "not meant to be a production-ready app". autogenstudio ui --host 127.0.0.1 --port 8081 --appdir ./my-app for the GUI; autogenstudio serve --team team.json --port 8084 to expose a team JSON as an endpoint for prototyping.

Keep up with security yourself. No upstream feature work means watching the repository directly, including fixes that land on main and never reach PyPI — the component-loading namespace restriction is the current example. Keep a note of which unreleased fixes matter to you, and what you have done instead.

Deploy the boring way. A stateless container behind your usual ingress, session state in Postgres or Redis, health checks, resource limits, and horizontal replicas. Skip the experimental gRPC distributed runtime unless you have a specific reason: agent paging and load balancing are documented as not implemented, and the official framing expects breaking changes.

Habits that keep a team honest over months

The errors above are one-off lessons. These are the standing habits that stop a working AutoGen service decaying as prompts, models and people change around it.

Keep a defaults file and review it quarterly. The behaviour of your system is dominated by about a dozen numbers: max_tool_iterations, reflect_on_tool_use, the context window sizes, the four termination limits, max_turns, timeout, max_retries, the executor timeout. Put them in one module as named constants rather than scattered through agent constructors. When somebody asks why a conversation cost what it did, you point at one file, and when a model changes generation you tune one file.

Treat prompts as reviewed artefacts. A system_message and a description are production configuration, not strings. Keep them in version control, make changes go through review, and have at least one replay test that fails when routing-relevant wording changes. The most common silent regression in a multi-agent system is somebody improving a description for readability and breaking the selector.

Record the stop reason of every run. TaskResult.stop_reason is a free classification of how your system behaves in aggregate. Counting stop reasons over a week tells you what share of conversations converged on your semantic condition versus hit a cap — and a rising share of cap hits is the earliest signal that quality is drifting, usually well before anyone complains.

Re-verify against the version, not against memory. AutoGen changed meaningfully across 0.4, 0.5, 0.6 and 0.7, and breaking changes landed in minors. When something behaves unexpectedly, check the release notes for the version you have pinned before concluding it is a bug: max_tool_iterations in 0.6.2, approval_func and the Docker default in 0.7.2, nested teams in 0.7.1, the resumable GraphFlow in 0.6.4. Half of the "AutoGen bugs" reported online are a version mismatch with a tutorial.

Write down who owns the framework. With no upstream feature work, somebody on the team has to watch the repository for security fixes, know which ones have landed only on main, and hold the migration decision. If that person is nobody, it is you by default and you will find out during an incident. A short page naming the owner, the pinned versions, the unreleased fixes you are exposed to, and the current migration position is the cheapest operational document you will write.

The mid-level checklist

Before a team goes near production traffic:

• max_tool_iterations set explicitly on every tool-using agent.
• reflect_on_tool_use chosen deliberately, with the token cost acknowledged.
• description on every agent written for the routing prompt.
• Termination: a semantic condition OR'd with message, token and time limits, plus max_turns.
• Model context chosen; not left unbounded for a long-lived session.
• parallel_tool_calls=False wherever handoffs, AgentTool or TeamTool are used.
• model_info complete, including structured_output, for every non-OpenAI model name.
• timeout and max_retries set on the client; close() called in a finally.
• State saved and loaded per session; no team instance shared across requests.
• No UserProxyAgent in a request path.
• Code execution in Docker or ACA dynamic sessions, behind an approval_func, with egress denied by default.
• Component JSON loaded only from a store you control; no dumped config containing api_key in version control.
• All three packages pinned to the same version, with a lock file that holds mcp<2.
• Replay-client tests covering routing, termination and state; tool functions unit-tested.
• OTLP spans exported, plus per-run token totals tagged with session and tenant.
• Someone named as responsible for watching upstream security fixes, and a migration decision recorded.

If every line of that is true, you are running AutoGen about as well as it can be run, and the habits transfer intact to whichever framework you move to next.

This is part three of three. Up to here the question has been how to make AutoGen do something useful. From here the questions change shape: what happens when four product teams build on the same runtime, who is accountable when a generated shell script deletes a volume, what the token bill looks like at the end of a quarter, and what you tell leadership when the framework you standardised on has been in maintenance mode for a year.

The README at github.com/microsoft/autogen states that "AutoGen is now in maintenance mode. It will not receive new features or enhancements and is community managed going forward," and adds that "New users should start with Microsoft Agent Framework." The last PyPI release of the Python packages is 0.7.5, published on 2025-09-30. The repository is not archived and commits continue, but they are bug fixes, security fixes and documentation. Owning AutoGen as a platform in 2026 therefore means owning something stable, widely deployed, and finite. That is a perfectly defensible position — plenty of infrastructure runs on frozen versions — but it has to be a decision you made deliberately, with a migration path written down, rather than a default you drifted into.

Where this picks up

I start with the architecture, because every ownership decision below follows from which of the three layers your teams are allowed to touch.

The three layers you now own

AutoGen is not one library. It is three packages with three different stability profiles, and a platform owner who treats them as one surface will be surprised in all three directions.

The practical consequence is a rule about what teams may import. AgentChat is the layer product teams should live in; it is opinionated, task-driven, and stable within 0.7.x. Core is the layer your platform code lives in, because that is where routing, multi-tenancy and interception are expressible. Extensions is the layer that needs a review gate, because every entry in it either sends your data somewhere or executes something.

There is also a naming problem that will cost you an afternoon if you do not pre-empt it. The PyPI names autogen and ag2 belong to AG2, the community fork at ag2ai/ag2, which has a different v0.2-style API built on ConversableAgent and initiate_chat. The pyautogen package is no longer published by Microsoft — the migration guide states that "We no longer have admin access to the pyautogen PyPI package, and the releases from that package are no longer from Microsoft since version 0.2.34" — and 0.10.0 is now just a proxy that depends on autogen-agentchat. Microsoft AutoGen is imported with underscores: autogen_agentchat, autogen_core, autogen_ext. Any import autogen in a pull request is either AG2 or v0.2 code, and either way it is a review finding.

# The only supported install line for this platform, pinned
pip install "autogen-agentchat==0.7.5" "autogen-ext[openai]==0.7.5"

# Prove which package is actually resolved before debugging anything
python -c "import autogen_agentchat, autogen_core; print(autogen_agentchat.__version__, autogen_core.__version__)"
# -> 0.7.5 0.7.5
pip show autogen-agentchat autogen-core autogen-ext

Which component breaks first

Under load, four things queue up, and they queue in a predictable order. Knowing the order means you stop guessing during an incident.

The first and most common bottleneck is the model API itself. AutoGen adds orchestration on top of chat completions; it does not add throughput. Every turn of a team is at least one request, and some topologies are more than one. SelectorGroupChat spends an extra model call per turn deciding who speaks next, and reflect_on_tool_use=True spends another one after tools return. A five-agent selector team with reflection enabled can be running three model calls per visible message. Your rate limit, not your CPU, is the ceiling.

The second is the runtime. SingleThreadedAgentRuntime processes messages on one asyncio event loop, and the Core documentation is explicit that it is not built for high throughput. For a web service this is usually fine, because each request gets its own short-lived runtime underneath a team, but it means you cannot treat one process as a shared agent bus for a whole organisation. If a tool blocks the loop — a synchronous HTTP call, a CPU-bound parse, a requests call someone forgot to make async — every other session in that process stalls behind it. Tool functions passed to tools=[...] may be sync or async, and the review rule is simple: anything doing I/O is async def, anything CPU-bound goes to a thread or a queue.

The third is the human. UserProxyAgent blocks a running team while it waits for input, and a team cannot be saved while it waits. One pending approval therefore holds a worker, a runtime and a session's memory hostage for as long as the person takes to answer. In anything web-facing this is the wrong shape entirely. End the run instead, using HandoffTermination(target="user") or a TextMentionTermination, persist the state, return to the caller, and resume with a fresh run() when the reply arrives.

The fourth is the context window, and it degrades rather than failing. Teams share a single message thread, so a long-running team accumulates every message from every participant, and each model call carries more of it. Cost and latency rise together while answer quality falls, which makes it the least obvious of the four. The defence is to set a model context explicitly rather than accepting UnboundedChatCompletionContext.

from autogen_agentchat.agents import AssistantAgent
from autogen_core.model_context import TokenLimitedChatCompletionContext
from autogen_ext.models.openai import OpenAIChatCompletionClient


def build_agent(name: str, system_message: str) -> AssistantAgent:
    """Platform factory: the defaults a team cannot forget to set."""
    client = OpenAIChatCompletionClient(
        model="gpt-4.1",
        timeout=60,
        max_retries=3,
        parallel_tool_calls=False,   # required if the agent ever gets handoffs or AgentTool
    )
    return AssistantAgent(
        name,                        # must be a valid Python identifier
        model_client=client,
        system_message=system_message,
        model_context=TokenLimitedChatCompletionContext(client, token_limit=12000),
        max_tool_iterations=4,
    )

State, sessions, and the only scaling story that works

The single most consequential architectural fact about AutoGen at this level is that agents are stateful objects in a process, and the scaling unit is therefore the session, not the agent.

Three errors in the 0.7.5 source tell you the whole story. ValueError: The team is already running, it cannot run again until it is stopped. appears when two requests touch the same team instance, which is exactly what happens the first time someone caches a team in a module-level global and puts two Gunicorn workers behind it. RuntimeError: The team cannot be loaded while it is running. and RuntimeError: The group chat is currently running. It must be stopped before it can be reset. close the other two doors. A team object is a single-occupancy resource for the duration of a run.

The pattern that scales is therefore stateless workers plus externalised state. Each request loads the session's state JSON from a store, rebuilds the team, runs one turn, saves the state back, and discards the object. The team becomes a pure function of its state, and the number of workers becomes a capacity decision rather than a correctness one.

import json
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination


async def run_turn(session_id: str, user_text: str, store) -> str:
    team = RoundRobinGroupChat(
        [build_agent("planner", "Plan, then hand to the writer."),
         build_agent("writer", "Write the answer. Say APPROVED when done.")],
        termination_condition=TextMentionTermination("APPROVED") | MaxMessageTermination(12),
    )

    saved = await store.get(session_id)          # Postgres, Redis, Cosmos DB: a JSON blob
    if saved:
        await team.load_state(json.loads(saved))

    result = await team.run(task=user_text)      # one turn, then control returns to you
    await store.put(session_id, json.dumps(await team.save_state()))
    return result.messages[-1].to_text()

Two details decide whether this holds up. First, save_state() returns a JSON-serialisable dictionary whose shape is tied to the component graph that produced it, so a state blob written by one topology will not reliably load into a different one. Treat the team definition as part of the session's schema: store a version alongside the state, and when you change participants or their order, start a new session rather than migrating the blob. Second, concurrency control is yours. Two tabs on the same session will both load the same state, both run, and the second write will silently win. A per-session lock in Redis, or an optimistic version check on the row, is the fix, and it belongs in the platform wrapper so that no product team has to think about it.

Serialisation has a second use at platform scale: every component can dump_component() to JSON and be rebuilt with load_component(), which is how Studio works. Two limits apply. Callables such as selector_func, GraphFlow lambda conditions and FunctionTool do not serialise, so a declarative topology is a less capable one; and dump_component() writes an explicitly-passed api_key into the JSON.

The trust model

Write down the trust boundaries before you write policy, because AutoGen's defaults assume a trusted developer on a laptop and your platform does not have one.

Model output is untrusted input. This is the sentence the whole security section hangs on. Anything the model emits — a tool call, a handoff target, a shell block, a URL for the web surfer, a file path — is derived from content that may include text an attacker controls: a retrieved document, a web page, an email body, a ticket comment. AutoGen does not mitigate prompt injection, and nothing in the framework claims to. The mitigations are structural: narrow tools, allowlists, approval functions, intervention handlers, and bounded authority per agent.

Tools execute with your process's privileges. A FunctionTool wrapping a database helper has whatever grants the service account has. An MCP server launched through StdioServerParams is a subprocess with your user's rights, which is why the README says plainly to "Only connect to trusted MCP servers," and why MCP credentials belong in the server's env, never in a prompt where the model can echo them into a transcript.

Handoffs are a routing decision made by a model. Swarm sets the next speaker from the most recent HandoffMessage, so an agent that can hand off to a privileged agent has effectively delegated authorisation to token prediction. Keep the privileged agent's own tools narrow rather than trusting the route into it, and note the related trap: parallel tool calls can produce several handoffs, at which point you get UserWarning: Multiple handoffs detected. Only the first is executed. Set parallel_tool_calls=False on the client of any agent with handoffs, or with an AgentTool or TeamTool, which the 0.7.2 notes document as unsupported in parallel.

Configuration is code, as the load_component() callout above establishes.

And finally, AutoGen Studio is a prototype. The documentation says it is "not meant to be a production-ready app" and describes it as a research prototype not meant for production environments. It has no production authentication or permission model. It defaults to 127.0.0.1 on port 8081, and that default is the policy: bind it to localhost, reach it over an SSH tunnel or an authenticating reverse proxy, and never put it on a shared network with a --host 0.0.0.0.

# Studio: separate venv, localhost only, its own app directory and database
python3 -m venv .venv-studio && source .venv-studio/bin/activate
pip install -U autogenstudio
autogenstudio ui --host 127.0.0.1 --port 8081 --appdir ./studio-app --database-uri "sqlite:///database.sqlite"

Code execution as a platform service

If your teams let models write and run code, that capability should be one service you operate rather than a configuration each team rediscovers.

Since 0.7.5 the defaults are on your side. DockerCommandLineCodeExecutor is the default wherever a default executor is created, create_default_code_executor() prefers Docker and falls back to Local with a warning, and MagenticOneGroupChat and the MagenticOne helper default to Docker as of 0.7.2. The Local executor always emits UserWarning: Using LocalCommandLineCodeExecutor may execute code on the local machine which can be unsafe. Treat that warning as a build failure in CI rather than as noise; it is the most reliable single signal that a repository has drifted off policy.

Approval is the second half. CodeExecutorAgent gained approval_func in 0.7.2, and without it you get UserWarning: No approval function set for CodeExecutorAgent. This means code will be executed automatically without human oversight. A platform-grade approval function is not a yes-man: it is a policy point that can log the code, hash it, check it against rules, route genuinely novel code to a human, and auto-approve the narrow shapes you have decided are safe.

import re
from autogen_agentchat.agents import CodeExecutorAgent, ApprovalRequest, ApprovalResponse
from autogen_ext.code_executors.docker import DockerCommandLineCodeExecutor

BLOCKED = re.compile(r"\b(rm\s+-rf|curl|wget|subprocess|os\.system|socket)\b")


def approve(request: ApprovalRequest) -> ApprovalResponse:
    code = "\n".join(m.to_text() for m in request.messages)
    if BLOCKED.search(code):
        return ApprovalResponse(approved=False, reason="blocked pattern; needs human review")
    return ApprovalResponse(approved=True, reason="matched allowlisted shapes")


async def run_generated_code(task: str, client) -> str:
    async with DockerCommandLineCodeExecutor(
        work_dir="sandbox",          # never the current directory: that is deprecated
        image="python:3-slim",
        timeout=60,
    ) as executor:
        agent = CodeExecutorAgent(
            "executor",
            code_executor=executor,
            model_client=client,     # required because max_retries_on_error > 0
            approval_func=approve,
            max_retries_on_error=2,
            supported_languages=["python"],
        )
        result = await agent.run(task=task)
        return result.messages[-1].to_text()

Three operational facts round this out. The executor is a context-managed resource: used before starting, it raises ValueError: Container is not running. Must first be started with either start or a context manager. A missing daemon gives RuntimeError: Failed to connect to Docker. Please ensure Docker is installed and running., which on a Kubernetes-hosted worker usually means somebody assumed a Docker socket they do not have. And work_dir pointing at the current directory emits DeprecationWarning: Using the current directory as work_dir is deprecated. — a warning worth heeding for a better reason than deprecation, since the current directory is often your source tree.

Where you cannot give workers a Docker socket, ACADynamicSessionsCodeExecutor moves execution into Azure Container Apps dynamic sessions, which is also the cleanest answer when data-residency rules require that generated code and the data it touches stay inside a specific region. For teams in the Gulf or Egypt serving regulated customers, that regional pinning is usually the deciding factor rather than a performance one: the sandbox has to sit in the same jurisdiction as the data it is allowed to read.

Secrets and the supply chain

Credentials in AutoGen flow through three paths, and only one of them is good. The clients read OPENAI_API_KEY and ANTHROPIC_API_KEY from the environment when no key is passed, which is the path you want: injected by your orchestrator from a secret manager, never present in code or configuration. The second path is an explicit api_key=, which works but then appears in any dump_component() output. The third is a key pasted into a system prompt or a tool description so the model can "use" it, which you prevent by making it impossible rather than by asking nicely — the model will reproduce it in a transcript, and the transcript will reach a log, a trace and possibly a support ticket.

On Azure, prefer no static key at all. AzureOpenAIChatCompletionClient and AzureAIChatCompletionClient accept azure_ad_token_provider=, and the documented pattern pairs DefaultAzureCredential with get_bearer_token_provider from azure.identity, installed via autogen-ext[azure]. Workload identity in AKS then gives you keyless auth with rotation handled outside your application entirely, which also removes the most common cause of a 401 at three in the morning.

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)

client = AzureOpenAIChatCompletionClient(
    azure_deployment="gpt-4o",
    model="gpt-4o",
    api_version="2024-06-01",
    azure_endpoint="https://my-resource.openai.azure.com/",
    azure_ad_token_provider=token_provider,   # no key in the process at all
)

The supply chain has its own shape here. You depend on three pinned AutoGen packages, a set of extras chosen per service, a model SDK, and — if you use MCP — an arbitrary number of MCP servers that are often npm packages fetched at launch by npx. That last one is the weakest link: StdioServerParams(command="npx", args=["@playwright/mcp@latest", ...]) resolves latest at process start, which means your production behaviour changes when someone else publishes. Pin MCP server versions exactly, vendor the ones that matter, and mirror them internally. For the Python side, build images from a lock file, scan them, and keep a reproducible record of what resolved, because maintenance mode guarantees that an unpinned transitive dependency will eventually move out from under you. The Trivy guide covers the scanning half of that, and Vault the credential-injection half.

Multi-tenancy

There are two correct answers, and they belong to the two layers.

At the AgentChat layer, a tenant is a session, and the rules are the ones from the state section: one team instance per in-flight request, state keyed by tenant and session, never a shared team object. Everything else follows from that — per-tenant budgets live in the termination conditions you attach when you build the team, per-tenant tool allowlists live in which tools the factory passes, and per-tenant isolation of data lives in the credential the tools use. It is a boring design, which is the point.

At the Core layer, multi-tenancy is expressed in the identity model, and this is the part worth understanding properly because it is AutoGen's most distinctive idea. An AgentId is a pair of agent type and agent key, written type/key. The type is bound to a factory rather than to a class, and the runtime instantiates lazily on first message. A TopicId is a pair of topic type and topic source. TypeSubscription(topic_type, agent_type) maps the topic source onto the agent key, so publishing to TopicId(type="tasks", source="tenant-42") delivers to AgentId("worker", "tenant-42") — and that agent instance is created on demand, holds only that tenant's state, and is addressable for the life of the runtime. One subscription declaration gives you per-tenant instance isolation without any tenant-aware code inside the agent.

from dataclasses import dataclass
from autogen_core import (
    AgentId, MessageContext, RoutedAgent, SingleThreadedAgentRuntime,
    TopicId, TypeSubscription, message_handler,
)


@dataclass
class Job:
    content: str


class Worker(RoutedAgent):
    def __init__(self) -> None:
        super().__init__("Per-tenant worker")
        self.seen = 0

    @message_handler
    async def handle(self, message: Job, ctx: MessageContext) -> str:
        self.seen += 1          # state belongs to this tenant's instance only
        return f"{self.id.key} handled {self.seen}: {message.content}"


async def main() -> None:
    runtime = SingleThreadedAgentRuntime()
    await Worker.register(runtime, "worker", lambda: Worker())
    await runtime.add_subscription(TypeSubscription(topic_type="jobs", agent_type="worker"))
    runtime.start()

    # Broadcast: source becomes the agent key, so each tenant gets its own instance
    await runtime.publish_message(Job("index documents"), TopicId(type="jobs", source="tenant-42"))
    # Direct RPC to one tenant's instance
    reply = await runtime.send_message(Job("status"), AgentId("worker", "tenant-42"))
    print(reply)

    await runtime.stop_when_idle()
    await runtime.close()

Two cautions. Core gives you isolation of state, not isolation of failure or of resources: all those instances share one event loop in one process, so a tenant that floods you still affects the others, and quotas have to be enforced before messages are published. And the distributed runtime is not the escape hatch it looks like. GrpcWorkerAgentRuntimeHost and GrpcWorkerAgentRuntime are documented as experimental with breaking changes expected, and the Core documentation states that agent paging in and out and load balancing are not implemented yet. For real horizontal scale, run stateless API workers that load state per request. That is less elegant than a distributed actor mesh and considerably more likely to still be running next quarter.

Cost, and the calls nobody counted

Every message carries models_usage, a RequestUsage with prompt_tokens and completion_tokens, and every client exposes total_usage() and actual_usage(). That is enough to build real attribution: after each run, walk result.messages, sum usage by source, and write a row tagged with tenant, session, topology and model. Do it in the platform wrapper and every team gets cost reporting without writing any.

The interesting part is the calls that do not appear as messages. Speaker selection in SelectorGroupChat is one model call per turn and shows up only as a SelectSpeakerEvent. Reflection after tools, when reflect_on_tool_use resolves to True, is another. Note that its default is None, which resolves to True if output_content_type is set and False otherwise — so enabling structured output quietly enables reflection too, and a team's bill can rise after a change nobody associated with cost. The inner tool loop multiplies calls by up to max_tool_iterations. And MagenticOneGroupChat runs an orchestrator with a task and progress ledger that re-plans on stalls, defaulting to max_turns=20 and max_stalls=3, which is the most expensive preset in the library by a wide margin.

Four controls, in order of leverage. Cap the conversation: max_turns on the team plus MaxMessageTermination, remembering that it counts the task message — with MaxMessageTermination(3) the stop reason reads "Maximum number of messages 3 reached, current message count: 3" after the task and two replies. Cap the tokens: TokenUsageTermination(max_total_token=...) is the only condition that stops on spend directly. Cap the context: BufferedChatCompletionContext or TokenLimitedChatCompletionContext so that prompt size stops growing with conversation length. And cache: model caching is off by default since v0.4, and ChatCompletionCache wrapping a client with a Redis or diskcache store pays for itself immediately in development and in any workload with repeated prefixes.

import json
from collections import defaultdict
from autogen_agentchat.conditions import (
    MaxMessageTermination, TimeoutTermination, TokenUsageTermination,
)
from autogen_ext.cache_store.diskcache import DiskCacheStore
from autogen_ext.models.cache import ChatCompletionCache, CHAT_CACHE_VALUE_TYPE
from diskcache import Cache

# Three independent stops, OR-ed: semantics, spend, and wall clock
budget = (
    TokenUsageTermination(max_total_token=40_000)
    | MaxMessageTermination(24)
    | TimeoutTermination(timeout_seconds=180)
)

cached_client = ChatCompletionCache(
    client, DiskCacheStore[CHAT_CACHE_VALUE_TYPE](Cache("/var/cache/autogen"))
)


def usage_rows(result, tenant: str, session: str) -> list[dict]:
    totals: dict[str, list[int]] = defaultdict(lambda: [0, 0])
    for message in result.messages:
        usage = getattr(message, "models_usage", None)
        if usage is not None:
            totals[message.source][0] += usage.prompt_tokens
            totals[message.source][1] += usage.completion_tokens
    return [
        {"tenant": tenant, "session": session, "agent": agent,
         "prompt_tokens": prompt, "completion_tokens": completion,
         "stop_reason": result.stop_reason}
        for agent, (prompt, completion) in totals.items()
    ]

Observability and the incident playbooks

AutoGen gives you two independent streams, and a platform needs both. Python logging exposes TRACE_LOGGER_NAME for human-readable debugging and EVENT_LOGGER_NAME for structured events, with child loggers encouraged in the form f"{TRACE_LOGGER_NAME}.my_module". OpenTelemetry instrumentation covers the runtime and emits GenAI semantic-convention spans named create_agent, invoke_agent and execute_tool, added in 0.6.2. You pass a tracer_provider= to the runtime or set the global provider; AUTOGEN_DISABLE_RUNTIME_TRACING=true or a NoOpTracerProvider turns runtime spans off when they are too noisy. Any OTLP backend works, which in practice means Langfuse or an OpenTelemetry collector feeding whatever you already run.

Four dashboards cover almost every question you will be asked. A stop-reason breakdown per topology, because the ratio of semantic stops to budget stops is your quality signal. Tokens per session split by agent, which is cost and also the fastest way to spot a runaway participant. Tool error rate by tool name from ToolCallExecutionEvent, since a single flaky tool explains most "the agent got dumber" reports. And turns per completed task, which rises before anything else breaks when a prompt change degrades routing.

Then the playbooks. The team never stops. Check stop_reason first: if it is None, the run is still going, and that means no termination condition and no max_turns, or a condition whose text the agents never emit. Stop it with an ExternalTermination you can .set() from outside, which halts gracefully after the current agent's turn, and keep CancellationToken.cancel() for genuine emergencies, because it aborts immediately and can leave team state inconsistent.

Costs doubled overnight with no code change. Look for a model swap, a prompt that grew, or a newly set output_content_type turning reflection on. Then check whether context limits are in place, because an unbounded context makes every day more expensive than the last.

Agents stopped calling tools. Read the exact error. ValueError: The model does not support function calling. means the model or its model_info says so, and for a local or OpenAI-compatible model the usual cause is a hand-written model_info with function_calling: False. On a custom model name with no model_info at all you get ValueError: model_info is required when model name is not a valid OpenAI model, and an incomplete dict gives UserWarning: Missing required field 'structured_output' in ModelInfo.

Handoffs silently vanish. That is the parallel-tool-calls warning; set parallel_tool_calls=False.

Concurrency errors under load. The team is already running means a shared team object, and the fix is the session-per-request pattern rather than a lock around the symptom.

MCP tools disappeared after a deploy. Either RuntimeError: McpWorkbench is not started. Call start() first. from using the workbench outside its async with, or the mcp 2.x import error from an unpinned dependency.

import logging
from autogen_core import EVENT_LOGGER_NAME, TRACE_LOGGER_NAME

logging.getLogger(TRACE_LOGGER_NAME).setLevel(logging.WARNING)   # chatty; keep it off by default

events = logging.getLogger(EVENT_LOGGER_NAME)                    # structured, ship these
events.setLevel(logging.INFO)
events.addHandler(logging.StreamHandler())

Upgrades and the migration you should already be planning

Upgrading AutoGen has two halves: staying safe on 0.7.x, and knowing where you go next.

Within 0.7.x the mechanics are simple and strict. Pin all three packages to the same exact version, because autogen-agentchat pins autogen-core== the same version and autogen-ext must match. Read the GitHub release notes for every minor, because breaking changes landed in minors across 0.4, 0.5, 0.6 and 0.7 — AzureAISearchTool renamed create_keyword_search() to create_full_text_search() in 0.5.7, GraphFlow lost its internal stop agent in 0.6.4 and moved the stop reason to TaskResult.stop_reason, and the default executor changed in 0.7.5. Keep Studio in its own environment, since 0.4.2.2 requires autogen-core, autogen-agentchat and autogen-ext below 0.6. And treat every DeprecationWarning as a ticket: model_capabilities= in favour of model_info, client.capabilities in favour of client.model_info, response_format= holding a Pydantic model in create args in favour of json_output= or AssistantAgent(output_content_type=...), DiGraphBuilder.add_conditional_edges in favour of add_edge(..., condition=...), and max_consecutive_empty_chunk_tolerance.

There is also a dependency you do not control. The OpenAI Assistants API was shut down on 2026-08-26, which means OpenAIAssistantAgent from autogen_ext.agents.openai no longer works against OpenAI regardless of what version you pin. Use OpenAIAgent, which targets the Responses API, or plain AssistantAgent. If any service in your estate still imports the assistant agent, that is not a deprecation, it is an outage waiting for the next deploy.

The other half is the destination. Microsoft Agent Framework was built by the AutoGen and Semantic Kernel teams, is at 1.x, and has an official AutoGen migration guide on Microsoft Learn. The mapping is close enough that a well-structured AutoGen codebase ports without redesign.

Two differences in that guide deserve emphasis because they change behaviour rather than names. AssistantAgent "is single-turn unless you increase max_tool_iterations", whereas the framework's Agent is multi-turn by default — so a naive port makes your agents more agentic and more expensive at the same time. And AutoGen has "embedded and experimental distributed runtimes" while the framework "focuses on single-process composition today; distributed execution is planned," which means a port is not a scaling upgrade. Also check the matrix before committing: the Anthropic and Ollama clients and ChatCompletionCache are listed as planned rather than available, so a team depending on them stays where it is for now.

Where AutoGen stops

Some problems are a poor fit, and saying so early is a large part of the value you add as the platform owner.

When a single agent is enough. Most production LLM features are one model, a few tools and a loop. A team adds coordination cost, extra model calls and far more ways to fail. Multi-agent earns its place when roles genuinely differ in tools, context or authority, not because the architecture diagram looks better with four boxes.

When you need a durable, resumable workflow engine. AutoGen's state is a JSON blob you persist between turns; it has no scheduler, no retries across process restarts, no backfills and no visibility into a half-finished graph. For long-running orchestration with those guarantees, you want Prefect, Dagster or Airflow driving the pipeline and AutoGen running inside a single step.

When the control flow is a graph you want to reason about rigorously. GraphFlow with DiGraphBuilder is real and supports conditional edges, fan-out and fan-in, loops and activation conditions, but it is documented as experimental, its callable edge conditions are not serialisable, and it accepts only List[ChatAgent] rather than nested teams. If the graph is the product, LangGraph is built around that premise.

When you need horizontal scale inside the framework. The gRPC distributed runtime is experimental, with paging and load balancing unimplemented. Scale with stateless workers, or pick a framework whose distribution story is finished.

When the team wants a different ergonomic. CrewAI is role-and-task shaped and gets simple collaborative flows running faster. Semantic Kernel has the deeper .NET and enterprise-plugin story, and AutoGen can borrow its model providers through SKChatCompletionAdapter. The OpenAI Agents SDK is thinner and closer to one vendor's platform.

When you are starting something new. This is the honest one. Microsoft's own guidance is that new users should start with Microsoft Agent Framework. If a team comes to you with a greenfield project and no AutoGen code, the default recommendation is the framework, and AutoGen is the choice only when something concrete argues for it: existing code, an in-house extension, a pattern the framework has not shipped, or a client the framework lists as planned.

Running it as a platform for other teams

Everything above turns into five concrete deliverables.

An SDK, not a tutorial. Ship an internal package that exposes build_agent, build_team, a tool registry and a session store, with your defaults baked in: timeouts, retries, parallel_tool_calls=False, a bounded model context, a budget termination, tracing wired up and usage rows emitted. Teams import it; they do not configure AutoGen. The payoff is that a policy change is a version bump rather than a migration across six repositories.

A tool catalogue with an owner per entry. Each tool has a name, a narrow scope, a credential, a quota, a rate limit and a named owner. New tools get reviewed for authority, not just correctness: the question is what the worst plausible call does, not whether the happy path works. MCP servers are entries in the same catalogue, pinned to exact versions, and nothing outside the catalogue reaches production.

Execution as a service. One Docker-backed executor configuration, one approval policy, one audit log of what was executed and who or what approved it. Teams get a function call, not a code executor to configure.

A paved road for state. Session storage, concurrency control, retention and deletion. Retention is where regulation lands: transcripts contain whatever users typed, so if you serve customers in Saudi Arabia, the UAE or Egypt, the storage region and retention period for session state and traces are compliance decisions, and a platform that answers them once saves every team from answering them differently. Sandboxed execution regions belong in the same conversation, which is where ACADynamicSessionsCodeExecutor becomes attractive.

A compatibility promise. State the supported version, the upgrade cadence, the deprecation policy and the migration stance in writing. For a maintenance-mode framework this is not bureaucracy, it is the thing that stops each team inventing its own answer to "should we still be using this?"

Two CI gates will catch most drift on their own. Fail the build on import autogen or on any pin that is not the exact supported version, and fail it on LocalCommandLineCodeExecutor outside an explicitly allowlisted test path. Add a nightly job that resolves your dependency tree from scratch and diffs it against the lock file, because that is how you find out about mcp 2.x before production does. The GitHub Actions guide covers the mechanics.

The review checklist

Use this on pull requests and design documents. Most findings cluster in the first three groups.

Versions and imports. All three packages pinned to the same exact version. No import autogen, no ConversableAgent, no initiate_chat, no llm_config or config_list. mcp pinned below 2 where the MCP extra is used. A lock file committed, and Studio in its own environment if present. No OpenAIAssistantAgent.

Lifecycle. Every model client closed with await client.close(). Every executor and McpWorkbench inside async with or explicitly started. A Core runtime started once and closed once, with the hook not running twice under a reloader. One team instance per in-flight run, never a module-level global shared across requests.

Budgets. Every team has a termination condition and a max_turns. At least one hard stop that does not depend on model behaviour: MaxMessageTermination, TimeoutTermination or TokenUsageTermination. An explicit model context rather than the unbounded default. max_tool_iterations set deliberately. Usage rows emitted with tenant and session tags.

Safety. No LocalCommandLineCodeExecutor outside sanctioned tests. approval_func present on every CodeExecutorAgent, and a policy rather than an unconditional yes. work_dir not the current directory. Tools narrow, with their own credentials. MCP servers on the allowlist, version-pinned, secrets in env. parallel_tool_calls=False wherever handoffs, AgentTool or TeamTool appear. No load_component() on anything a user could influence. No api_key= passed explicitly where an environment variable would do, and no committed dump_component() output.

Operability. Stop reasons recorded on every run. Event-logger output shipped and indexed. Traces exported, with the sampling decision made on purpose. A kill switch that is an ExternalTermination, with cancellation reserved for emergencies. A documented degraded-mode response for executor and model outages.

Design documents, additionally. Why multiple agents rather than one, in terms of differing tools, context or authority. Why this topology, with the per-turn cost of SelectorGroupChat acknowledged if chosen. Where state lives, how it is keyed, how long it is kept and in which region. What the worst plausible tool call does. And the migration stance: is this team staying on 0.7.5, and what would change that.

The complete picture

A production AutoGen platform, assembled from the pieces above, has a recognisable shape. Stateless API workers behind a load balancer, each one rebuilding a team per request from a factory in your internal SDK, loading session state from Postgres or Redis under a per-session lock, running one bounded turn, saving state, and emitting usage rows and OTLP spans on the way out. Model access goes through pinned clients with keyless auth where the provider supports it, optionally fronted by a cache or a gateway such as LiteLLM when you need provider fallback. Generated code goes to a Docker or Azure Container Apps sandbox behind an approval policy with an audit trail. Tools come from a catalogue, each with its own credential and quota. Traces land in Langfuse or your own collector; metrics land in Prometheus and get read in Grafana. The whole thing is containerised with Docker, scheduled on Kubernetes, and its dependency tree is re-resolved nightly and diffed against a lock file.

None of that is exotic, and that is the point. The hard parts of owning AutoGen are not AutoGen-specific: they are session state, budgets, trust boundaries and dependency hygiene. What is specific is the small set of facts that bite: agents are stateful and teams are single-occupancy; the defaults are tuned for a laptop; load_component() imports what it is told; the selector and the reflection step cost money invisibly; and the framework is finished, so your pins are load-bearing.

Where the series leaves you

Across three levels you have gone from a single AssistantAgent printing a greeting, through teams, tools, memory, MCP, structured output and state, to the Core runtime, multi-tenancy, sandboxed execution, cost attribution and a migration plan. You can now design a multi-agent system and, more valuably, argue against building one when a single agent would do. You can read the 0.7.5 error messages as a map of the framework's assumptions. You can say what your platform supports and what it does not, and you can tell leadership exactly where AutoGen sits: a stable, capable, maintenance-mode framework worth running deliberately and worth leaving deliberately.

The natural next steps depend on which part interested you most. If it was the orchestration, read the LangGraph guide for a graph-first treatment of the same problems, and the Semantic Kernel guide for the other half of the Agent Framework lineage. If it was the operational side, Langfuse for tracing and RAGAS for evaluating whether any of it actually got better. And if it was the trust model, the MCP guide goes considerably deeper into the protocol whose servers you have just taken responsibility for.

Sources

• AutoGen repository and README, including the maintenance-mode notice
• AutoGen release notes
• python-v0.7.5 release
• PR #7463: component-provider namespace restriction (unreleased)
• AutoGen stable documentation
• Core concepts: architecture
• Core concepts: agent identity and lifecycle
• Core concepts: topic and subscription
• Core: distributed agent runtime
• Core: component config
• Core: telemetry
• Core: logging
• Core: command line code executors
• Core: workbench
• Cookbook: Azure OpenAI with AAD auth
• AgentChat: state
• AgentChat: termination
• AgentChat: human in the loop
• AgentChat: SelectorGroupChat
• AgentChat: Swarm
• AgentChat: GraphFlow
• AgentChat: Magentic-One
• AgentChat: serialize components
• AgentChat: tracing
• AgentChat: migration guide from v0.2
• AutoGen Studio documentation
• Extensions installation and extras
• Microsoft Agent Framework: migrating from AutoGen
• Microsoft Agent Framework repository
• OpenAI API deprecations, including the Assistants API sunset

Where this picks up

The mid-level round asked you to debug a slow team and justify a topology. A senior round asks three different things: design a system around AutoGen and defend the boundaries, respond to an incident with a method rather than a guess, and make a recommendation to people who do not care which framework won. This file is built around those three, and it assumes you already know the API.

One fact frames every answer below, and omitting it is the fastest way to look out of date. AutoGen is in maintenance mode: the README says it "will not receive new features or enhancements and is community managed going forward," and directs new users to Microsoft Agent Framework. The last PyPI release of the Python packages is 0.7.5 from 2025-09-30. A senior candidate brings that up unprompted, with a position attached.

The thirty-second frame

"AutoGen is Microsoft's multi-agent framework, in three layers: autogen-core is an event-driven actor runtime with its own identity and subscription model, autogen-agentchat is the high-level task API with preset agents and teams, and autogen-ext holds everything that touches the outside world — model clients, code executors, tools, memory. It is stable at 0.7.5 and in maintenance mode, with Microsoft Agent Framework as the official successor and a published migration guide. I would run it deliberately on a pinned version with an exit plan, and I would not start a greenfield project on it without a specific reason."

System design: a multi-agent service for a thousand users

The prompt is usually some version of "design a document-analysis assistant where a planner, a researcher and a writer collaborate, for a thousand concurrent users."

Start with the constraint that decides the whole design: agents are stateful objects and a team is a single-occupancy resource. The 0.7.5 source will tell you so with ValueError: The team is already running, it cannot run again until it is stopped. if two requests share one team instance, and it refuses both load_state and reset mid-run. So the architecture is stateless workers plus externalised state: each request loads a session's save_state() JSON from Postgres or Redis, rebuilds the team from a factory, runs one bounded turn, saves the state back and discards the object. Scale is then a worker-count decision rather than a correctness one.

Say what you deliberately do not use. The gRPC distributed runtime, GrpcWorkerAgentRuntimeHost with GrpcWorkerAgentRuntime, is documented as experimental with agent paging and load balancing not implemented, so it is not the scaling answer however much it looks like one. And SingleThreadedAgentRuntime runs on a single asyncio loop, so one blocking tool stalls every session in that process — which is why every I/O-bound tool is async def and anything CPU-bound leaves the loop.

Then the per-request shape. A factory that sets timeouts, max_retries, parallel_tool_calls=False, a bounded model context (TokenLimitedChatCompletionContext or BufferedChatCompletionContext, never the unbounded default), max_tool_iterations set deliberately rather than left at its default of 1, and a termination condition that ORs a semantic stop with hard stops: TextMentionTermination(...) | MaxMessageTermination(n) | TimeoutTermination(s) | TokenUsageTermination(max_total_token=...). Concurrency control per session belongs in the platform wrapper, because two browser tabs on one session will otherwise both load the same state and the second write wins silently.

Close on topology, and prefer the cheap one. RoundRobinGroupChat costs nothing to route. SelectorGroupChat spends an extra model call per turn choosing a speaker, which is invisible in the transcript and very visible in the bill; a selector_func that returns a fixed name for deterministic hops and None to fall back to the LLM removes most of that cost. Swarm routes on the latest HandoffMessage. GraphFlow with DiGraphBuilder is the only one that expresses a real graph, and it is still documented as experimental.

What a strong answer adds: the state-schema problem. A state blob is tied to the component graph that produced it, so changing participants or their order means starting a new session rather than migrating the blob. Store a topology version next to the state.

System design: multi-tenancy

"Four internal teams want to use your agent platform. How do you isolate them?"

Answer at both layers, because the interviewer is usually testing whether you know Core exists.

At the AgentChat layer a tenant is a session: one team per in-flight request, state keyed by tenant and session, per-tenant budgets expressed in the termination conditions the factory attaches, per-tenant tool access expressed in which tools it passes, and per-tenant data isolation expressed in the credential each tool uses.

At the Core layer, isolation is in the identity model. An AgentId is (type, key) where the type binds to a factory rather than a class and instances are created lazily on first message. A TopicId is (type, source). TypeSubscription(topic_type, agent_type) maps the topic source to the agent key, so publishing to TopicId(type="jobs", source="tenant-42") lands on AgentId("worker", "tenant-42") — per-tenant instance isolation from one subscription declaration, with no tenant-aware code inside the agent.

The caveat that marks seniority: Core isolates state, not resources or failure. All those instances share one event loop in one process, so a noisy tenant still affects the others and quotas must be enforced before messages are published.

Incident response

Interviewers want a method, so narrate the first thing you look at and why.

"A run has been going for twenty minutes and will not stop." Check TaskResult.stop_reason first. If it is None, the run never had a stop: no termination condition and no max_turns, or a condition keyed on text the agents never emit. Stop it with an ExternalTermination you can .set() from outside, which halts gracefully after the current agent's turn. Keep CancellationToken.cancel() for emergencies, because it aborts immediately and can leave team state inconsistent — a distinction worth naming, since "my kill switch corrupts the session" is a real failure mode.

"Token spend doubled overnight and nobody deployed code." Three suspects in order. A model change. A prompt or retrieved context that grew, which an unbounded model context turns into a compounding cost. Or something enabling an extra call: reflect_on_tool_use defaults to None, which resolves to True when output_content_type is set, so turning on structured output silently turns on a post-tool reflection call. Attribution comes from models_usage on every message plus total_usage() on the client, summed by source.

"The agents stopped calling tools." Read the exact error. ValueError: The model does not support function calling. means the model or a hand-written model_info says so. A custom or local model name with no model_info gives ValueError: model_info is required when model name is not a valid OpenAI model, and an incomplete dict gives UserWarning: Missing required field 'structured_output' in ModelInfo.

"Handoffs are being dropped." UserWarning: Multiple handoffs detected. Only the first is executed. Parallel tool calls produced several; set parallel_tool_calls=False. The same applies wherever AgentTool or TeamTool is used, which the 0.7.2 notes document as unsupported in parallel.

"MCP tools vanished after a deploy." Either RuntimeError: McpWorkbench is not started. Call start() first. from using the workbench outside its async with, or ImportError: cannot import name 'RequestContext' from 'mcp.shared.context' because autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound and a fresh install resolved 2.x. Pin mcp pinned below 2 and commit a lock file.

"Errors under load that never appear in staging." The team is already running is a shared team object; the fix is the session-per-request pattern, not a lock around the symptom.

System design: cost and observability from day one

"How would you know what this platform costs, and how would you debug it?"

Treat the two as one answer, because both come from the same instrumentation pass and interviewers like seeing them joined.

Cost first. Every message carries models_usage, a RequestUsage with prompt_tokens and completion_tokens, and every client exposes total_usage() and actual_usage(). After each run you walk result.messages, sum usage by source, and write a row tagged with tenant, session, topology, model and stop_reason. Doing that in the platform wrapper rather than in each product repository means every team gets attribution without writing any, and it means a cost question has a single answer rather than four. Be candid that currency amounts are your own arithmetic: the per-call cost tracking from v0.2 was listed as a future item in the migration guide, so you maintain a price table and keep it in one place.

Then name the controls in leverage order, because "we will watch it" is not a plan. Cap the conversation with max_turns and MaxMessageTermination, noting that it counts the task message — with MaxMessageTermination(3) the stop reason reads "Maximum number of messages 3 reached, current message count: 3" after the task and two replies. Cap spend directly with TokenUsageTermination. Cap the prompt with a bounded model context so cost stops growing with conversation length. And cache: model response caching has been off by default since v0.4, and ChatCompletionCache wrapping a client with a Redis or diskcache store pays for itself immediately in development.

Observability second. AutoGen gives two independent streams and a platform wants both. Python logging exposes TRACE_LOGGER_NAME for human-readable debugging, which is chatty enough to keep at warning level by default, and EVENT_LOGGER_NAME for structured events, which is the one you ship and index. OpenTelemetry instrumentation covers the runtime and emits GenAI semantic-convention spans named create_agent, invoke_agent and execute_tool; you pass tracer_provider= to the runtime or set the global provider, and AUTOGEN_DISABLE_RUNTIME_TRACING=true turns runtime spans off when they are too noisy. Any OTLP backend works.

Finish with the four dashboards, because a list of metrics shows you have operated something rather than read about it. Stop reasons per topology, where the ratio of semantic stops to budget stops is the quality signal. Tokens per session split by agent, which is cost and also how you find a runaway participant. Tool error rate by tool name from ToolCallExecutionEvent, since one flaky tool explains most reports that "the agent got dumber". And turns per completed task, which rises before anything else breaks when a prompt change degrades routing.

What a weak answer misses: that a run ending on a budget stop rather than a semantic one is a product signal and deserves an alert, not a log line. A platform where that ratio is unwatched will quietly start returning truncated answers, and nobody will notice until a customer does.

Security questions

"What is the threat model?" Model output is untrusted input. Every tool call, handoff target, URL and shell block derives from content that may include attacker-controlled text. AutoGen does not mitigate prompt injection and makes no claim to; the mitigations are structural — narrow tools with their own credentials, allowlists, approval functions, Core intervention handlers that can return DropMessage, and bounded authority per agent.

"How do you run model-generated code?" DockerCommandLineCodeExecutor, which has been the default wherever a default executor is created since 0.7.5, plus an approval_func on CodeExecutorAgent. Without one you get UserWarning: No approval function set for CodeExecutorAgent. LocalCommandLineCodeExecutor always warns that it "may execute code on the local machine which can be unsafe" — treat that string as a CI failure. ACADynamicSessionsCodeExecutor is the answer where workers have no Docker socket, or where generated code must stay in a specific region for data-residency reasons.

"Anything unusual you would flag?" Yes, and this is the answer that distinguishes a candidate who has read the source. In released 0.7.5, load_component() imports whatever module path the JSON's provider string names, so loading component JSON from an untrusted source is remote code execution. A fix on main restricts providers to the autogen_core, autogen_agentchat, autogen_ext and autogenstudio namespaces and adds AUTOGEN_ALLOWED_PROVIDER_NAMESPACES, but it is in no PyPI release — so on 0.7.5 you allowlist before the call. Also: dump_component() serialises an explicitly-passed api_key into the JSON, and AutoGen Studio is documented as a research prototype not meant for production, with no auth model, so it binds to 127.0.0.1.

Arguing the trade-off to leadership

This is the question the role actually turns on, and the mistake is to answer it in framework terms.

"Should we keep using AutoGen?" Lead with the fact and then with the decision. Maintenance mode means no new features and slow dependency updates; the mcp upper-bound problem is a concrete symptom, and two hardening fixes exist only on main. Against that: it is stable, your teams know it, and the code works. So the recommendation is usually to stay on a pinned 0.7.5 for existing services, build nothing new on it, and write down the triggers that would start a migration — an unpatchable CVE, a provider whose client is unmaintained, or hiring difficulty. Name an owner and revisit quarterly. Leadership can act on that; "it's in maintenance mode" alone just creates anxiety.

"How hard is the migration?" Give the mapping, not a feeling. AssistantAgent becomes Agent, FunctionTool becomes the @tool decorator, AgentTool(agent) becomes agent.as_tool(), RoundRobinGroupChat becomes SequentialBuilder, MagenticOneGroupChat becomes MagenticBuilder, GraphFlow becomes WorkflowBuilder, a nested team becomes a WorkflowExecutor, and model context and state become AgentSession. Then the two behavioural differences: AssistantAgent "is single-turn unless you increase max_tool_iterations" while Agent is multi-turn by default, so a naive port makes everything more agentic and more expensive; and the framework "focuses on single-process composition today", so a port is not a scaling upgrade. Check the matrix first, since the Anthropic and Ollama clients and ChatCompletionCache are listed as planned.

"Do we need multiple agents at all?" The honest answer scores well. Most production features are one model, a few tools and a loop. Multi-agent earns its place when roles genuinely differ in tools, context or authority, and costs coordination, extra model calls and more failure modes otherwise.

"Why not LangGraph or CrewAI?" Be specific and short. If the graph is the product, LangGraph is built around that and AutoGen's GraphFlow is experimental. If the goal is a simple collaborative flow quickly, CrewAI is role-shaped and faster to start. If the estate is .NET and plugin-heavy, Semantic Kernel, which also shares lineage with the successor framework. AutoGen's distinctive strength is the Core actor model and its subscription-based multi-tenancy.

The sixty-second self-test

Answer these out loud. Hesitating on any of them is where to spend your preparation.

Why can two requests not share one team object, and what does the error say? Where does session state live and what breaks when the topology changes? Which component queues first under load, and which one degrades silently instead of failing? What does max_tool_iterations default to, and what does reflect_on_tool_use=None resolve to? Which two model calls are invisible in a transcript but present in the bill? How does TypeSubscription turn a topic source into per-tenant isolation? What is the difference between ExternalTermination().set() and CancellationToken.cancel()? Why is load_component() dangerous on 0.7.5 and what landed on main? What exactly does maintenance mode mean for your risk register, and what are your migration triggers?

Where this picks up

The mid-level tips were about using AutoGen well in one service. These are about the failures that only appear when several teams build on the same runtime and the thing has to survive a quarter unattended. Every trap below is reproducible on 0.7.5, and every habit is one that saves an incident rather than a keystroke.

Traps that only appear at scale

A shared team object. The single most common production bug. Someone caches a team in a module-level global, and under two workers or two browser tabs you get ValueError: The team is already running, it cannot run again until it is stopped. It never reproduces in local testing with one request at a time. The fix is architectural: one team instance per in-flight run, built from a factory, state loaded and saved around it.

A lost write on a session. Two concurrent requests on one session both load the same state blob, both run, and the second save silently overwrites the first. Nothing errors. You find it weeks later as "the assistant forgot what we agreed". Use a per-session lock in Redis or an optimistic version check on the row, in the platform wrapper so no product team has to remember.

A state blob that no longer fits. save_state() output is tied to the component graph that produced it. Change participants or their order and old sessions load into a shape that does not match. Store a topology version alongside the state and start new sessions on a change rather than attempting a migration.

One blocking tool stalling every session. SingleThreadedAgentRuntime runs on one asyncio event loop. A synchronous HTTP call inside a tool blocks every other session in that process, and the symptom is a latency cliff with no error anywhere. Every I/O-bound tool is async def; anything CPU-bound goes to a thread or a queue.

A human holding a worker hostage. UserProxyAgent blocks a running team while it waits, and the team cannot be saved while it waits. One pending approval pins a worker and a session's memory for as long as the person takes. End the run instead — HandoffTermination(target="user") or a text mention — persist, return, and resume with a fresh run() when the reply arrives.

The unbounded context tax. Teams share one message thread, so a long session carries every message into every call. Cost and latency rise while quality falls, and nothing fails, which makes it the slowest trap to notice. Set BufferedChatCompletionContext or TokenLimitedChatCompletionContext explicitly rather than accepting UnboundedChatCompletionContext.

The two invisible model calls. SelectorGroupChat spends one call per turn choosing a speaker, visible only as a SelectSpeakerEvent. Reflection after tools spends another. And reflect_on_tool_use defaults to None, which resolves to True when output_content_type is set — so enabling structured output quietly enables reflection, and the bill rises after a change nobody connected to cost.

max_tool_iterations=1. The default. One model call, one round of tools, done. The inner loop arrived in 0.6.2, so older tutorials do not mention it and engineers conclude the agent is ignoring the second tool. Set it deliberately and bound it.

The silently dropped handoff. UserWarning: Multiple handoffs detected. Only the first is executed. Parallel tool calls produced several. Set parallel_tool_calls=False on the client of any agent with handoffs, and likewise wherever AgentTool or TeamTool appears, which the 0.7.2 notes document as unsupported in parallel.

The transitive release you did not choose. autogen-ext[mcp]==0.7.5 declares mcp>=1.11.0 with no upper bound, so a fresh install resolves 2.x and import autogen_ext.tools.mcp fails with ImportError: cannot import name 'RequestContext' from 'mcp.shared.context'. Pin mcp pinned below 2. Expect more of these: in maintenance mode nobody is updating the bounds.

An npx MCP server resolving latest in production. StdioServerParams(command="npx", args=["@playwright/mcp@latest", ...]) changes behaviour when somebody else publishes. Pin exact versions and mirror what matters.

Studio in the wrong place. It pins autogen-* below 0.6, so installing it next to 0.7.x code downgrades the runtime. It is documented as a research prototype with no production auth model. Separate virtualenv, --host 127.0.0.1, and never production credentials in its environment.

A dumped config with a key in it. dump_component() serialises an explicitly-passed api_key into its JSON. Rely on OPENAI_API_KEY and ANTHROPIC_API_KEY from the environment, and never commit a dumped config.

load_component() on anything a user can influence. On released 0.7.5 it imports whatever module path the provider string names. That is remote code execution in your worker. The namespace restriction and the AUTOGEN_ALLOWED_PROVIDER_NAMESPACES variable exist on main and in no release, so you allowlist before the call yourself.

A kill switch that corrupts sessions. CancellationToken.cancel() aborts immediately and can leave team state inconsistent. ExternalTermination().set() stops gracefully after the current agent's turn. Know which one your operations runbook invokes.

Platform-ownership habits

Ship an SDK, not a tutorial. An internal package with build_agent, build_team, a tool registry and a session store, carrying your defaults: timeouts, max_retries, parallel_tool_calls=False, a bounded context, a budget termination, tracing wired up, usage rows emitted. Teams import it rather than configuring AutoGen, and a policy change becomes a version bump instead of six pull requests.

Give every team a budget they cannot forget. Build the termination condition in the factory as an OR of a semantic stop and hard stops, so no team can ship a run without one. Remember that MaxMessageTermination counts the task message.

Keep a tool catalogue with an owner per entry. Name, scope, credential, quota, rate limit, owner. Review new tools for authority rather than correctness: the question is what the worst plausible call does. MCP servers are entries in the same catalogue, version-pinned, with secrets passed via env and never in a prompt where the model can echo them into a transcript.

Operate code execution as one service. One Docker-backed executor configuration, one approval policy that logs and hashes what it sees rather than returning an unconditional yes, one audit trail. Teams get a function call. Where workers have no Docker socket, or where generated code must stay in-region for data-residency reasons — a routine requirement for regulated customers in Saudi Arabia, the UAE and Egypt — ACADynamicSessionsCodeExecutor is the cleaner answer.

Answer the retention question once. Transcripts contain whatever users typed. Storage region, retention period and deletion path for session state and traces are compliance decisions, and a platform that settles them centrally saves every team from settling them differently.

Instrument before anyone asks. Ship EVENT_LOGGER_NAME output, keep TRACE_LOGGER_NAME at warning, export the create_agent, invoke_agent and execute_tool spans over OTLP, and stand up four dashboards: stop reasons per topology, tokens per session by agent, tool error rate by tool name, and turns per completed task.

Make the pins load-bearing and prove it nightly. All three packages at the same exact version, a committed lock file, and a scheduled job that re-resolves the tree from scratch and diffs it. That job is how you learn about the next mcp 2.x before production does.

Hold a written position on the framework. Supported version, upgrade cadence, deprecation policy, migration stance, and the triggers that would start a move to Microsoft Agent Framework: an unpatchable CVE, an unmaintained provider client, a needed capability that will not land, hiring difficulty. Name an owner, revisit quarterly. A frozen dependency is only a risk when nobody has agreed what would change their mind.

Say no to multi-agent when one agent would do. The most valuable thing a platform owner does with this framework is decline to use it. Roles must differ in tools, context or authority to be worth the coordination cost.

Operational recipes

Rolling out a defaults change across teams. Because the defaults live in your SDK, a change is a version bump — but a bump that alters agent behaviour is a product change for every consumer. Treat a narrowing change (a lower token cap, a shorter timeout, a new blocked pattern in the approval policy) as a minor release with a note naming the metric that will move, ship it behind a configuration flag defaulting to the old value, flip it per team, and watch the stop-reason dashboard for a week before flipping the default. The failure mode to avoid is a cap that silently starts truncating answers in a team you have never spoken to.

Rotating a model credential with no downtime. If clients read keys from the environment, rotation is a restart, and a restart is safe only because every request rebuilds its team from externalised state. That is the second payoff of the stateless design: there is no in-memory conversation to lose. On Azure, prefer no key at all — azure_ad_token_provider= with DefaultAzureCredential and get_bearer_token_provider from azure.identity, installed through the azure extra, so rotation happens outside your process entirely.

Investigating "the assistant got worse this week". Resist reading transcripts first. Go to turns per completed task, which rises before anything else when routing degrades, then to tool error rate by name, which catches the single flaky dependency that explains most of these reports. Only then look at prompts. More than half of the reports I would expect you to receive resolve to one tool timing out and the agents talking around the gap.

Draining a worker for deploy. A run in flight holds a team instance. Stop accepting new sessions, let current runs finish against a deadline, and for the stragglers use an ExternalTermination you can set from outside so they stop after the current agent's turn and their state saves cleanly. Reserve cancellation for the ones that outlast the deadline, and accept that those sessions may need to be restarted rather than resumed.

Onboarding a new team in an afternoon. Give them the SDK, a session store already provisioned, two tools from the catalogue, a budget they cannot exceed, and a dashboard that already has their tenant tag in it. Do not give them a blank repository and a link to the documentation; that is how you end up supporting four different answers to the state question.

Reproducing a production run locally. Keep the session state blob, the topology version and the SDK version in your incident notes, because together they are the only reproducible description of what ran. A ReplayChatCompletionClient is then enough to walk the orchestration without spending tokens or touching a provider, which also makes it the right client for tests in CI.

The habits that pay off most

In rough order of leverage, these are the things worth doing before anything else on the list.

Put the defaults in code, once. Every other habit here is easier if a factory already sets the timeout, the retries, the bounded context, the budget and the tracing. A written guideline nobody can enforce is worth roughly a tenth of a function nobody can bypass.

Make the unsafe thing hard to reach rather than forbidden. A CI gate on LocalCommandLineCodeExecutor beats a paragraph in a wiki, and an approval policy that auto-approves the shapes you have blessed beats one that asks a human about everything, because the second one gets switched off within a month.

Record the stop reason on every run. It is one field and it answers more questions than any other single piece of telemetry: whether your budgets are biting, whether a prompt change broke routing, whether a team is quietly shipping truncated answers.

Read the error strings. AutoGen's 0.7.5 messages are unusually precise, and most of them name the fix. Teams that habitually paste the exact message into a search box resolve these in minutes; teams that describe the symptom in their own words spend an afternoon.

Re-resolve the dependency tree on a schedule. For a framework in maintenance mode this is the single highest-value piece of automation you can own, because the next breakage will almost certainly arrive from a dependency rather than from AutoGen.

Keep the migration position current. One page, reviewed quarterly, naming the supported version, the triggers and the owner. It costs an hour a quarter and it is the difference between a deliberate choice and a slow accident.

The pull-request checklist

Run this on anything touching an agent. Most findings land in the first two groups.

• All three AutoGen packages pinned to the same exact version; mcp pinned below 2 where the MCP extra is used; lock file committed.
• No import autogen, ConversableAgent, initiate_chat, llm_config or config_list — those are AG2 or v0.2, not Microsoft AutoGen, whose imports are autogen_agentchat, autogen_core, autogen_ext.
• No OpenAIAssistantAgent: the OpenAI Assistants API was sunset on 2026-08-26, so it no longer works regardless of pin. Use OpenAIAgent or AssistantAgent.
• Every model client closed with await client.close(); every executor and McpWorkbench inside async with or explicitly started; a Core runtime started once and closed once, with the startup hook safe under a reloader.
• One team instance per in-flight run. No module-level team. Session lock present.
• A termination condition and a max_turns on every team, including at least one stop that does not depend on model behaviour.
• An explicit model context; max_tool_iterations set on purpose; usage rows emitted with tenant and session tags.
• No LocalCommandLineCodeExecutor outside an allowlisted test path; approval_func on every CodeExecutorAgent; work_dir not the current directory.
• parallel_tool_calls=False wherever handoffs, AgentTool or TeamTool appear.
• No load_component() on user-influenced input; no explicit api_key= where an environment variable would do; no committed dump_component() output.
• Every DeprecationWarning treated as a ticket: model_capabilities=, client.capabilities, response_format= holding a Pydantic model in create args, DiGraphBuilder.add_conditional_edges, max_consecutive_empty_chunk_tolerance.

The design-document checklist

• Why multiple agents, stated as a difference in tools, context or authority — not as a diagram preference.
• Why this topology, with the per-turn selector cost acknowledged if SelectorGroupChat is chosen, and the experimental status acknowledged if GraphFlow is.
• Where state lives, how it is keyed, how concurrency is controlled, how long it is kept and in which region.
• What the worst plausible tool call does, and which approval or intervention stands between the model and it.
• What the run costs at p50 and p95, which stop fires first, and what the caller sees when a budget stop fires.
• What degraded mode looks like when the model provider or the executor is unavailable, expressed as a response rather than a stack trace.
• The scaling claim, with no reliance on the experimental gRPC distributed runtime.
• The migration stance: staying on 0.7.5 or not, and what would change that.

Two CI gates catch most of the drift above on their own: fail the build on import autogen or on a pin that is not the exact supported version, and fail it on LocalCommandLineCodeExecutor outside the allowlist. Add the nightly dependency-resolution diff and you have covered the three failure classes — wrong package, unsafe default, and a dependency that moved — that account for most of what goes wrong with a framework nobody is developing any more.