The Complete CrewAI Guide

This is part one of three. It covers everything you need to build a working multi-agent application with CrewAI from an empty folder, not a teaser. By the end you can install the toolchain, scaffold a project, write agents and tasks that produce something you would actually send to a colleague, give an agent a tool, read the errors CrewAI throws at you, and explain the difference between a Crew and a Flow without hand-waving. Mid-level and Senior take the same topics further; nothing here is wasted.

Each section ends with a Try it task. Do them as you go. Multi-agent systems are unusually easy to misunderstand from reading, because the interesting failures are behavioural: an agent loops, an agent invents a tool that does not exist, an agent answers the wrong question confidently. You only learn to recognise those by watching your own run scroll past with verbose=True.

What CrewAI is, and the problem it solves

CrewAI is a Python framework for building applications in which several large-language-model agents work together on a job that is too big, too multi-step or too varied for a single prompt. You describe each agent by who it is and what it is for, you describe the work as a list of tasks, and the framework runs the agents, passes one task's output into the next task's context, lets them call tools, and hands you a result object at the end.

The problem it solves is the gap between a prompt and a program. A single call to a language model is a function: text in, text out. That is enough for summarising a paragraph. It is not enough for "research this market, check the numbers against a second source, and write me a two-page brief with citations", because that job has stages, each stage needs different instructions, some stages need to fetch data from the outside world, and the output of each stage has to be checked before it feeds the next.

Before frameworks like this, you wrote that orchestration by hand. The code was not hard, exactly, but it was tedious and it was all the same every time: a loop that calls the model, parses out whether the model wants to use a tool, runs the tool, appends the result to the message history, calls the model again, and gives up after N rounds so a confused model cannot spin forever. On top of that you built your own prompt templates, your own retry logic, your own way of keeping the second stage's prompt from accidentally dropping the first stage's findings, and your own logging so you could tell what went wrong.

CrewAI's bet is that this scaffolding should be a library, and that the natural way to describe the work is by role. You do not write "call the model with this system prompt, then that one". You write: here is a Researcher whose goal is to find verifiable facts about a topic; here is an Analyst whose goal is to turn findings into a brief; here are the two tasks; run them in order. The framework turns that into prompts, runs the tool loop for each agent, and threads the outputs together.

That framing is why it reads so easily and also why it bites beginners. Describing agents in natural language feels like configuration, so people write three vague sentences and expect a system. In practice the role, goal and backstory text is the prompt, and a vague prompt gives a vague system. A large part of becoming competent with CrewAI is learning to write those fields as specifications rather than as flavour.

Two things worth noticing before you write a line of code. First, CrewAI runs in your process: it is an ordinary Python library, not a server you deploy and talk to, and there is no scheduler or job queue underneath. When crew.kickoff() returns, everything has already happened on your machine. Second, every step an agent takes is a paid LLM call. A crew with three agents, four tasks and a tool each can easily make twenty model calls in one run. Keeping that number visible is a skill, and we come back to it.

The four nouns

Almost everything in CrewAI is built from four objects. Learn these precisely and the rest of the API is discoverable.

An Agent is an autonomous worker driven by a language model. It has three required fields — role, goal and backstory — and they are all prompt text. Optionally it gets an llm (which model it thinks with), tools (what it can do besides think), and limits. Internally the agent runs a loop called the executor: the model is asked what to do, and if it asks for a tool the tool is run and the result is fed back, repeating until the model produces a final answer or the loop hits max_iter. In CrewAI 1.15.23 that default is 25 iterations in the shipped code, even though the Agents documentation page still says 20; set it explicitly and you never have to care who is right.

A Task is a unit of work. It has two required fields, description (what to do) and expected_output (what a finished answer looks like), and it is normally assigned to an agent. Beyond that it can declare context — a list of other Task objects whose outputs should be made available to this one — plus its own tools, a structured output type, validation via guardrail, and an output_file to write the result to disk. The expected_output field is the single most under-used lever for beginners: it is what the model is told to aim at, so "a 300-word brief in markdown with a bulleted findings list and at least two source URLs" produces something usable where "a report" produces a shrug.

A Crew is a team: a list of agents, a list of tasks, and a Process. You run it with crew.kickoff(inputs={...}). The inputs dictionary does two jobs at once — it fills {placeholder} templates anywhere in your agent and task text, and it is the only sanctioned way to get runtime values into the run. What comes back is a CrewOutput object, not a string: result.raw is the text, result.pydantic and result.json_dict hold structured output when you asked for it, result.tasks_output is the list of per-task results, and result.token_usage is what the run cost in tokens.

A Process is the execution strategy, and there are exactly two. Process.sequential is the default and the one to learn first: tasks run in the order you listed them, and each task's output automatically becomes context for the next. Process.hierarchical instead puts a manager in charge — you supply either manager_llm or manager_agent — and the manager plans, delegates each task to whichever agent it judges suitable, and validates the result before moving on. There is no third option; if you read about a "consensual" process, you are reading about a plan that was never shipped.

Two more nouns appear often enough that you should recognise them now even though you will not configure them today. Tools are callables you hand an agent so it can search the web, read a file or call your API — the LLM decides when to invoke them and with what arguments. Knowledge is a set of documents attached to an agent or crew and queried automatically, which is CrewAI's built-in retrieval-augmented generation; Memory is the separate, unified store of things the system chooses to remember between runs.

Crews and Flows: the second half of the model

Crews give you autonomy: you say what the goal is, and the agents decide how many tool calls and reasoning steps to take. That is exactly what you want for research and drafting, and exactly what you do not want for the parts of your application that must behave the same way every time, such as deciding which branch of your pipeline to run, or saving a record.

For that, CrewAI has a second primitive: a Flow. A Flow is a Python class that inherits from Flow[StateModel] and whose methods are wired together with decorators: @start() marks an entry point, @listen(...) runs a method after another finishes, and @router(...) returns a string that decides which listener fires next. Combinators or_ and and_ let a step wait for either or both of two predecessors. A Flow carries state — either a plain dictionary or a Pydantic model, always with an automatic UUID id — so steps share data by reading and writing self.state rather than by passing arguments. The flow's result is the return value of whichever method completes last.

The official production guidance is worth internalising early, because it saves you from the most common architectural mistake: use a Flow as the backbone and call Crews, or single agents, from inside Flow steps. The deterministic skeleton — routing, validation, persistence, waiting for a human to approve something — lives in the Flow, where you can test it. The open-ended parts live in Crews, where non-determinism is the point. Beginners who build everything as one large hierarchical crew end up with a system whose control flow is decided by a language model, which is impossible to debug.

from pydantic import BaseModel
from crewai.flow import Flow, start, listen, router, or_


class State(BaseModel):
    topic: str = ""
    report: str = ""


class BriefFlow(Flow[State]):
    @start()
    def begin(self):
        self.state.topic = self.state.topic.strip()

    @router(begin)
    def pick_depth(self):
        return "long" if len(self.state.topic) > 10 else "short"

    @listen("long")
    def deep_pass(self):
        self.state.report = f"Detailed brief on {self.state.topic}"

    @listen(or_("short", deep_pass))
    def finish(self):
        return self.state.report or f"Short note on {self.state.topic}"


print(BriefFlow().kickoff(inputs={"topic": "AI agents in logistics"}))

Three details make Flows more than syntax. A flow can be persisted with the @persist decorator, which stores its state (SQLite by default) so a run can be resumed later with kickoff(inputs={"id": "<uuid>"}). It can pause for a human using the @human_feedback decorator, which is how you build an approval step without inventing your own queue. And flow.plot("my_flow") writes an interactive HTML diagram of the wiring, which is the fastest way to confirm that the graph in your head matches the graph in your code.

A third, independent axis is how you write a crew down. Since version 1.15.0 the default scaffold is declarative: the crew is described in JSONC files (JSON with comments) and loaded with crewai.project.load_crew. The older classic layout — a Python class decorated with @CrewBase plus config/agents.yaml and config/tasks.yaml — is still fully supported behind the --classic flag. Both produce the same objects. Start with the JSONC default, because that is what crewai create crew gives you and what the current documentation walks through.

Installing CrewAI and checking the setup

You need Python 3.10, 3.11, 3.12 or 3.13. Python 3.14 is explicitly not supported at 1.15.23, which is a real trap if your machine just upgraded itself.

python3 --version     # macOS and Linux
py --version          # Windows

The officially supported path uses uv, Astral's Python package manager. This is not a stylistic preference: CrewAI's own CLI shells out to uv for crewai install and crewai run, so a project set up another way will fight you.

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows PowerShell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Now install the CrewAI command-line tool globally, in its own isolated environment, so it never collides with a project's dependencies.

uv tool install crewai
uv tool update-shell        # only if you get a PATH warning; then open a new shell

Verify two things, because they are two different installations and they drift apart:

uv tool list       # shows: crewai v1.15.23  (plus its executables)
crewai version     # the CLI's own version; add --tools to also see crewai-tools

# inside a project, the library version is separate from the CLI version
uv pip show crewai

That distinction is the single most confusing thing about CrewAI's setup, so state it to yourself clearly: the global CLI is what provides the crewai command, and each project has its own .venv with its own copy of the library. Upgrading one does not upgrade the other, and crewai install syncs a project to its lockfile without ever bumping a version. Keep both on the same release, because the crewai, crewai-core, crewai-cli and crewai-tools packages are version-locked to each other.

uv tool install crewai --upgrade        # the global CLI
uv add "crewai[tools]>=1.15.23"         # the project: edits pyproject.toml and uv.lock
crewai install                          # sync the project venv

If you only want the library — you are adding agents to an existing application and do not need the scaffolding — skip the tool install entirely:

uv add crewai                        # or: pip install crewai
uv add "crewai[tools]"               # the built-in tool collection
uv add "crewai[anthropic]"           # native Anthropic SDK
uv add "crewai[google-genai]"        # native Gemini
uv add "crewai[litellm]"             # anything with no native provider

Those extras matter more than they look. Since 1.12.0 CrewAI talks to a set of providers natively — OpenAI, Anthropic/Claude, Azure, Google/Gemini, Bedrock/AWS, OpenRouter, DeepSeek, Ollama, vLLM via hosted_vllm, Cerebras, DashScope and Snowflake — and anything outside that list needs the optional LiteLLM fallback. OpenAI itself is a core dependency and needs no extra.

Finally, the version numbers you will see in old material. The installation page's sample output still shows crewai v0.102.0, and a note there says CrewAI 0.175.0 requires openai >= 1.13.3. Both are stale: 1.15.23 requires openai>=2.30.0. Treat the version you can see with crewai version as the truth.

Your first crew, step by step

Scaffold a project. The command prompts you for a provider and a model, then writes the API key you give it into .env.

crewai create crew research_crew
cd research_crew
crewai install

Use --provider <name> to preselect a provider non-interactively, or --skip-provider to skip the prompt entirely and fill .env in yourself. If you want the older Python-and-YAML layout, add --classic.

What you get is a JSON-first project:

research_crew/
├── .env
├── .gitignore
├── agents/
│   └── researcher.jsonc
├── crew.jsonc
├── knowledge/
├── skills/
├── tools/
├── pyproject.toml
└── README.md

An agent lives in its own file under agents/, and the file name is the agent's name everywhere else:

{
  "role": "Senior Research Specialist for {topic}",
  "goal": "Find verifiable, recent facts about {topic} and keep the source for each one",
  "backstory": "A careful analyst who distrusts single sources and always notes the date of a claim.",
  "llm": "openai/gpt-4.1-mini",
  "settings": { "verbose": true, "allow_delegation": false }
}

The crew file names the agents, lists the tasks in order, and sets the process:

{
  "name": "Research Crew",
  "agents": ["researcher", "analyst"],
  "tasks": [
    {
      "name": "research_task",
      "description": "Research the current state of {topic}. Prefer sources from the last 18 months.",
      "expected_output": "Eight findings. Each one sentence, each followed by its source URL.",
      "agent": "researcher"
    },
    {
      "name": "analysis_task",
      "description": "Turn the findings into a brief for a non-specialist decision maker.",
      "expected_output": "A 400-word markdown brief: a two-sentence summary, four themes, and the source list.",
      "agent": "analyst",
      "context": ["research_task"],
      "output_file": "output/report.md",
      "markdown": true
    }
  ],
  "process": "sequential",
  "verbose": true,
  "inputs": { "topic": "Artificial Intelligence in Healthcare" }
}

Three rules govern these files and each one is a real error if you break it. Every name in agents must match a file in agents/. Every task's agent must match one of those names. And every {placeholder} in any agent or task text must either appear in inputs or be supplied at run time — crewai run will prompt you for anything missing rather than silently leaving the literal braces in the prompt.

Then run it:

crewai run

With verbose on you will see each agent announce its role, think, optionally call a tool, and produce a final answer, then the next task start with the previous output in its context. A fresh scaffold writes output/report.md when it finishes. Open that file: the point of output_file is that the deliverable lands on disk rather than scrolling off your terminal.

The same crew can be loaded from Python when you want to embed it in an application:

from pathlib import Path
from crewai.project import load_crew

crew, default_inputs = load_crew(Path("crew.jsonc"))
result = crew.kickoff(inputs={**default_inputs, "topic": "AI in radiology"})
print(result.raw)
print(result.token_usage)

The same crew in plain Python

The JSONC scaffold is convenient, but you should be able to write a crew in a single file, because that is how you will experiment and how most examples you read are written.

from crewai import Agent, Task, Crew, Process, LLM

llm = LLM(model="openai/gpt-4.1-mini", temperature=0.2)

researcher = Agent(
    role="Researcher",
    goal="Find verifiable facts about {topic}",
    backstory="A careful analyst who always records the source of a claim.",
    llm=llm,
    max_iter=25,
    verbose=True,
)

research = Task(
    description="Research {topic} and note the date of each claim.",
    expected_output="Five findings, one sentence each, each with a source URL.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[research], process=Process.sequential, verbose=True)

result = crew.kickoff(inputs={"topic": "AI agents"})
print(result.raw)
print(result.token_usage)

Note the model string. CrewAI always wants the provider/model-id form — openai/gpt-4.1-mini, anthropic/claude-sonnet-4-6, ollama/llama3 — because the prefix is how it decides which provider implementation to use. A bare model name is the most common reason a beginner's first script cannot find a provider at all.

If you set no llm, CrewAI resolves one from the environment in order: MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME, and failing all three it falls back to gpt-4.1-mini. That fallback is why a script with no OpenAI key can fail with an OpenAI authentication error even when you thought you had configured Anthropic everywhere. (The documentation table that says the default is gpt-4 is stale.)

For jobs that need no team at all, an Agent can be used alone. This is the cheapest way to test whether your role and goal text is any good, because there is no task plumbing in the way:

from crewai import Agent

agent = Agent(
    role="Editor",
    goal="Rewrite text so a busy executive can read it in thirty seconds",
    backstory="Twenty years of cutting other people's paragraphs in half.",
)

out = agent.kickoff("Rewrite: our Q3 initiative achieved synergies across verticals.")
print(out.raw)

agent.kickoff(...) returns a LiteAgentOutput with raw, pydantic, agent_role and usage_metrics — not a CrewOutput. There is an async twin, agent.kickoff_async(...), and the crew equivalents are crew.akickoff(...) for native async and crew.kickoff_async(...) for a thread-based version. You need one of them if you are calling CrewAI from inside FastAPI or a Jupyter notebook, because invoking the synchronous kickoff from within a running event loop raises an error that tells you exactly this.

To run the same crew over many inputs, crew.kickoff_for_each(inputs=[{...}, {...}]) loops for you and returns a list of outputs, with kickoff_for_each_async and akickoff_for_each as the concurrent versions.

Writing agents and tasks that actually work

Everything about the quality of a crew's output traces back to four text fields. Here is what separates a definition that works from one that produces mush.

The goal should name the decision the output feeds. The backstory is where you put constraints and taste — "never state a number you cannot attribute", "prefer primary sources", "write for a reader who is not a doctor" — because that text is in the system prompt for every step the agent takes. The expected_output is the shape of the artefact, and it is the field people leave weakest even though it has the most direct effect on whether the result is usable.

Beyond text, a handful of Task features turn a demo into something you can rely on.

Context makes dependencies explicit. In a sequential crew each output flows to the next task automatically, but context=[research_task, pricing_task] says precisely which earlier outputs this task should see. Order matters: a task may not reference a task that comes later, and CrewAI refuses the run with a message naming the forward dependency rather than silently producing nonsense.

Structured output replaces parsing prose. Pass a Pydantic model as output_pydantic and the finished result is available as result.pydantic:

from pydantic import BaseModel
from crewai import Agent, Task, Crew


class Brief(BaseModel):
    summary: str
    themes: list[str]
    sources: list[str]


analyst = Agent(role="Analyst", goal="Summarise findings", backstory="Blunt and brief.")

task = Task(
    description="Summarise the findings about {topic}.",
    expected_output="A summary, three themes, and the source URLs.",
    agent=analyst,
    output_pydantic=Brief,
)

result = Crew(agents=[analyst], tasks=[task]).kickoff(inputs={"topic": "AI agents"})
print(result.pydantic.themes)

Two rules here, both of which CrewAI enforces with a clear error. Pass the class (Brief), never an instance (Brief(...)). And set either output_pydantic or output_json, never both.

Guardrails validate the output and retry if it fails. A guardrail is either a function taking one TaskOutput and returning a (bool, value) tuple, or a plain string that becomes an LLM-judged check using the task's own agent. Failures retry up to guardrail_max_retries, which defaults to 3.

from typing import Any
from crewai import TaskOutput


def has_sources(result: TaskOutput) -> tuple[bool, Any]:
    if "http" not in result.raw:
        return (False, "Output contained no source URLs. Add at least two.")
    return (True, result.raw)


# Task(..., guardrail=has_sources)
# Task(..., guardrail="Must cite at least two URLs")
# Task(..., guardrails=[has_sources, "Under 400 words"])

Human input is the simplest safety valve there is: Task(..., human_input=True) pauses and asks you to review the output before the crew moves on. For an application rather than a terminal session, the Flow decorator @human_feedback is the grown-up version.

output_file writes the result to disk, creating the directory when you pass create_directory=True. CrewAI validates this path strictly — no .., no ~, no shell characters — and refuses anything that looks like path traversal, so use plain relative paths like output/report.md.

Giving an agent tools

An agent with no tools can only reason over what is already in its prompt. Tools are how it reaches the world: search the web, read a file, call your internal API. The agent's model decides when to call a tool and what arguments to pass, which is both the power and the risk.

The built-in collection ships in crewai-tools, installed with the tools extra. It includes SerperDevTool for web search (needs SERPER_API_KEY), ScrapeWebsiteTool, FileReadTool, FileWriterTool, DirectoryReadTool, PDFSearchTool, CSVSearchTool, WebsiteSearchTool, ExaSearchTool and GithubSearchTool, among others.

uv add "crewai[tools]"

from crewai import Agent, Task, Crew
from crewai_tools import SerperDevTool

search = SerperDevTool()

researcher = Agent(
    role="Market researcher",
    goal="Find and cite current facts about {topic}",
    backstory="Never states a figure without a source URL.",
    tools=[search],
    verbose=True,
)

task = Task(
    description="Search for recent developments in {topic}.",
    expected_output="Six findings, each with a source URL.",
    agent=researcher,
)

print(Crew(agents=[researcher], tasks=[task]).kickoff(inputs={"topic": "agentic AI"}).raw)

Writing your own tool is a small class. Note the import path: it is from crewai.tools import BaseTool, tool. The old crewai_tools.BaseTool and crewai.agents.tools.tool paths no longer exist, and a surprising amount of published example code still uses them.

from crewai.tools import BaseTool
from pydantic import BaseModel, Field


class ConvertInput(BaseModel):
    amount: float = Field(..., description="Amount in US dollars")
    rate: float = Field(..., description="Dollars per unit of target currency")


class ConvertTool(BaseTool):
    name: str = "Convert USD"
    description: str = "Convert an amount in US dollars to another currency at a given rate."
    args_schema: type[BaseModel] = ConvertInput

    def _run(self, amount: float, rate: float) -> str:
        return f"{amount / rate:.2f}"

For something trivial, the decorator form is shorter, and the docstring becomes the description:

from crewai.tools import tool


@tool("Word count")
def word_count(text: str) -> str:
    """Count the words in a piece of text and return the number."""
    return str(len(text.split()))

The description and the field descriptions are not documentation; they are the only thing the model reads when deciding whether this tool is the right one. A vague description is the direct cause of the two most common tool failures: the agent calls the wrong tool, or it invents a tool name that does not exist and CrewAI replies with You tried to use the tool {tool}, but it doesn't exist.

Three current facts about tools that older material gets wrong. Tool-result caching is off by default: Crew.cache has defaulted to False since 1.14.3, and Agent.cache=True only means that agent participates when caching is enabled on the crew. Enable Crew(cache=True) deliberately, and only for tools whose results do not change. There is no built-in code execution any more: CodeInterpreterTool was removed in 1.14.0, and Agent.allow_code_execution and code_execution_mode are deprecated no-ops whose warning points you to external sandboxes such as E2B or Modal. And MCP is built in: Model Context Protocol servers are attached with Agent(mcps=[...]), taking either string references or MCPServerStdio, MCPServerHTTP and MCPServerSSE objects from crewai.mcp. If you are curious about that protocol, the MCP guide covers it from first principles.

The everyday commands, grouped by what you want

You will use perhaps eight commands regularly. Grouping them by intent is easier than memorising a list.

Starting something. crewai create crew NAME scaffolds a JSON-first crew, with --classic for the YAML layout, --provider to preselect a provider and --skip-provider to skip the prompt. crewai create flow NAME scaffolds a Python Flow project, with --declarative for the JSON-described variant. crewai create tool NAME and crewai create skill NAME add a custom tool or a reusable instruction package.

Running it. crewai install is uv sync into the project's .venv. crewai run reads [tool.crewai] in pyproject.toml to decide whether this project is a crew or a flow and runs it; for flows it accepts --definition PATH and --inputs '{"topic":"AI"}'. crewai chat opens an interactive session with your crew, which requires a chat_llm set on the crew.

Looking at state. crewai version, with --tools to include crewai-tools. crewai flow plot draws the flow graph. crewai log-tasks-outputs lists the task IDs from the most recent kickoff, and crewai replay -t TASK_ID re-runs from that task onward — the fastest way to iterate on a late task without paying for the early ones again.

Clearing state. crewai reset-memories needs at least one flag or it refuses: -m for memory, -kn for knowledge, -akn for agent knowledge, -k for the latest kickoff outputs, -a for everything.

crewai reset-memories -m        # the unified memory store
crewai reset-memories -a        # everything, after changing an embedder

Two more command families exist, and you should know they are there even though a beginner does not need them. crewai train -n 5 runs iterations collecting your feedback into trained_agents_data.pkl, which agents then load as suggestions, and crewai test -n 3 runs a crew repeatedly and scores it with a model. And a set of commands — crewai login, crewai deploy create, crewai deploy push, crewai deploy logs, crewai traces enable — connect to CrewAI AMP, the commercial Agent Management Platform at app.crewai.com that offers hosted deployment, a REST endpoint per crew, and trace viewing. None of it is required to use the framework.

Also note the CLI's recent renames, because the old spellings appear everywhere: flags became kebab-case (--n-iterations, --task-id, --skip-provider), crewai tool create became crewai create tool, and crewai flow kickoff became crewai run — the old form still works but prints a deprecation notice.

Configuration: models, keys, memory and knowledge

Configuration is mostly environment variables plus a handful of constructor arguments, and the thing to understand is not the list but where hidden model calls come from.

Your .env needs a provider key and, usually, a model:

OPENAI_API_KEY=sk-...
MODEL=openai/gpt-4.1-mini
SERPER_API_KEY=...

Other providers follow the same shape: ANTHROPIC_API_KEY, GEMINI_API_KEY or GOOGLE_API_KEY, AZURE_API_KEY with AZURE_ENDPOINT, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with a region, SNOWFLAKE_PAT with SNOWFLAKE_ACCOUNT_URL. Always write the model as provider/model-id. The scaffold's .gitignore excludes .env; keep it that way.

Now the part that costs people money. Several CrewAI features default to an OpenAI model independently of the agent you configured. Crew planning (planning_llm), the memory system's analysis model, and the default for crewai test -m all default to gpt-5.4-mini in the 1.15.23 code, and memory's default embedder is OpenAI's text-embedding-3-large while knowledge's is text-embedding-3-small. So a crew where every agent runs on Anthropic can still fail without OPENAI_API_KEY, or quietly bill OpenAI. If you are standardising on one provider, override every one of them: each agent's llm, plus planning_llm, manager_llm, Memory(llm=..., embedder=...), and the knowledge embedder.

from crewai import Crew, Process

crew = Crew(
    agents=[...],
    tasks=[...],
    process=Process.sequential,
    verbose=True,
    memory=False,                  # default; True needs an embedder and a key
    cache=False,                   # default since 1.14.3
    max_rpm=20,                    # rate-limit the whole crew
    output_log_file=True,          # writes logs.txt
)

The Crew defaults worth knowing by heart at this level: process=sequential, verbose=False, memory=False, cache=False, planning=False, share_crew=False. On the Agent: max_iter=25, allow_delegation=False, max_rpm=None, max_execution_time=None, max_retry_limit=2, respect_context_window=True (which auto-summarises when a prompt would overflow the model's window, at the cost of extra calls). Note that verbose is now a plain boolean — passing an integer to select a log level is a 0.x habit that no longer does anything.

Knowledge is the feature to reach for when your agents need to know about your documents. Attach sources at crew or agent level and CrewAI embeds them into ChromaDB and queries them automatically:

from crewai import Crew
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

policy = StringKnowledgeSource(content="Refunds are issued within 14 days of purchase.")

crew = Crew(agents=[...], tasks=[...], knowledge_sources=[policy])

File-based sources exist for text, PDF, CSV, Excel and JSON, and their paths are resolved relative to the project's knowledge/ directory, which is the single most common cause of a "file not found" here. The store itself lives in a per-OS application-data directory — ~/Library/Application Support/CrewAI/{project}/knowledge/ on macOS, ~/.local/share/CrewAI/{project}/knowledge/ on Linux, %LOCALAPPDATA%\CrewAI\{project}\knowledge\ on Windows — and you can move all CrewAI storage by setting CREWAI_STORAGE_DIR.

Memory is different from knowledge and the two are constantly confused. Knowledge is documents you provide, read-only, for retrieval. Memory is what the system records as it runs so a later run can recall it. Since the unification there is one Memory class backed by LanceDB, stored by default in ./.crewai/memory, with memories filed under hierarchical scopes like /project/alpha and recall scored by a blend of semantic similarity, recency and importance. Turning it on is Crew(memory=True); that requires an embedder, which by default means an OpenAI key.

Two operational notes. If you change embedder, reset the stores — old stores were 1536-dimensional, the current default is 3072, and mixing them produces a dimension-mismatch error. And all of this state is local files by default, which matters the moment you put a crew in a container: mount a volume or set CREWAI_STORAGE_DIR, or your memory vanishes between runs.

Reading the errors

CrewAI's error messages are unusually direct, and most of them name the fix. Learning to recognise the common ones turns an afternoon of confusion into a two-minute correction.

Provider and model problems. Unable to initialize LLM with model '<m>'. The model did not match any supported native provider ... and the LiteLLM fallback package is not installed. means your prefix is not one of the native providers; install the fallback with uv add 'crewai[litellm]' or switch providers. A message like Anthropic native provider not available, to install: uv add "crewai[anthropic]" means the right provider but a missing extra. An OpenAI authentication complaint in a crew you thought was all-Anthropic almost always means a hidden default: an agent with no llm, or planning, memory or the test command reaching for OpenAI.

Configuration refusals. Attribute 'manager_llm' or 'manager_agent' is required when using hierarchical process. and Manager agent should not be included in agents list. are the two hierarchical-crew mistakes, in that order. Sequential process error: Agent is missing in the task with the following description: {description} means exactly what it says. Only one output type can be set, either output_pydantic or output_json. and a Pydantic validation error on output_pydantic (you passed an instance instead of the class) are the structured-output pair. Missing required template variable '<var>' in description means a {placeholder} with nothing to fill it.

Ordering refusals. Task '<d>' has a context dependency on a future task '<d2>', which is not allowed. means reorder your tasks. The crew must end with at most one asynchronous task. means make the trailing tasks synchronous or add a synchronous aggregator at the end.

Behavioural failures, which are the interesting ones. If your output ends with the sentence "You can't keep going, here is the best final answer you generated:", the agent hit max_iter. CrewAI makes one final call demanding a best effort and gives you that. The cause is almost never that 25 iterations is too few; it is usually a tool that keeps failing, so the agent keeps retrying it, or a task so broad the agent cannot tell when it is done. Fix the tool or tighten the task before you raise the limit.

Invalid response from LLM call - None or empty. means the provider returned nothing: check the model id for a typo, raise max_tokens, consider content filtering, and look at the verbose log. Context length exceeded. Summarizing content to fit the model context window. Might take a while... is respect_context_window doing its job; the fix is less input, not a bigger window. Task '<d>' execution timed out after <n> seconds. is your own max_execution_time.

Tool failures have their own family: You tried to use the tool {tool}, but it doesn't exist. You must use one of the following tools, use one at time: {tools}. is a hallucinated tool name, usually from weak descriptions or a weak model. Error: the Action Input is not a valid key, value dictionary. means the model produced malformed arguments; simplify the args_schema. And I encountered an error while trying to use the tool. This was the error: {error}. is your tool raising an exception — which it should not do in normal operation; return a ToolFailure instead so the agent can react.

Finally, two environment classics. Agent execution was invoked synchronously from within a running event loop. tells you to use kickoff_async or akickoff, and it happens the first time you call a crew from FastAPI or a notebook. And Error. A valid pyproject.toml file is required. from crewai uv ... simply means you are in the wrong directory.

Putting it all together

Here is one small end-to-end project that uses everything above: a Flow as the backbone, a two-agent crew inside it, a guardrail, structured output and a file on disk. It is deliberately small enough to read in one sitting.

from pathlib import Path
from typing import Any

from pydantic import BaseModel
from crewai import Agent, Crew, LLM, Process, Task, TaskOutput
from crewai.flow import Flow, listen, start


class Brief(BaseModel):
    summary: str
    themes: list[str]
    sources: list[str]


def has_two_sources(result: TaskOutput) -> tuple[bool, Any]:
    if result.raw.count("http") < 2:
        return (False, "Fewer than two source URLs. Add more and cite them inline.")
    return (True, result.raw)


def build_crew() -> Crew:
    llm = LLM(model="openai/gpt-4.1-mini", temperature=0.2)

    researcher = Agent(
        role="Research specialist on {topic}",
        goal="Find recent, attributable facts about {topic}",
        backstory="Never states a figure without a source. Notes the date of every claim.",
        llm=llm,
        max_iter=25,
        verbose=True,
    )
    analyst = Agent(
        role="Analyst writing for a non-specialist decision maker",
        goal="Turn findings about {topic} into a brief someone can act on",
        backstory="Cuts jargon. Refuses to pad. Keeps every claim traceable to a source.",
        llm=llm,
        max_iter=25,
        verbose=True,
    )

    research = Task(
        description="Research the current state of {topic}, preferring the last 18 months.",
        expected_output="Six findings, one sentence each, each followed by its source URL.",
        agent=researcher,
        guardrail=has_two_sources,
    )
    write = Task(
        description="Write a brief from the findings for a reader with no background in {topic}.",
        expected_output="A summary of two sentences, three themes, and the list of sources.",
        agent=analyst,
        context=[research],
        output_pydantic=Brief,
        output_file="output/brief.md",
        create_directory=True,
        markdown=True,
    )

    return Crew(
        agents=[researcher, analyst],
        tasks=[research, write],
        process=Process.sequential,
        verbose=True,
    )


class State(BaseModel):
    topic: str = ""
    brief: Brief | None = None


class BriefFlow(Flow[State]):
    @start()
    def normalise(self):
        self.state.topic = self.state.topic.strip() or "AI agents in healthcare"

    @listen(normalise)
    def research_and_write(self):
        result = build_crew().kickoff(inputs={"topic": self.state.topic})
        self.state.brief = result.pydantic
        print("tokens:", result.token_usage)
        return self.state.brief


if __name__ == "__main__":
    brief = BriefFlow().kickoff(inputs={"topic": "AI agents in radiology"})
    print(brief.summary)
    print(Path("output/brief.md").read_text()[:400])

Walk through what happens when you run it. The flow starts, normalises the topic in state, and calls its single listener. That listener builds a crew and kicks it off with the topic as an input, which interpolates {topic} into both agents' role and goal text and both tasks' descriptions. The researcher runs its executor loop, producing findings; the guardrail inspects the raw output and, if fewer than two URLs appear, returns a failure message that is fed back to the agent for another attempt, up to three times. The analyst then runs with the research output in its context, and because the task declares output_pydantic, CrewAI coerces the answer into a Brief and also writes the markdown to output/brief.md, creating the directory. The crew returns a CrewOutput; the flow stores the parsed model in state, prints the token cost, and returns it as the flow's result.

Everything here is a lever you now understand. Want cheaper? Give the researcher a smaller model via its own llm and leave the analyst on the larger one. Want it auditable? Add output_log_file=True to the crew. Want a human in the loop? Add human_input=True to the writing task, or move the approval into the flow with @human_feedback. Want it resumable? Decorate the flow with @persist and resume with kickoff(inputs={"id": "<uuid>"}). Want to run it over fifty topics? Replace the single kickoff with kickoff_for_each.

What you can now do, and what comes next

You can install the CrewAI toolchain the supported way and explain why the global CLI and the project library are separate installs. You can scaffold a JSON-first project, read and edit crew.jsonc and the files under agents/, and write the same crew directly in Python. You know the four nouns and the two processes, and you know that a Flow is the deterministic backbone that should own your control flow while crews own the open-ended parts. You can write role, goal, backstory and expected_output text that specifies rather than gestures. You can give an agent built-in or custom tools, with the correct crewai.tools import path. You can enforce output shape with Pydantic and output quality with guardrails. You know where hidden OpenAI calls come from and how to override each one. And you can read the error messages, including the behavioural ones that have no stack trace.

What you have deliberately not covered: the hierarchical process in anger, async and parallel execution, streaming, event listeners and the event bus, checkpointing and forking runs, crew-level and agent-level planning, declarative flows, MCP servers in depth, and deployment. Those are the mid-level material, and they all build on exactly the model you now have.

Three habits to carry forward. Keep verbose=True until a crew is boring, because the log is the only account of what the agents actually did. Pin your versions and read the changelog, because 1.x ships roughly weekly and defaults have genuinely changed inside the minor line — tool caching, max_iter, the default models. And treat the number of LLM calls per run as a first-class metric from the beginning; it is the difference between a demo and something you can afford to run.

For where to go next in this catalogue: LangGraph is the natural comparison, a graph-first framework where you build the state machine yourself, and reading the two against each other sharpens what CrewAI's role abstraction is actually buying you. Once a crew runs more than a few times a day you will want to see inside it, which is what Langfuse is for — CrewAI's event bus and OpenTelemetry-based tracing integrate with it directly. And when you need to know whether a change made your output better rather than just different, RAGAS covers evaluation. If your crews talk to tools over Model Context Protocol, the MCP guide explains the protocol your mcps=[...] list is speaking.

Sources

• CrewAI documentation
• Installation
• Quickstart
• Build your first crew
• Changelog
• Upgrading CrewAI
• CLI reference
• Agents
• Tasks
• Crews
• Processes
• Flows
• Tools
• LLMs
• Memory
• Knowledge
• MCP overview
• Telemetry
• crewai on PyPI

The thirty-second answer

CrewAI is a Python framework for building applications where several LLM-driven agents collaborate on a multi-step job: you define each agent by its role, goal and backstory, define the work as tasks with explicit expected outputs, and group them into a crew that runs the tasks in order and threads each output into the next. It also ships a second, deterministic primitive called a Flow — a decorator-wired state machine — and the recommended production shape is a Flow as the backbone that calls crews or single agents from its steps.

Say that and stop. If you want one extra sentence to show currency, use this: "I work against the 1.x line, 1.15.23 at the moment, where the default scaffold is JSON-first rather than the YAML layout most tutorials still show."

That last clause does disproportionate work in an interview. CrewAI changed heavily between 0.x and 1.x, 1.0.0 only arrived in October 2025, and the majority of available training material describes behaviour that is now wrong. An interviewer who uses the tool will notice immediately whether you are quoting 2024 blog posts or the current framework.

The model you must be able to draw

Four nouns, and you should be able to say each one's required fields without pausing.

Agent — an autonomous worker driven by a model. Required: role, goal, backstory, all of which are prompt text. It runs an executor loop (reason, maybe call a tool, feed the result back, repeat) until it produces a final answer or hits max_iter. In the 1.15.23 source that default is 25; the Agents documentation page still says 20, which is a good thing to know because it shows you have read the code and not only the docs.

Task — a unit of work. Required: description and expected_output. Normally assigned to an agent. Can declare context (a list of earlier tasks whose output feeds in), output_pydantic or output_json for structured output, guardrail/guardrails for validation, output_file, and human_input.

Crew — agents plus tasks plus a process. crew.kickoff(inputs={...}) interpolates {placeholders} throughout the agent and task text and returns a CrewOutput with raw, pydantic, json_dict, tasks_output and token_usage.

Process — exactly two: Process.sequential (the default; tasks run in order and each output becomes context for the next) and Process.hierarchical (a manager, supplied as either manager_llm or manager_agent, plans, delegates and validates). There is no third. If an interviewer mentions a "consensual" process, the correct answer is that it does not exist.

Then the second layer: Flows for deterministic control (@start, @listen, @router, or_/and_, typed state, @persist, @human_feedback), tools as the agent's reach into the world, knowledge as documents you supply for retrieval, and memory as what the system records across runs.

The commands to have ready

You will not be asked to recite a CLI, but being fluent in six commands reads as experience.

uv tool install crewai            # the global CLI, isolated
crewai create crew my_crew        # JSON-first scaffold; --classic for Python/YAML
crewai install                    # uv sync into the project .venv
crewai run                        # crew or flow, decided by [tool.crewai] in pyproject
crewai version --tools            # CLI version plus crewai-tools
crewai reset-memories -m          # unified memory; -kn knowledge, -a everything

Two things to volunteer alongside them. First, the global CLI and the project library are separate installations, and crewai install only syncs the lockfile — it never bumps a version. Upgrading properly is uv tool install crewai --upgrade for the CLI and uv add "crewai[tools]>=1.15.23" then crewai install for the project. Second, crewai log-tasks-outputs plus crewai replay -t TASK_ID is how you iterate on a late task without paying to re-run the early ones. Mentioning replay unprompted signals that you have actually sat and tuned a crew.

Also worth knowing: uv is not optional in spirit. The CLI shells out to it, so "I set it up with pip and a venv" invites a follow-up you do not need.

The minimal code you should be able to write on a whiteboard

from crewai import Agent, Task, Crew, Process, LLM

llm = LLM(model="openai/gpt-4.1-mini", temperature=0.2)

researcher = Agent(
    role="Researcher",
    goal="Find verifiable facts about {topic}",
    backstory="Never states a figure without a source.",
    llm=llm,
    max_iter=25,
    verbose=True,
)

task = Task(
    description="Research {topic}.",
    expected_output="Five findings, one sentence each, each with a source URL.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task], process=Process.sequential)
result = crew.kickoff(inputs={"topic": "AI agents"})
print(result.raw, result.token_usage)

Three details in that snippet are deliberate and each one is a possible question. The model string is provider/model-id, because the prefix selects the provider implementation. max_iter is set explicitly rather than left to a default that has shifted between releases. And the result is an object, not a string, so result.token_usage is available — which is the natural bridge into any cost question.

The classic traps, and why they happen

Trap: quoting 0.x behaviour. The scaffold is JSON-first (crew.jsonc, agents/*.jsonc) since 1.15.0, with --classic for the old layout. Memory was unified into one LanceDB-backed Memory class around 1.10, so the four kinds — short-term, long-term, entity, external — are gone, and crewai reset-memories -l/-s/-e are deprecated hidden flags that print a warning telling you to use -m. Tool-result caching defaults to off since 1.14.3, although the Crews documentation table still says True. CodeInterpreterTool was removed in 1.14.0, and allow_code_execution is a deprecated no-op whose warning points you at external sandboxes such as E2B or Modal. Each of these is a one-line answer that distinguishes you from a candidate working off memory.

Trap: the hidden OpenAI call. This is the question that catches most people. With no llm set, an agent resolves one from MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME, falling back to gpt-4.1-mini. Independently, crew planning (planning_llm), the memory system's analysis model and crewai test -m all default to gpt-5.4-mini in the 1.15.23 code, memory's default embedder is OpenAI's text-embedding-3-large, and knowledge's is text-embedding-3-small. So a crew where every agent is on Anthropic can still fail without OPENAI_API_KEY. The complete answer names every override: each agent's llm, planning_llm, manager_llm, Memory(llm=..., embedder=...) and the knowledge embedder.

Trap: confusing memory with knowledge. Knowledge is documents you supply, embedded into ChromaDB and queried automatically; its file paths resolve relative to ./knowledge/. Memory is what the system records as it runs, stored in LanceDB under ./.crewai/memory, filed in hierarchical scopes like /project/alpha, and recalled by a blend of semantic similarity, recency and importance. They have different default embedders, which is why changing one does not fix the other.

Trap: the forced final answer. If output ends with "You can't keep going, here is the best final answer you generated:", the agent hit max_iter and CrewAI made one last call demanding a best effort. The weak answer is "raise max_iter". The strong answer is that this nearly always means a tool is failing in a loop or the task is too broad for the agent to know when it is finished, so you fix the tool or tighten expected_output first.

Trap: calling kickoff from async code. Invoking the synchronous kickoff inside a running event loop — FastAPI, a notebook — raises an error telling you to use agent.kickoff_async() or crew.kickoff_async(). CrewAI has both native-async (akickoff, akickoff_for_each) and thread-based (kickoff_async, kickoff_for_each_async) variants.

Trap: structured output mistakes. Pass the Pydantic class, not an instance, and set either output_pydantic or output_json, never both — CrewAI refuses with Only one output type can be set, either output_pydantic or output_json.

Trap: hierarchical misconfiguration. A hierarchical crew needs manager_llm or manager_agent, and the manager must not also appear in the agents list. Both mistakes have their own explicit error messages.

Trap: ordering. A task may not list a later task in its context — CrewAI names the forward dependency and refuses. And a crew must end with at most one asynchronous task, so trailing async_execution=True tasks need a synchronous aggregator.

Common interview questions

"What is CrewAI, and when would you not use it?" Use the thirty-second answer, then add the boundary. It fits jobs that are genuinely multi-stage and benefit from role specialisation, where some variability in how the work gets done is acceptable. It is the wrong tool when the task is a single well-specified transformation — one model call with a good prompt is cheaper and more predictable — and it is the wrong tool when every step must be deterministic and auditable, in which case the orchestration should be ordinary code, with at most a single agent called from it.

"Crew or Flow?" If you can draw the steps as a diagram with conditions, it belongs in a Flow: deterministic routing, typed state, persistence, human approval. If the honest answer to "how many steps will this take?" is "depends what it finds", it belongs in a Crew. The official production guidance is Flow as the backbone calling crews from its steps, and the reason is debuggability: in a large hierarchical crew, your control flow is being decided by a language model.

"Sequential or hierarchical?" Sequential is the default and should be the default choice: the order is yours, each output feeds the next, and the run is reproducible in shape even if not in wording. Hierarchical adds a manager that decides delegation at run time, which costs extra manager LLM calls on every step and makes the run harder to predict. Reach for it when the right assignment genuinely is not knowable in advance. Mention that the manager must be supplied and must not be in the agents list.

"What makes a good agent definition?" That the four text fields are a specification rather than flavour. The goal names the decision the output feeds. The backstory carries the constraints and taste — what the agent refuses to guess, who it writes for — because it sits in the system prompt for every step. The expected_output names the format, the length and the required elements, so that two readers would produce nearly the same artefact from it. The usual failure is a vague expected_output, which has the most direct effect on usability and gets the least attention.

"How do you stop an agent producing garbage?" Four layers, cheapest first. A tight expected_output. Structured output with output_pydantic, so the shape is validated by Pydantic rather than hoped for. A guardrail — either a function taking one TaskOutput and returning a (bool, value) tuple, or a string that becomes an LLM-judged check — which retries up to guardrail_max_retries, default 3, feeding your failure message back to the agent. And a human: human_input=True on a task, or @human_feedback in a Flow for an application.

"How do you give an agent a tool, and what is the risk?" Subclass BaseTool with name, description, args_schema and _run, or use the @tool("Name") decorator where the docstring becomes the description. The import is from crewai.tools import BaseTool, tool — the old crewai_tools.BaseTool path is gone. Built-ins such as SerperDevTool and FileReadTool come from the crewai[tools] extra. The risk is that the model chooses the arguments, so every tool is an action an LLM can be talked into taking by text it read. Give each agent the narrowest tools that let it finish, and treat fetched content as untrusted data rather than instructions.

"How do you control cost?" Cost scales with agents × tasks × iterations × tool calls, because every agent step is a model call. Measure first: CrewOutput.token_usage, crew.usage_metrics, flow.usage_metrics. Then the levers: per-agent llm so cheap roles get cheap models, max_iter, max_execution_time, max_rpm, fewer agents, and switching features off that add calls — planning=True adds a planner call per iteration, memory adds analysis and embedding calls, hierarchical adds manager calls, and respect_context_window=True adds summarisation calls when a prompt overflows. Then hunt the hidden OpenAI calls listed above.

"How do you debug a crew that produced the wrong answer?" Start with verbose=True and read the log as a sequence of decisions: which agent, which tool, what arguments, what came back. Add output_log_file=True for a durable record. Use crewai log-tasks-outputs and crewai replay -t TASK_ID to iterate on the failing stage cheaply. If the problem is upstream, look at result.tasks_output to see exactly what the earlier task handed over, because the usual cause is a vague first task that the second task then faithfully built on. For anything running regularly, wire up tracing — CrewAI emits typed events on crewai_event_bus, has built-in opt-in tracing, and documents OpenTelemetry-based integrations with Langfuse, MLflow, Arize Phoenix and others.

"Where does CrewAI keep state, and why does that matter?" Flow persistence is SQLite by default, checkpoints are JSON files or SQLite, memory is LanceDB in ./.crewai/memory, and knowledge is ChromaDB in a per-OS application-data directory. All local files. In a container that means state disappears between pods unless you mount a volume or set CREWAI_STORAGE_DIR. It also means the framework has no tenant concept, so multi-tenant isolation is yours to build with separate storage paths or memory scopes.

"What changed recently that you watch for?" Name three concretely: the JSON-first scaffold in 1.15.0, unified memory around 1.10, and caching turning off by default in 1.14.3. Add that 1.x ships roughly weekly, defaults have changed inside the minor line, and the four packages — crewai, crewai-core, crewai-cli, crewai-tools — are version-locked, so you pin exactly in production and read the changelog each release.

What a strong answer adds

A weak answer describes features. A strong answer names a default, says where it comes from, and says what you do about it. "Memory uses an OpenAI embedder by default, so enabling it on an Anthropic-only project fails without an OpenAI key; I pass Memory(embedder=...) or set CREWAI_STORAGE_DIR and a local Ollama embedder when data must stay in-region" is a different class of answer from "CrewAI supports memory".

The second differentiator is admitting boundaries. The framework has no scheduler, no built-in code execution since 1.14.0, no multi-tenancy, and documentation that is in several places behind the shipped code. Saying so, with examples, reads as someone who has used it rather than someone who has read about it.

Sixty-second self-test

Answer out loud. Any hesitation marks a section to reread.

1. Name the four nouns and the required fields of each.
2. What are the two processes, and what does a hierarchical crew require that a sequential one does not?
3. What does crew.kickoff(inputs={...}) do besides start the run?
4. Where does an agent's model come from if you set no llm?
5. Name three features that make a hidden OpenAI call.
6. What is the difference between knowledge and memory, including where each is stored?
7. What does it mean when output ends with "You can't keep going, here is the best final answer you generated:"?
8. What is the import path for BaseTool, and what was it before?
9. Is tool-result caching on or off by default, and since when?
10. You are calling a crew from FastAPI. Which method do you call?
11. Two rules about output_pydantic.
12. What is the recommended production architecture, in one sentence?
13. Which crewai reset-memories flags are deprecated, and what replaces them?
14. How do you re-run only the final task of a crew?
15. Which Python versions are supported?

For the model: Agent/Task/Crew/Process with role-goal-backstory, description-expected_output, agents-tasks, and no required fields; hierarchical needs manager_llm or manager_agent and the manager stays out of agents; inputs interpolate {placeholders}; the model resolves MODEL → MODEL_NAME → OPENAI_MODEL_NAME → gpt-4.1-mini; planning, memory and crewai test plus the two default embedders; knowledge is supplied documents in ChromaDB under the app-data directory while memory is recorded facts in LanceDB under ./.crewai/memory; the forced final answer means max_iter was reached; from crewai.tools import BaseTool replaces crewai_tools.BaseTool; caching is off since 1.14.3; kickoff_async or akickoff; pass the class and never alongside output_json; a Flow backbone calling crews; -l/-s/-e are deprecated in favour of -m; crewai log-tasks-outputs then crewai replay -t; and Python 3.10 through 3.13, with 3.14 unsupported.

If you want to extend this into the next interview tier, the mid-level pane covers the executor loop in detail, event listeners, checkpointing, async patterns and deployment. The LangGraph guide is the comparison interviewers most often ask you to draw, and Langfuse is the answer to "how would you observe this in production".

How to use this page

These are the mistakes beginners actually make with CrewAI, in roughly the order they happen to you. Each one has a symptom you can recognise, the cause, and the fix. After that there are habits worth adopting on day one, and a checklist to run before you call a crew finished.

One framing note before the list. CrewAI's failures split cleanly into two kinds, and treating them the same is itself a mistake. Configuration failures throw an exception with a message that usually names the fix; they are annoying but cheap. Behavioural failures produce a plausible wrong answer with no error at all: the agent looped, skipped a tool, or answered a question you did not ask. There is no stack trace for those, only the verbose log and the task outputs. Most of the habits below exist to make behavioural failures visible.

The mistakes beginners make

1. Following a tutorial written for 0.x

Symptom. You copy an example and nothing matches: there is no src/<name>/crew.py, the import from crewai_tools import BaseTool fails, crewai reset-memories -l prints a deprecation warning, allow_code_execution=True does nothing.

Cause. CrewAI 1.0.0 only shipped in October 2025 and the 1.x line has changed defaults repeatedly since. The overwhelming majority of blog posts, videos and model-generated answers describe 0.x.

Fix. Date every piece of material you read against these markers. The scaffold is JSON-first (crew.jsonc plus agents/*.jsonc) since 1.15.0, with --classic for the old Python-and-YAML layout. Memory was unified into one class around 1.10. Tool caching turned off by default in 1.14.3. CodeInterpreterTool was removed in 1.14.0. The tool base class moved to crewai.tools. If a tutorial contradicts any of those, assume the rest of it is also stale and check the current docs instead.

2. Expecting crewai install to upgrade anything

Symptom. You ran uv tool install crewai --upgrade, crewai version reports the new release, and your project still behaves like the old one.

Cause. There are two separate installations: the global CLI in its own isolated environment, and the library inside the project's .venv. crewai install is a uv sync against the lockfile and never bumps a version.

Fix. Upgrade both, and keep them on the same release, because crewai, crewai-core, crewai-cli and crewai-tools are version-locked to one another.

uv tool install crewai --upgrade        # the CLI
uv add "crewai[tools]>=1.15.23"         # the project: pyproject.toml + uv.lock
crewai install
uv pip show crewai                      # confirm what the project actually has

3. Writing the model name without a provider prefix

Symptom. Unable to initialize LLM with model '<m>'. The model did not match any supported native provider ... and the LiteLLM fallback package is not installed.

Cause. CrewAI uses the prefix to select a provider implementation. Since 1.12.0 a set of providers is native — OpenAI, Anthropic/Claude, Azure, Google/Gemini, Bedrock/AWS, OpenRouter, DeepSeek, Ollama, vLLM via hosted_vllm, Cerebras, DashScope, Snowflake — and anything else needs the optional LiteLLM fallback.

Fix. Always write provider/model-id, as in openai/gpt-4.1-mini or anthropic/claude-sonnet-4-6. If your provider is genuinely outside the native list, uv add 'crewai[litellm]'. If it is in the list but you get Anthropic native provider not available, to install: uv add "crewai[anthropic]", install that provider's extra instead.

4. Being billed by a provider you never configured

Symptom. Every agent is on Anthropic, and the run fails with an OpenAI authentication error — or succeeds, and OpenAI invoices you.

Cause. Several things default to OpenAI independently of your agents. An agent with no llm resolves MODEL → MODEL_NAME → OPENAI_MODEL_NAME and then falls back to gpt-4.1-mini. Crew planning, the memory system's analysis model and crewai test -m default to gpt-5.4-mini in the 1.15.23 code. Memory's default embedder is OpenAI text-embedding-3-large; knowledge's is text-embedding-3-small.

Fix. If you are standardising on one provider, override every one of them: each agent's llm, plus planning_llm, manager_llm, Memory(llm=..., embedder=...) and the knowledge embedder. Then grep your project for agents with no llm.

5. Vague expected_output

Symptom. The crew runs cleanly and returns something shaped like a report that you cannot actually use — wrong length, no sources, prose where you wanted a table.

Cause. expected_output is what the agent is told to aim at. "A detailed report" is a wish, not a target.

Fix. Name the format, the length and the required elements, specifically enough that two people would produce nearly the same artefact.

6. Treating role, goal and backstory as decoration

Symptom. All your agents produce the same voice and the same generic output, regardless of which one you asked.

Cause. Those three fields are the system prompt. "You are an expert with many years of experience" tells the model nothing it does not already assume.

Fix. Put constraints in the backstory, because it is present for every step the agent takes: what it refuses to guess, what it distrusts, who it writes for, what it must always record. Put the downstream decision in the goal. If two agents' definitions could be swapped without changing their output, you have one agent, not two — and you are paying for two.

7. Hitting max_iter and raising it

Symptom. The output ends with "You can't keep going, here is the best final answer you generated:".

Cause. The agent exhausted its iteration budget, so CrewAI made one final call demanding a best effort. The default in 1.15.23 is 25.

Fix. Raising the limit is almost always the wrong first move. Read the verbose log and count the repeats: nine times out of ten a tool is failing and the agent keeps retrying it, or the task is so broad the agent cannot tell when it is done. Fix the tool, or tighten the task and its expected output. Raise max_iter only when the log shows genuine, varied progress that simply needed more steps.

8. Weak tool descriptions

Symptom. You tried to use the tool {tool}, but it doesn't exist. You must use one of the following tools, use one at time: {tools}. Or the agent ignores the tool you gave it. Or Error: the Action Input is not a valid key, value dictionary.

Cause. The description and the args_schema field descriptions are the only thing the model reads when deciding whether this tool applies and what to pass. A vague description produces guesses.

Fix. Write the description as an instruction with the shape of the arguments implied, and describe every field.

from crewai.tools import BaseTool
from pydantic import BaseModel, Field


class LookupInput(BaseModel):
    order_id: str = Field(..., description="Order ID, format ORD-12345")


class LookupTool(BaseTool):
    name: str = "Look up order"
    description: str = (
        "Fetch the status and shipping date of one order by its ID. "
        "Use when asked about a specific order. Returns 'not found' for unknown IDs."
    )
    args_schema: type[BaseModel] = LookupInput

    def _run(self, order_id: str) -> str:
        return "shipped 2026-10-01"

Also simplify the schema if arguments keep coming back malformed — flat scalar fields are far more reliable than nested objects.

9. Letting a tool raise

Symptom. I encountered an error while trying to use the tool. This was the error: {error}. followed by the agent trying again, and again.

Cause. The tool threw an exception. The agent sees an error string and has no idea whether to retry, change arguments or give up, so it usually retries until max_iter.

Fix. Handle expected conditions inside the tool and return a sentence the model can act on — "no order found with that ID; ask the user to confirm it" — or return a ToolFailure, which has failure policies (ignore, warn, raise) settable at the tool, task, agent or crew level.

10. Mixing up memory and knowledge

Symptom. You add documents and the agent ignores them. Or you turn on memory expecting it to read your files.

Cause. They are different features with different jobs and different storage.

Fix. For "the agent should know about our documents", use knowledge. For "it should remember what happened last time", use memory. And remember that knowledge file paths resolve relative to the project's knowledge/ directory, which is the most common cause of a file-not-found here.

11. Changing the embedder over an existing store

Symptom. A dimension-mismatch error, typically 1536 against 3072.

Cause. The vectors already in the store were produced by a different model. Legacy stores were 1536-dimensional; the current memory default is 3072.

Fix. Reset after any embedder change: crewai reset-memories -m for memory, -kn for knowledge, -a for everything. Or delete ./.crewai/memory and the knowledge directory outright.

12. Calling kickoff from async code

Symptom. Agent execution was invoked synchronously from within a running event loop. the first time you put a crew behind FastAPI or run one in a notebook cell.

Cause. The synchronous path cannot run inside an active event loop.

Fix. Use the async variants: await crew.akickoff(...) for native async, crew.kickoff_async(...) for the thread-based version, and the matching agent.kickoff_async(...). For batches, akickoff_for_each and kickoff_for_each_async.

13. Structured-output slips

Symptom. A Pydantic validation error, or Only one output type can be set, either output_pydantic or output_json.

Cause. You passed an instance (Brief(...)) instead of the class (Brief), or set both output types.

Fix. Pass the class, pick one output type, and read the result from result.pydantic or result.json_dict rather than parsing result.raw.

14. Task ordering that CrewAI refuses

Symptom. Task '<d>' has a context dependency on a future task '<d2>', which is not allowed. or The crew must end with at most one asynchronous task.

Cause. context may only reference earlier tasks — there are no forward references — and a crew cannot finish on a pile of concurrent async tasks with nothing to join them.

Fix. Reorder so dependencies point backwards, and end with a synchronous aggregating task that lists the async ones in its context.

15. Fighting output_file

Symptom. Path traversal attempts are not allowed in output_file paths, a shell-character refusal, or Directory <dir> does not exist and create_directory is False.

Cause. output_file is validated strictly as a security measure: no .., no ~, no $ or |.

Fix. Use a plain relative path such as output/report.md and pass create_directory=True.

16. Not noticing what a run costs

Symptom. A bill, or a latency complaint, that arrives as a surprise.

Cause. Every agent step is a model call, so cost scales with agents × tasks × iterations × tool calls. Options multiply it further: planning=True adds a planner call per iteration, hierarchical adds manager calls on every step, memory adds analysis and embedding calls, and respect_context_window=True adds summarisation calls when a prompt would overflow.

Fix. Read result.token_usage on your very first successful run and write the number down. Then use the levers deliberately: per-agent llm so cheap roles get cheap models, max_iter, max_execution_time, max_rpm to rate-limit, and fewer agents. Enable Crew(cache=True) only for genuinely idempotent tools, since caching is off by default now.

Habits worth adopting on day one

Keep verbose=True until the crew is boring. The log is the only account of what the agents actually did — which tools, which arguments, how many steps. Turn it off when the run stops surprising you, not before. Add output_log_file=True for a durable copy when you want to compare two runs.

Set max_iter explicitly on every agent. Not because 25 is wrong, but because the default has shifted between releases and the documentation disagrees with the code. An explicit number makes your runs reproducible across upgrades and makes the budget visible in the source.

Build the Flow first, even if it has one step. Starting with a Flow as the backbone and a crew inside it costs you five lines and saves the rewrite you would otherwise do when you need routing, persistence or an approval step. Putting control flow in a hierarchical crew means a language model decides it, and you cannot test that.

Write expected_output before description. Deciding what a finished answer looks like usually tells you what the task should ask for, and it stops you writing a description that quietly expects three different deliverables.

Reach for a solo agent to test prompt text. agent.kickoff("...") runs one agent with no task plumbing, so you can iterate on role, goal and backstory in seconds instead of paying for a whole crew per attempt.

Learn the replay loop early. crewai log-tasks-outputs then crewai replay -t TASK_ID re-runs from a chosen task onward. When you are tuning the last task of a four-task crew, this is the difference between a ten-second iteration and a two-minute one.

Check result.tasks_output when the final answer is wrong. The usual cause is not the last agent but the first: a vague opening task that every later task then faithfully built on. Looking at what was actually handed over finds that in one step.

Give each agent the narrowest tools that let it finish. The model chooses the arguments, so every tool is an action an LLM can be talked into taking by text it read from a web page or a document. Treat fetched content as untrusted data, never as instructions, and keep file and network tools scoped tightly. There is no built-in code execution since 1.14.0 — if you need it, use an external sandbox such as E2B or Modal, as the deprecation warning itself suggests.

Keep .env out of git and share_crew at False. The scaffold's .gitignore already excludes .env; leave it. share_crew=True sends goals, backstories, inputs and task outputs to CrewAI, which is rarely what you want with a client's data. If you work under data-residency requirements common with Gulf and Egyptian employers, note that enabling knowledge or memory sends your documents to a hosted embedding provider by default: either use a provider with a Middle East region, such as Azure OpenAI or Bedrock, or keep embedding local with Memory(embedder={"provider": "ollama", ...}).

Mount your storage if you containerise. Flow persistence, checkpoints, memory and knowledge are all local files by default. In a container they vanish between runs unless you mount a volume or set CREWAI_STORAGE_DIR.

Pin versions and read the changelog. 1.x ships roughly weekly and defaults have changed inside the minor line. Pin exact versions in production, commit uv.lock, and skim the changelog on each upgrade rather than discovering a changed default through behaviour.

The checklist

Run through this before you call a crew finished.

1. Every agent has an explicit llm with a provider/model-id string, and an explicit max_iter.
2. Every task's expected_output names a format, a length and the required elements.
3. No {placeholder} exists without a matching entry in inputs or a value passed at kickoff.
4. Task order is such that every context reference points backwards.
5. The deliverable task uses output_pydantic or output_json if anything downstream will parse it.
6. At least one guardrail protects the output you care about most.
7. If memory or knowledge is on, the embedder is set deliberately and the store has been reset since the last embedder change.
8. If anything standardises on a non-OpenAI provider, planning_llm, manager_llm, the memory LLM and both embedders are overridden too.
9. Tools are the narrowest set each agent needs, and no tool raises on an expected condition.
10. result.token_usage from a representative run is written down somewhere a budget conversation can find it.
11. .env is ignored by git, share_crew is False, and no key appears in a committed file.
12. The CLI and the project library report the same version.
13. If it is containerised, CREWAI_STORAGE_DIR or a mounted volume keeps state alive.
14. You have run it twice and compared the outputs, so you know how variable it is before a user does.

When this list is clean, you have done the beginner's job properly. The next tier — the executor loop in detail, event listeners, async and parallel execution, checkpointing, planning and deployment — is the mid-level pane. For the comparison interviewers like to draw, see LangGraph; for seeing inside a crew in production, Langfuse; and for knowing whether a change actually improved anything, RAGAS.

This is part two of three. It assumes you have built a crew that runs, and takes every topic from Beginner further: the loop inside an agent, the cost of each process, Flows as the structure that holds a real service together, and the operational work of running all of it on someone else's infrastructure.

Where this picks up

Everything here is verified against CrewAI 1.15.23 (28 September 2026), on Python 3.10–3.13. That version number matters more than it usually does. CrewAI 1.0 shipped in October 2025, and the 1.x line has changed defaults inside minor releases: tool caching, max_iter, the default model, the project scaffold, and the whole memory subsystem are all different from what most blog posts and most model training data describe. When this guide and something you read elsewhere disagree, check crewai version first.

Each section opens with the problem it solves and closes with a Try it you can run against a project you already have.

The executor loop, precisely

At Beginner level an agent is a role with a goal. In production it is a loop, and almost every surprising bill, timeout and truncated answer comes from that loop running more times than you expected.

When a task is given to an agent, CrewAI builds a prompt from the agent's role, goal and backstory, the task's description and expected_output, the outputs of any tasks listed in context, and the schemas of the available tools. It sends that prompt to the model. The model either returns a final answer, or asks to call a tool. If it calls a tool, CrewAI runs the tool, appends the result to the message history, and calls the model again. That cycle — model call, tool call, model call — is one iteration, and the component that runs it is the executor.

Two numbers bound the loop. max_iter caps the iterations, and its default in the 1.15.23 source is 25, not the 20 the Agents documentation page still prints. max_execution_time, in seconds, caps wall-clock time and defaults to None, meaning no limit at all. A stuck agent with the defaults will therefore make up to twenty-five model calls, with no deadline, before it gives up.

What happens when max_iter runs out is worth knowing by sight, because it is the single most common cause of a crew producing confident nonsense. CrewAI does not raise. It makes one more model call containing an instruction that begins "Now it's time you MUST give your absolute best final answer", and the output you receive is prefixed with:

You can't keep going, here is the best final answer you generated:

That string in your logs means the agent never finished its reasoning. The answer is whatever the model could assemble from a partial history. Treat it as a failure signal, not a result.

Three things cause the loop to run long. A tool that raises keeps being retried, because the model sees the error text and tries again with different arguments. A task whose expected_output is vague gives the model no stopping condition. And a hierarchical manager that keeps delegating adds iterations in a second loop above the first.

The other bound on the loop is context. respect_context_window defaults to True, and when the accumulated history exceeds the model's window CrewAI summarises it and continues, printing:

Context length exceeded. Summarizing content to fit the model context window. Might take a while...

That is a kindness with a cost: the summarisation is itself a model call, and the agent loses detail it may need. Set it to False only if you would rather fail than silently lose fidelity — then you get Context length exceeded and user opted not to summarize. and can route the work to a retrieval tool instead.

Set the limits explicitly rather than inheriting them:

from crewai import Agent, LLM

researcher = Agent(
    role="Market researcher",
    goal="Find and verify facts about {topic}",
    backstory="A careful analyst who cites sources and refuses to guess.",
    llm=LLM(model="openai/gpt-4.1-mini", temperature=0.2, timeout=120),
    max_iter=12,                 # the default is 25; 12 is plenty for a 2-tool agent
    max_execution_time=180,      # seconds; default is None, i.e. forever
    max_rpm=20,                  # per-agent rate limit
    max_retry_limit=2,
    respect_context_window=True,
    verbose=True,
)

verbose is a plain boolean in 1.x. Passing an integer no longer selects a log level, as it did in 0.x.

Processes, and what each one costs

A Crew is agents plus tasks plus a Process. There are exactly two: Process.sequential and Process.hierarchical. If you remember a "consensual" process from an old tutorial, it does not exist.

sequential is the default and the one to reach for. Tasks run in declaration order and each task's output becomes context for the next, so the data flow is the list order. Every task needs an agent, or you get Sequential process error: Agent is missing in the task with the following description: {description}.

hierarchical adds a manager that plans, delegates and validates. You must supply either manager_llm or manager_agent; omitting both raises Attribute manager_llmormanager_agent is required when using hierarchical process. And the manager must not also appear in agents — that raises Manager agent should not be included in agents list.

from crewai import Crew, Process

crew = Crew(
    agents=[researcher, writer],      # note: the manager is NOT in this list
    tasks=[research_task, write_task],
    process=Process.hierarchical,
    manager_llm="openai/gpt-5.4",
    verbose=True,
)

The cost difference is structural, not marginal. In a sequential crew the number of model calls is roughly the sum over tasks of the iterations each agent needs. In a hierarchical crew the manager makes its own calls to decide who does what, then more to check the result, and it frequently sends work back. Teams routinely find a hierarchical crew costs two to four times a sequential one for the same deliverable.

That last line is the mid-level insight. Most teams reach for hierarchical because they need branching, then pay an LLM to make a decision that an if statement could have made. A Flow @router gives you the branch for free and keeps the crews simple. Dynamic routing by a model is worth paying for only when the branch condition is genuinely linguistic.

Task wiring is where sequential crews get interesting. context takes a list of Task objects, not names, and the references must point backwards. A forward reference raises Task '<d>' has a context dependency on a future task '<d2>', which is not allowed. Async tasks have two more rules: The crew must end with at most one asynchronous task. and an async task cannot list an adjacent async task in its context. The practical pattern is a fan-out of async_execution=True tasks followed by one synchronous aggregator.

from crewai import Task

gulf = Task(description="Research {topic} in GCC markets", expected_output="Bullet points with sources",
            agent=researcher, async_execution=True)
egypt = Task(description="Research {topic} in Egypt", expected_output="Bullet points with sources",
             agent=researcher, async_execution=True)
summary = Task(description="Merge both regional briefs into one note",
               expected_output="A 400-word brief", agent=writer,
               context=[gulf, egypt])        # synchronous aggregator, runs last

Flows as the backbone

The official production guidance, and the one architectural decision that matters most at this level, is this: make a Flow the backbone and call crews or single agents from inside it. A crew is a good way to let a model decide the details of one step. It is a bad way to decide whether the step should happen at all.

A Flow is a Python class parameterised by a state model, with methods wired by decorators. @start() marks entry points, @listen(...) marks a method that runs after another, and @router(...) returns a string that other listeners can subscribe to. or_ and and_ combine triggers. The flow's result is the return value of whichever method completes last.

from pydantic import BaseModel
from crewai.flow import Flow, start, listen, router, or_

class BriefState(BaseModel):
    topic: str = ""
    depth: str = ""
    report: str = ""

class BriefFlow(Flow[BriefState]):
    @start()
    def classify(self):
        self.state.depth = "deep" if len(self.state.topic) > 10 else "shallow"

    @router(classify)
    def pick_path(self):
        return self.state.depth           # "deep" or "shallow"

    @listen("deep")
    def full_research(self):
        self.state.report = ResearchCrew().crew().kickoff(
            inputs={"topic": self.state.topic}).raw

    @listen("shallow")
    def quick_answer(self):
        self.state.report = summariser.kickoff(f"One paragraph on {self.state.topic}").raw

    @listen(or_(full_research, quick_answer))
    def finish(self):
        return self.state.report

result = BriefFlow().kickoff(inputs={"topic": "AI agents in Gulf banking"})

Three details repay attention. First, state is typed, so a typo in a field name fails at assignment rather than three steps later. Every flow state carries an auto-generated UUID id even when you define your own model. Second, kickoff is synchronous; kickoff_async is the awaitable form, and you need it anywhere there is already a running event loop. Third, flow.usage_metrics aggregates every model call in the whole flow, whereas the token_usage on a returned CrewOutput covers only that crew. If you report cost from the crew output inside a flow, you will under-report.

Human review belongs in the flow, not in a task. @human_feedback pauses a step for approval and records what came back in flow.human_feedback_history:

from crewai.flow import human_feedback

class ReviewFlow(Flow[BriefState]):
    @start()
    @human_feedback(message="Approve this brief before it goes to the client?")
    def draft(self):
        return self.state.report

Visualising a flow is genuinely useful once it has more than four nodes: flow.plot("my_flow") writes an interactive HTML graph, and crewai flow plot does the same from the CLI. crewai flow add-crew NAME scaffolds a crew under crews/ inside a flow project.

State, persistence and checkpoints

Flows have two durability mechanisms and they solve different problems. Confusing them is a common mid-level mistake.

@persist saves flow state. Apply it to the class or to a single method; the default backend is SQLiteFlowPersistence. With it in place, you can resume a run by passing its id, or fork a run into a new branch:

from crewai.flow import Flow, persist, start

@persist
class BriefFlow(Flow[BriefState]):
    @start()
    def begin(self): ...

BriefFlow().kickoff(inputs={"id": "0f6c…"})                   # resume that run
BriefFlow().kickoff(restore_from_state_id="0f6c…")            # fork; the copy gets a new id

For anything beyond a single container you implement FlowPersistence over Postgres or another shared store, because SQLite on a pod's local disk disappears with the pod.

Checkpoints, added in 1.14.0, save execution progress for crews, flows and agents. They are written automatically, by default after each completed task, into ./.checkpoints:

from crewai import Crew
from crewai.state import CheckpointConfig, SqliteProvider

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, write_task],
    checkpoint=CheckpointConfig(
        location="./.checkpoints",
        on_events=["task_completed"],
        provider=SqliteProvider(),      # or JsonProvider() for one file per checkpoint
        max_checkpoints=50,
    ),
)
crew.kickoff(from_checkpoint=CheckpointConfig(restore_from="./.checkpoints/<file>.json"))

Crew.from_checkpoint(path) rebuilds a crew from one, and Crew.fork(CheckpointConfig(restore_from=...), branch="experiment-a") branches a run so you can try a different prompt from the same midpoint — which is the cheapest A/B harness CrewAI gives you. The checkpoint field accepts True, False to stop inheritance, a CheckpointConfig, or None to inherit from the parent object.

There is a CLI for the files: crewai checkpoint opens a terminal UI, with list, info, resume, diff ID1 ID2 and prune [--keep N] [--older-than 7d] [--dry-run]. prune matters, because fifty checkpoints per run across many runs fills a disk quietly.

Structured output and guardrails

Free text is not an interface. The two mechanisms that turn a task into something a downstream system can consume are structured output and guardrails, and they compose.

Structured output is one field, not two. Set output_pydantic or output_json, never both — the second raises Only one output type can be set, either output_pydantic or output_json. And pass the class, not an instance; passing Article(...) gives you a Pydantic validation error that reads as though your model misbehaved.

from pydantic import BaseModel, Field
from crewai import Task

class Brief(BaseModel):
    headline: str
    sources: list[str] = Field(..., description="At least two absolute URLs")
    confidence: float

write = Task(
    description="Write a brief on {topic}",
    expected_output="A headline, the URLs you used, and a confidence between 0 and 1",
    agent=writer,
    output_pydantic=Brief,                 # the class
    output_file="output/brief.json",
    create_directory=True,
    markdown=False,
)

output_file is validated now. Paths containing .., ~, $, | or shell metacharacters are rejected with messages such as Path traversal attempts are not allowed in output_file paths. A missing directory raises Directory <dir> does not exist and create_directory is False, which is why create_directory=True is worth setting in containers.

Guardrails validate the output and, crucially, retry the task with the failure message as feedback. A function guardrail takes exactly one TaskOutput and returns a (bool, Any) tuple. A string guardrail becomes an LLMGuardrail judged by the task's own agent, so the task must have an agent assigned.

from typing import Any
from crewai import Task, TaskOutput

def has_two_sources(result: TaskOutput) -> tuple[bool, Any]:
    if result.pydantic and len(result.pydantic.sources) >= 2:
        return (True, result.pydantic)
    return (False, "Fewer than two sources. Add more and re-answer.")

def not_a_forced_answer(result: TaskOutput) -> tuple[bool, Any]:
    if "best final answer you generated" in result.raw:
        return (False, "The agent ran out of iterations. Answer concisely and stop.")
    return (True, result.raw)

write = Task(..., guardrails=[not_a_forced_answer, has_two_sources, "Under 400 words"],
             guardrail_max_retries=3)

guardrails run in order, so put the cheap deterministic checks before the LLM-judged ones. The retry budget is guardrail_max_retries, default 3. Task.max_retries still exists but is deprecated in favour of it. Common misuse produces clear errors: Guardrail function must accept exactly one parameter, Agent is required to use LLMGuardrail, Guardrail must be a callable or a string, and Task guardrail returned None as result. This is not allowed.

The budget arithmetic is worth doing once. A task whose agent has max_iter=12, with three guardrail retries, can make up to forty-eight model calls before it gives up. Multiply by the number of tasks and by kickoff_for_each inputs, and you have the worst case your rate limits and your budget must survive.

Tools you can trust

A tool is where your agent touches the real world, so its description is a prompt and its failure mode is a production incident. Import from crewai.tools — the old crewai_tools.BaseTool and crewai.agents.tools.tool paths are gone.

from crewai.tools import BaseTool
from pydantic import BaseModel, Field

class LookupInput(BaseModel):
    customer_id: str = Field(..., description="The internal customer id, e.g. CUS-4821")

class CustomerLookup(BaseTool):
    name: str = "Customer lookup"
    description: str = (
        "Return the billing record for one customer id. "
        "Use this instead of guessing. Returns an error string for unknown ids."
    )
    args_schema: type[BaseModel] = LookupInput

    def _run(self, customer_id: str) -> str:
        ...

Three things make the difference between a tool that works in a demo and one that works at scale. The description is read by the model and is the only thing that tells it when not to call the tool, so write it as instructions rather than as documentation. The args_schema field descriptions are read too, so a format example in a Field(description=...) prevents Error: the Action Input is not a valid key, value dictionary. And an unhandled exception inside _run becomes agent-visible text — I encountered an error while trying to use the tool. This was the error: {error}. followed by the accepted inputs — which the model will then try to work around, burning iterations. Return a ToolFailure instead, and set the failure policy (ignore, warn or raise) at the tool, task, agent or crew level depending on whether a dead tool should stop the run.

If you see You tried to use the tool {tool}, but it doesn't exist. You must use one of the following tools, use one at time: {tools}., the model hallucinated a name. That is almost always a sign of too many tools on one agent with overlapping descriptions. Splitting the work across two agents with three tools each outperforms one agent with six.

Caching changed, and it is the trap most likely to bite someone carrying 0.x habits. Crew.cache now defaults to False (since 1.14.3a1). Agent.cache=True only means the agent participates when crew-level caching is on. The Crews documentation table still prints True for the crew default; that is stale. Enable it deliberately, and only for idempotent tools:

crew = Crew(agents=[...], tasks=[...], cache=True)

For per-tool control, cache_function lets you decide from the arguments and result whether a particular call is cacheable — the right place to avoid caching a search whose results change hourly while still caching a currency conversion.

The other removal to know: CodeInterpreterTool is gone as of 1.14.0, and Agent.allow_code_execution and code_execution_mode are deprecated no-ops. The warning names the alternative: "Use dedicated sandbox services like E2B or Modal." The Tools documentation page still lists the removed tool. Built-ins that do exist in crewai[tools] include SerperDevTool, ScrapeWebsiteTool, FileReadTool, FileWriterTool, PDFSearchTool, CSVSearchTool, DirectoryReadTool, WebsiteSearchTool, FirecrawlScrapeWebsiteTool, ExaSearchTool and GithubSearchTool.

MCP servers as tools

The Model Context Protocol client is now a core dependency (mcp~=1.28.1), so attaching an MCP server needs no extra install. You pass servers to an agent through mcps=[...], either as strings or as typed server objects, and their tools appear to the agent alongside your own, prefixed per server so two servers cannot collide.

from crewai import Agent
from crewai.mcp import MCPServerStdio, MCPServerHTTP, create_static_tool_filter

analyst = Agent(
    role="Data analyst",
    goal="Answer questions about the warehouse",
    backstory="Reads data, never writes it.",
    mcps=[
        "https://api.weather.com/mcp#get_forecast",          # one tool from a URL
        MCPServerHTTP(url="https://mcp.internal/api",
                      headers={"Authorization": "Bearer …"},
                      streamable=True, cache_tools_list=True),
        MCPServerStdio(command="python", args=["server.py"],
                       tool_filter=create_static_tool_filter(
                           allowed_tool_names=["read_table"])),
    ],
)

The URL#tool form and create_static_tool_filter are the two levers that keep an MCP integration least-privilege. A server that exposes thirty tools, five of which can write, should reach your agent as the three read tools it actually needs. Schemas are cached for about five minutes and connections open lazily, so the first tool call in a run pays the connection cost; timeouts are in the ten-to-thirty-second range, which is long enough to matter against a tight max_execution_time.

The security position is explicit in the docs: trust the server before you use it. An MCP server's tool descriptions enter your prompt, and its results enter your agent's reasoning, so a hostile server is a prompt-injection vector with a function-call surface attached. For local SSE servers, validate Origin, do not bind to 0.0.0.0, and require authentication — DNS rebinding against a local server is a real attack. If you need manual control over connection lifetime rather than CrewAI's lazy management, MCPServerAdapter from crewai-tools[mcp] gives it to you. The MCP guide covers the protocol itself in more depth.

Alongside MCP, apps=[...] attaches CrewAI AMP platform integrations — Gmail, Slack, Salesforce, HubSpot, GitHub and others — which need CREWAI_PLATFORM_INTEGRATION_TOKEN in the environment. These are managed connectors rather than servers you run.

Memory and knowledge, and where the files live

These two subsystems get conflated constantly. Knowledge is retrieval over documents you supply. Memory is what the system remembers across runs. They use different stores, different default embedders, and different reset commands.

Knowledge attaches at agent or crew level through knowledge_sources, is embedded into ChromaDB, and is queried automatically when a task runs. Sources include String, TextFile, PDF, CSV, Excel, JSON, and CrewDocling for URLs (which needs crewai[docling]). File paths are resolved relative to ./knowledge/, which is the detail that produces most "file not found" reports.

from crewai import Crew
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

crew = Crew(
    agents=[analyst],
    tasks=[analysis_task],
    knowledge_sources=[StringKnowledgeSource(content="Our SLA is 4 hours in GCC regions.")],
    embedder={"provider": "openai", "config": {"model_name": "text-embedding-3-small"}},
)

Memory was rewritten. From around 1.10 there is one Memory class backed by LanceDB, replacing the separate short-term, long-term, entity and external/Mem0 memories. Memories live under hierarchical scopes, and recall ranks results by a composite of semantic similarity (weight 0.5), recency (0.3) and importance (0.2).

from crewai import Crew, Memory

mem = Memory(
    llm="gpt-5.4-mini",
    storage="lancedb",
    embedder={"provider": "ollama",
              "config": {"model_name": "mxbai-embed-large", "url": "http://localhost:11434"}},
    semantic_weight=0.5, recency_weight=0.3, importance_weight=0.2,
    recency_half_life_days=30,
)
mem.remember("The Riyadh tenant requires Arabic summaries", scope="/tenant/riyadh")
mem.recall("language requirements", limit=10, depth="deep", scope="/tenant/riyadh")

crew = Crew(agents=[...], tasks=[...], memory=mem)      # or memory=True for defaults

Memory also offers forget(scope=...), reset(), tree() and info("/"), plus read_only and root_scope fields that make per-tenant slicing practical. crewai memory opens a TUI over the store.

Now the operational part, which is where mid-level engineers get caught. All of this is local files by default.

CREWAI_STORAGE_DIR relocates the storage root. In a container that is not optional: without a mounted volume, memory and knowledge vanish on every restart, and with a carelessly shared volume two tenants read each other's memories. For a multi-tenant deployment, use a separate Memory per tenant with its own storage path or a root_scope="/tenant/<id>", never a process-global directory.

Also note that memory=True with no OpenAI key fails with Memory requires an embedder for vector search but initialization failed: ... and a hint about OPENAI_API_KEY. That is the first of several hidden OpenAI dependencies, which the next section is about.

Model routing and the calls you did not ask for

Every agent step is a model call, so cost and latency scale as agents × tasks × iterations × tool calls. That much is intuitive. What catches teams out is the set of model calls CrewAI makes on your behalf, each of which defaults to OpenAI.

When an agent has no llm, CrewAI resolves one from the environment in order — MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME — and falls back to gpt-4.1-mini. The documentation table that says "gpt-4" is stale. Separately, three auxiliary systems default to gpt-5.4-mini in the 1.15.23 source: crew planning (planning_llm), Memory(llm=...), and the model used by crewai test. The Planning, Memory and CLI documentation pages still say gpt-4o-mini. All three need OPENAI_API_KEY unless you override them.

So "we moved to Anthropic" is not done when every agent has llm="anthropic/claude-sonnet-4-6". You must also set planning_llm, Memory(llm=..., embedder=...), manager_llm on hierarchical crews, and the knowledge embedder. Miss one and you get an OpenAI authentication error from a component you did not know existed.

Providers are native first since 1.12.0. Native prefixes are openai, anthropic/claude, azure/azure_openai, google/gemini, bedrock/aws, openrouter, deepseek, ollama/ollama_chat, hosted_vllm, cerebras, dashscope and snowflake. Anything else goes through LiteLLM, which is now optional; without it you get a long error whose first line is Unable to initialize LLM with model '<m>'. The model did not match any supported native provider (openai, anthropic, claude, azure, ...), and the LiteLLM fallback package is not installed. The fix is in the message: uv add 'crewai[litellm]'. Native SDKs come from their own extras, and omitting one gives you, for example, Anthropic native provider not available, to install: uv add "crewai[anthropic]".

from crewai import LLM

cheap = LLM(model="openai/gpt-4.1-mini", temperature=0.1)
strong = LLM(model="anthropic/claude-sonnet-4-6", max_tokens=4000, timeout=120,
             thinking={"type": "enabled", "budget_tokens": 5000})
local = LLM(model="hosted_vllm/Qwen/Qwen2.5-7B-Instruct", base_url="http://vllm:8000/v1")

Always use the provider/model-id form; a bare model name is the most common cause of Invalid response from LLM call - None or empty., alongside a max_tokens set too low and provider content filters. LLM(model="openai/gpt-5.4", api="responses") selects the OpenAI Responses API, and reasoning_effort is available where the provider supports it. The 1.15.22 release added an llm_overlay context variable for routing agent roles to models without editing agent definitions, which is useful for running the same crew cheaply in CI and expensively in production.

For cost accounting, read CrewOutput.token_usage for one crew, crew.usage_metrics after a run, and flow.usage_metrics for a whole flow. Totals are prompt plus completion; Anthropic cache reads and writes are folded into prompt_tokens, so a cache-heavy workload looks prompt-heavy. If you front everything with a gateway, both LiteLLM proxy and Portkey work through base_url — see the LiteLLM guide for the proxy side.

Rate limiting is per agent or per crew via max_rpm, and since 1.15.23 the LLM layer retries throttled provider calls itself. For Gulf and Egyptian deployments with data-residency requirements, the native Bedrock and Azure providers are the usual route, because they let you pin a region; hosted_vllm against your own cluster is the option when nothing may leave your network at all.

Observability: events, tracing and logs

Underneath everything is a typed event bus. Every lifecycle step emits an event on crewai_event_bus, and a BaseEventListener subscribing to those events is how tracing, cost dashboards and third-party integrations all work.

from crewai.events import BaseEventListener, LLMCallCompletedEvent, ToolUsageErrorEvent

class CostListener(BaseEventListener):
    def setup_listeners(self, crewai_event_bus):
        @crewai_event_bus.on(LLMCallCompletedEvent)
        def on_llm(source, event):
            ...        # push tokens to Prometheus, or a row to your warehouse

        @crewai_event_bus.on(ToolUsageErrorEvent)
        def on_tool_error(source, event):
            ...        # alert: tool errors are the usual cause of iteration blowups

listener = CostListener()      # instantiate at import time so it stays referenced

That last line is not a style preference. If the listener is garbage-collected, its subscriptions stop firing and you get silence rather than an error. Instantiate it at module import in the same module that defines your crew.

The event families are Crew (CrewKickoffStarted, Completed, Failed), Agent (AgentExecutionStarted, Completed, Error), Task (TaskStarted, Completed, Failed), Tool (ToolUsageStarted, Finished, Error), LLM (LLMCallStarted, Completed, Failed, LLMStreamChunkEvent), Flow (FlowStarted, Finished, Failed) and Memory (MemorySaveStarted, Completed, Failed). Every class name ends in Event.

Built-in tracing exports spans to CrewAI AMP. Enable it with Crew(tracing=True), Flow(tracing=True), CREWAI_TRACING_ENABLED=true, or crewai traces enable. Spans buffer locally — 1,000 spans or 8 MiB by default, tunable with CREWAI_EPHEMERAL_TRACE_MAX_SPANS and CREWAI_EPHEMERAL_TRACE_MAX_BYTES — and upload only after an authenticated export or an explicitly approved anonymous one. Since 1.15.23, task spans carry the declared output format, which makes "the model ignored my schema" diagnosable from the trace alone.

Telemetry is a different thing: anonymous usage statistics, containing no prompts or secrets, on by default. Opt out with CREWAI_DISABLE_TELEMETRY=true for CrewAI only, or OTEL_SDK_DISABLED=true for all OpenTelemetry. It uses a private TracerProvider, so it will not interfere with your own OTel pipeline — relevant if you are exporting through the collector described in the OpenTelemetry guide.

For most teams the practical choice is a third-party backend. The docs cover Langfuse, Arize Phoenix, MLflow, OpenLIT, Langtrace, LangDB, Portkey, Opik, W&B Weave and Patronus AI, most of them via OpenTelemetry or OpenInference instrumentation. If you want per-run token cost, prompt diffs and user feedback in one place, Langfuse is the common pairing; if your organisation already standardises on MLflow for models, MLflow tracing keeps agents in the same UI.

Plain logging still earns its place. verbose=True prints the reasoning and tool calls, and output_log_file captures a run: True writes logs.txt, or pass a .txt or .json path. The JSON form is the one to ship to your log pipeline.

Testing a system that is not deterministic

You cannot assert on model output, but you can test almost everything around it, and the layering matters because the fast tests are the ones that will actually run on every commit.

The deterministic layer is large. Tool _run methods are plain functions: test them with ordinary unit tests and no model involved. Guardrail functions take a TaskOutput and return a tuple: construct one and assert. Flow routers are Python: call the method with a state and assert the returned string. A flow's wiring can be checked by running it with stubbed crews. Treat all of this as normal Python with normal coverage expectations.

def test_router_sends_long_topics_deep():
    flow = BriefFlow()
    flow.state.topic = "AI agents in Gulf banking"
    assert flow.pick_path() == "deep"

def test_guardrail_rejects_forced_answer():
    out = TaskOutput(description="d", raw="You can't keep going, here is the best final answer…",
                     agent="writer")
    ok, msg = not_a_forced_answer(out)
    assert ok is False

The non-deterministic layer needs different instruments. CrewAI ships two.

crewai test -n 3 -m gpt-5.4-mini runs the crew N times and scores the runs with a model. The code default model is gpt-5.4-mini — the CLI docs page still says gpt-4o-mini — and the CLI itself notes it is "For now only accepting only OpenAI models", so this is an OpenAI-billed operation regardless of what your agents use. Treat the score as a regression signal across prompt changes, not as an absolute measure of quality.

crewai train -n 5 -f trained_agents_data.pkl runs iterations and collects human feedback into a pickle that agents then load as suggestions. Five iterations and that filename are the defaults, and CREWAI_TRAINED_AGENTS_FILE relocates the file. It is a local artefact, which means it is also a thing you must deliberately ship or deliberately exclude from your image.

For re-running one step rather than the whole crew, crewai log-tasks-outputs lists the task IDs from the latest kickoff and crewai replay -t TASK_ID replays from that task onward using the stored outputs. This is the fastest loop available when you are iterating on the final task of a six-task crew.

Then there is evaluation proper. crewai eval [--run EXECUTION_ID] evaluates the last traced run through AMP, and — the detail that makes it useful — exits 0 only when the goal gate passes, so it works directly as a CI gate. For richer offline evaluation of retrieval-heavy crews, the usual pairing is a dataset-driven harness: see RAGAS for retrieval metrics and promptfoo for prompt-level assertions.

Packaging, configuration and deployment

A CrewAI project has two installs that drift apart if you let them: the global CLI and the project virtual environment. They are separate, and because crewai, crewai-core, crewai-cli and crewai-tools are version-locked, they must be on the same version.

uv tool install crewai --upgrade        # the global CLI
uv add "crewai[tools]>=1.15.23"         # the project: updates pyproject.toml AND uv.lock
crewai install                          # uv sync only; it never bumps versions
uv pip show crewai                      # verify what the project venv actually has

crewai install syncing without upgrading is the behaviour behind most "but I upgraded" confusion. If you hand-edit pyproject.toml, run uv lock --upgrade-package crewai first. crewai update migrates a Poetry-era pyproject.toml to uv, and crewai version checks for updates unless CREWAI_DISABLE_VERSION_CHECK is set.

Since 1.15.0 the default scaffold is JSON-first. crewai create crew NAME produces crew.jsonc, agents/*.jsonc, tools/, knowledge/, skills/, .env and pyproject.toml; --classic gives you the older src/<name>/crew.py plus config/agents.yaml and config/tasks.yaml layout. Both are supported, and the choice is a real one at this level: JSONC configuration is easy to template per environment and can be loaded with from crewai.project import load_crew, while the classic @CrewBase layout keeps everything in Python where your type checker and your tests can see it. A sensible rule is JSONC when a crew's shape is data your product changes, classic when it is code your team owns.

{ "name": "Research Crew",
  "agents": ["researcher", "analyst"],
  "tasks": [
    { "name": "research_task", "description": "Research {topic}", "expected_output": "Sourced bullets",
      "agent": "researcher" },
    { "name": "analysis_task", "description": "Turn the research into a brief", "expected_output": "400 words",
      "agent": "analyst", "context": ["research_task"],
      "output_file": "output/report.md", "markdown": true }
  ],
  "process": "sequential", "verbose": true, "memory": true,
  "inputs": { "topic": "AI in healthcare" } }

Names in agents must match file names under agents/, and each task's agent must match an agent name. crewai run prompts interactively for any {placeholder} with no default in inputs — fine locally, a hang in CI, so always pass --inputs or set defaults. A missing placeholder raises Missing required template variable '<var>' in description. One security note the docs make explicitly: declarative crews can execute Python through {"python": "module.attr"} and custom:<name> tool references, so run JSON crews only from sources you trust.

pyproject.toml must carry a [tool.crewai] section with type = "crew" or type = "flow". That is how crewai run and the deploy commands decide what to execute.

For serving, the two routes are your own container or AMP. In a container, the things to get right are the Python version (3.10–3.13; 3.14 is not supported), a committed uv.lock, CREWAI_STORAGE_DIR pointed at a mounted volume, and secrets injected as environment variables rather than baked into an image — the scaffold's .gitignore excludes .env for a reason. See the Docker and Kubernetes guides for the surrounding mechanics, and Vault for the secret injection.

Behind an API, remember that sync kickoff inside a running event loop raises:

Agent execution was invoked synchronously from within a running event loop. Use `agent.kickoff_async()` / `crew.kickoff_async()` (or `await agent.aexecute_task(...)`) when calling from async code.

So in FastAPI, or in a notebook, use the native async entry points: akickoff and akickoff_for_each are the true coroutines, while kickoff_async and kickoff_for_each_async are thread-based wrappers. Execution is in-process Python with no built-in distributed scheduler, so horizontal scale means many processes or containers behind a queue, with async_execution tasks and akickoff giving you concurrency inside one of them.

Streaming is a crew flag plus an iterator:

crew = Crew(agents=[...], tasks=[...], stream=True)
streaming = crew.kickoff(inputs={"topic": "AI"})
for chunk in streaming:
    print(chunk.content, end="", flush=True)
result = streaming.result

On a Flow, set the class attribute stream = True instead.

The managed route is AMP. crewai login uses a device-code flow, then crewai deploy create, deploy push, deploy status, deploy logs, deploy list and deploy remove manage a deployment, and crewai deploy validate checks it first. crewai enterprise configure <url> points the CLI at a self-hosted Factory instance. Each deployed crew gets a REST API: GET /inputs lists required inputs, POST /kickoff takes {"inputs": {...}, "meta": {...}} plus optional taskWebhookUrl, stepWebhookUrl and crewWebhookUrl, and returns {"kickoff_id": "<uuid>"}, which you then poll with GET /status/{kickoff_id}. Authentication is Authorization: Bearer <crew token>. From CI you can redeploy with a POST to https://app.crewai.com/crewai_plus/api/v1/crews/{UUID}/deploy using a personal access token — the pattern to wire into the workflow described in the GitHub Actions guide, with crewai eval as the gate before it.

Debugging beyond verbose

By now the failure modes sort into a handful of families, and each has a first move. The table below is the one worth keeping next to your terminal.

Three habits make all of these faster to resolve. First, run with verbose=True and output_log_file="logs/run.json" on every non-production run, so a failure always has a trace. Second, isolate: agent.kickoff("…") runs one agent standalone and returns a LiteAgentOutput with raw, pydantic, agent_role and usage_metrics, which is the quickest way to find out whether a problem is the agent, the task wiring or the crew. Third, when a crew misbehaves on one specific input, use crewai log-tasks-outputs and crewai replay -t TASK_ID rather than re-running the whole thing.

One more deprecation to watch for while reading old code, because it changes how you debug: CrewAgentExecutor is deprecated and agents now default to AgentExecutor (opt back in with executor_class). Agent internals such as AgentFinish are no longer public API — use event listeners or step_callback instead of reaching into them. Agent.reasoning and max_reasoning_attempts are deprecated in favour of planning_config=PlanningConfig(...), function_calling_llm is deprecated on Agent, and Agent.multimodal is marked for removal in v2.0 in favour of passing files natively. CLI flags moved to kebab-case too: --n-iterations, --task-id, --skip-provider, with the underscore spellings hidden and deprecated, and several commands were renamed — crewai create tool, crewai create skill, crewai create template, and crewai run in place of crewai flow kickoff.

Putting it all together

Here is the shape that combines everything above: a Flow backbone with a deterministic router, a bounded crew in one branch and a single agent in the other, typed state, persistence, structured output, guardrails, explicit limits, a cost listener, and no hidden OpenAI calls.

from typing import Any
from pydantic import BaseModel, Field
from crewai import Agent, Crew, LLM, Memory, Process, Task, TaskOutput
from crewai.events import BaseEventListener, LLMCallCompletedEvent
from crewai.flow import Flow, start, listen, router, or_, persist

CHEAP = LLM(model="anthropic/claude-sonnet-4-6", temperature=0.1, max_tokens=2000, timeout=90)
STRONG = LLM(model="anthropic/claude-sonnet-4-6", temperature=0.2, max_tokens=4000, timeout=120)

class Brief(BaseModel):
    headline: str
    body: str
    sources: list[str] = Field(..., description="At least two absolute URLs")

def needs_two_sources(result: TaskOutput) -> tuple[bool, Any]:
    if result.pydantic and len(result.pydantic.sources) >= 2:
        return (True, result.pydantic)
    return (False, "Fewer than two sources. Add more and answer again.")

def not_truncated(result: TaskOutput) -> tuple[bool, Any]:
    if "best final answer you generated" in result.raw:
        return (False, "You ran out of steps. Be concise and finish.")
    return (True, result.raw)

researcher = Agent(role="Researcher", goal="Find verifiable facts about {topic}",
                   backstory="Cites sources, never guesses.", llm=CHEAP,
                   max_iter=10, max_execution_time=180, max_rpm=20, verbose=True)
writer = Agent(role="Editor", goal="Turn research into a client brief",
               backstory="Writes tightly and keeps every citation.", llm=STRONG,
               max_iter=8, max_execution_time=120, verbose=True)

research_task = Task(description="Research {topic}. Prefer primary sources.",
                     expected_output="Bulleted findings with URLs", agent=researcher)
write_task = Task(description="Write the brief from the research",
                  expected_output="Headline, body, and the URLs used", agent=writer,
                  context=[research_task], output_pydantic=Brief,
                  guardrails=[not_truncated, needs_two_sources], guardrail_max_retries=2,
                  output_file="output/brief.json", create_directory=True)

MEMORY = Memory(llm="anthropic/claude-sonnet-4-6",
                embedder={"provider": "ollama",
                          "config": {"model_name": "mxbai-embed-large",
                                     "url": "http://localhost:11434"}},
                root_scope="/tenant/default")

def build_crew() -> Crew:
    return Crew(agents=[researcher, writer], tasks=[research_task, write_task],
                process=Process.sequential, memory=MEMORY, cache=False,
                output_log_file="logs/run.json", verbose=True)

class BriefState(BaseModel):
    topic: str = ""
    mode: str = ""
    brief: dict = {}

@persist
class BriefFlow(Flow[BriefState]):
    @start()
    def validate(self):
        if not self.state.topic.strip():
            raise ValueError("topic is required")
        self.state.mode = "full" if len(self.state.topic.split()) > 2 else "quick"

    @router(validate)
    def choose(self):
        return self.state.mode

    @listen("full")
    def deep_brief(self):
        out = build_crew().kickoff(inputs={"topic": self.state.topic})
        self.state.brief = out.pydantic.model_dump()

    @listen("quick")
    def short_brief(self):
        out = writer.kickoff(f"Two sentences on {self.state.topic}", response_format=Brief)
        self.state.brief = out.pydantic.model_dump()

    @listen(or_(deep_brief, short_brief))
    def done(self):
        return self.state.brief

class CostListener(BaseEventListener):
    def setup_listeners(self, crewai_event_bus):
        @crewai_event_bus.on(LLMCallCompletedEvent)
        def on_call(source, event):
            print("llm call completed")

listener = CostListener()

if __name__ == "__main__":
    flow = BriefFlow()
    print(flow.kickoff(inputs={"topic": "AI agents in Gulf banking"}))
    print(flow.usage_metrics)

Read it as a set of decisions rather than as code. The router is Python, so the branch is free and testable. Both branches are bounded: the crew by max_iter and max_execution_time on each agent, the single-agent path by being one call. Output is a Pydantic class, validated by guardrails that run cheapest-first, with a hard check for the forced-final-answer string. Memory is explicitly configured with a non-OpenAI embedder and a tenant scope, so nothing reaches a provider you did not choose. cache=False is written out even though it is the default, because the default changed and the next reader will wonder. Cost is read from flow.usage_metrics, which covers the whole run rather than the last crew. And @persist means a failure halfway through is resumable by id instead of being paid for twice.

Where you are now

You can predict what a CrewAI run will do before you pay for it. You know that an agent is a loop with two bounds, that the forced final answer is a failure disguised as a result, and that the hierarchical process buys dynamism with manager tokens you can usually avoid by routing in Python. You can lay a Flow down as the deterministic backbone, keep autonomy inside bounded steps, give the whole thing typed state, persistence and human review, and resume a crashed run from a checkpoint instead of re-running it.

You also know where the sharp operational edges are: every store is local files by default, every auxiliary subsystem defaults to OpenAI, caching is off unless you turn it on, the CLI and the project are separate installs that must match, and the documentation lags the code on several specific defaults — max_iter, Crew.cache, the default model, the test model, and the removal of the code interpreter.

What comes next is ownership rather than usage: multi-tenancy, RBAC and token scoping in AMP or Factory, a cost model per tenant, upgrade policy against a dependency set that moves weekly, and the trust model for tools and MCP servers when the people writing them are not you. The Senior level takes those on. In the meantime, two neighbouring guides are worth reading alongside this one: LangGraph for the graph-first alternative that CrewAI's own migration guide addresses, and Langfuse for making every one of these runs observable before you need to explain one.

Sources

• CrewAI documentation
• Changelog
• Upgrading CrewAI
• Agents
• Tasks
• Crews
• Processes
• Flows
• Tools
• Memory
• Knowledge
• Planning
• Checkpointing
• Event listeners
• LLMs
• CLI reference
• MCP overview
• Observability overview
• Tracing
• Telemetry
• Deploy to AMP
• AMP API reference
• crewai on PyPI

Mid-level CrewAI interviews are rarely about syntax. The interviewer wants to know whether you can predict what a multi-agent run will cost, where it will break, and which parts of it you would refuse to let a model decide. Everything below is against CrewAI 1.15.23, and several answers turn on defaults that changed inside the 1.x line — knowing those is itself a signal.

The framing that scores

Before the questions, the posture. A strong mid-level candidate talks about bounds: iteration limits, retry budgets, timeouts, token totals, and which component is deterministic. A weak one talks about capabilities: how many agents, how smart the manager is, how autonomous the system feels. Interviewers who have run agents in production are listening for the first vocabulary.

Have four numbers ready for any crew you have built: how many model calls a typical run makes, what it costs, what the worst case is when every retry fires, and how long it takes at the 95th percentile. If you cannot answer those about your own work, that is the gap to close before the interview.

Scenario: "The crew works locally but produces nonsense in production"

Start by asking what "nonsense" looks like, then go straight to the forced final answer. When an agent hits max_iter CrewAI does not raise; it makes one last call and returns output prefixed with "You can't keep going, here is the best final answer you generated:". That string in a log means the agent never finished reasoning, and nothing in CrewOutput flags it.

So the first check is: grep the production logs for that string. If it is there, the next question is why the loop ran long, and the answer is usually a tool. A tool that raises returns its error text to the model, which tries different arguments, which burns iterations. Production has different network access, different credentials and different data than your laptop, so a tool that always succeeds locally can fail every call in production while the crew keeps "working".

The fix sequence is: fix the tool (return a ToolFailure rather than letting exceptions escape), add a guardrail that rejects the forced-answer string so the failure becomes visible, then set max_iter deliberately rather than inheriting the default of 25. Mention that the Agents docs page still says 20 — the 1.15.23 source says 25 — and that you set it explicitly for exactly that reason.

A weak answer blames the model and suggests a bigger one. A strong answer explains why a silent failure mode existed at all, and closes it.

Scenario: "The pipeline is slow. What do you check first?"

Latency in CrewAI is model calls in series, so the first thing to establish is how many there are. Read crew.usage_metrics after a run, or flow.usage_metrics for a whole flow, and compare against the floor: one call per task if nothing goes wrong. A crew with four tasks making sixty calls has a loop problem, not a model-speed problem.

Then walk the multipliers in order of size. Is the process hierarchical? A manager adds planning and validation calls on top of every agent's loop, and teams commonly measure two to four times the cost of the sequential equivalent. Is planning=True set on the crew? That adds a planner call per iteration. Is planning_config on agents set to reasoning_effort="high"? That adds observation calls after every step; "medium" only replans on failure, and "low" uses a heuristic with no observation calls at all. Is memory on? Memory adds LLM analysis on save and on deep recall, plus embedding calls.

Only after that do you look at the per-call latency: model choice, max_tokens, whether the provider is throttling you (1.15.23 retries throttled calls, which turns a rate-limit problem into a latency problem), and whether context-window summarisation is firing — "Context length exceeded. Summarizing content…" is an extra call each time it appears.

For parallelism, the honest answer is that execution is in-process Python with no distributed scheduler. Inside one process you get async_execution=True tasks, akickoff, and akickoff_for_each; beyond that you run more containers behind a queue. And note the constraint: a crew must end with at most one asynchronous task, so the pattern is a fan-out of async tasks plus a synchronous aggregator.

Scenario: "We switched from OpenAI to Anthropic and it still calls OpenAI"

This is the single most common CrewAI gotcha, and it is a good question because it rewards having read the source rather than the docs.

Setting llm on every agent is not enough. Four other components resolve their own model, and all of them default to OpenAI. Crew planning uses planning_llm, which defaults to gpt-5.4-mini in the code. Memory(llm=...) defaults to the same. crewai test -m defaults to the same, and the CLI notes it accepts only OpenAI models. The memory embedder defaults to OpenAI text-embedding-3-large, and the knowledge embedder to text-embedding-3-small. The Planning, Memory and CLI documentation pages still print gpt-4o-mini; that is stale.

On top of that, an agent with no llm at all resolves from the environment — MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME — and falls back to gpt-4.1-mini, not the "gpt-4" in the docs table.

So the answer is a checklist: every agent's llm, manager_llm on hierarchical crews, planning_llm, Memory(llm=…, embedder=…), and the knowledge embedder. Then prove it by unsetting OPENAI_API_KEY and running the full workload. Mention the 1.15.22 llm_overlay context variable as the clean way to route roles to models per environment without editing agent definitions.

Bonus credit: note that non-native providers need uv add 'crewai[litellm]' since 1.12.0, and that native providers each have their own extra, so switching to Anthropic also means crewai[anthropic] or you get "Anthropic native provider not available".

Scenario: "Memory is empty every time the pod restarts"

The mechanism is simple and the answer should be immediate: every CrewAI store is local files by default. Memory is LanceDB under ./.crewai/memory. Knowledge is ChromaDB in the per-OS app-data directory. Flow @persist is SQLite. Checkpoints are JSON or SQLite under ./.checkpoints. Training data is a pickle. None of that survives an ephemeral container.

The fix is CREWAI_STORAGE_DIR pointed at a mounted volume, or an explicit storage= path on Memory. The follow-up the interviewer is usually fishing for is the multi-tenant hazard: if two tenants share that directory, they read each other's memories. The isolation levers are a separate Memory instance per tenant, distinct storage paths, root_scope="/tenant/<id>", or MemoryScope and MemorySlice with read_only for a shared corpus. The OSS framework has no tenant concept, so this is your job, not the framework's.

Say the data-residency part out loud if the role is at a Gulf or Egyptian employer: knowing that memory, knowledge and traces are all local artefacts by default, and that share_crew=True sends goals, backstories, inputs and task outputs to CrewAI, is exactly the awareness a regulated team wants.

Scenario: "A tool keeps being called with the wrong arguments"

Two errors matter here. Error: the Action Input is not a valid key, value dictionary. means the model produced malformed arguments. You tried to use the tool {tool}, but it doesn't exist… means it invented a name.

For malformed arguments, the levers are the args_schema and its field descriptions. The model reads Field(description=...), so a format example there — "the internal customer id, e.g. CUS-4821" — fixes more cases than any prompt change. Flatten nested schemas; a two-level object is much harder for a model to emit reliably than three scalars.

For hallucinated names, the cause is almost always too many tools on one agent with overlapping descriptions. Two agents with three tools each outperform one agent with six, and the fix is architectural rather than textual. Also mention that every tool's schema is in the prompt on every iteration, so trimming tools is a cost control as well as an accuracy one — which is why create_static_tool_filter and the URL#tool form matter when attaching MCP servers.

Close with the operational point: wrap _run so no exception escapes, return a ToolFailure, and choose the failure policy (ignore, warn, raise) at the tool, task, agent or crew level according to whether a dead tool should stop the run.

Trade-off: crews or flows?

The answer the interviewer wants is that this is not symmetrical. The official production guidance is a Flow as the backbone with crews or single agents inside @listen steps, and the reasoning is that a Flow gives deterministic routing, typed state, persistence and human-in-the-loop, while a crew gives autonomy. You want autonomy inside bounded steps, not around them.

Concretely: use a @router when the branch condition is checkable in Python, and a crew when the work genuinely needs a model to decide the details. A single agent via agent.kickoff() is the right unit for a one-shot step — it returns a LiteAgentOutput and skips all crew overhead. Many teams reach for Process.hierarchical and end up paying a manager LLM to make a decision an if statement could have made.

The nuance worth adding: Flow state is typed, so a field typo fails at assignment instead of three steps later, and flow.usage_metrics covers the whole run while a CrewOutput.token_usage inside a flow covers only that crew. Reporting cost from the wrong one under-reports.

Trade-off: sequential or hierarchical?

Default to sequential. It is deterministic in order, auditable, and cheapest: each task's output becomes the next one's context, so the data flow is the list. Hierarchical requires manager_llm or manager_agent (omitting both raises an explicit error) and the manager must not appear in agents (which raises another).

Pick hierarchical only when the set of steps genuinely depends on the input, when peers need to delegate mid-run, and when you have budget for the manager's calls. Then say the thing that distinguishes a mid-level answer: before choosing it, check whether a Flow router gives you the same dynamism deterministically. If the branch is linguistic — "decide which of nine specialists should handle this complaint" — the manager earns its tokens. If it is structural, it does not.

Trade-off: JSONC crews or the classic Python layout?

Since 1.15.0, crewai create crew NAME scaffolds JSON-first: crew.jsonc, agents/*.jsonc, tools/, knowledge/, skills/. --classic gives the older src/<name>/crew.py with config/agents.yaml and config/tasks.yaml.

The trade-off is configuration versus code. JSONC is data: easy to template per environment, loadable with crewai.project.load_crew, and changeable without a deploy if your product treats crews as content. The classic @CrewBase layout keeps everything where your type checker, your tests and your reviewers can see it, which matters when a crew is a core product behaviour rather than a configurable one.

Add the security note, because it is the part people miss: declarative crews can execute Python via {"python": "module.attr"} and custom:<name> tool references, and the docs say to run JSON crews only from trusted sources. A crew definition submitted by a user is executable input.

Trade-off: caching on or off?

Crew.cache defaults to False as of 1.14.3a1, and Agent.cache=True only means the agent participates when crew caching is enabled. The Crews docs table still says the crew default is True; it is stale, and knowing that is a good signal in an interview.

Enable it for idempotent tools — a currency conversion, a fixed lookup table, a document fetch by immutable id. Leave it off for anything time-sensitive, and for the in-between cases use cache_function to decide per call from the arguments and result. The failure mode of over-caching is subtle and expensive: a crew that confidently reports last week's numbers.

Scenario: "The same input gives a different answer every run"

Do not promise determinism you cannot deliver, but do not shrug either. The right answer separates variance you can remove from variance you must manage.

Removable variance: temperature left at a provider default, a model id without a pinned version, tool results that change between runs, memory recalling different items because recency is weighted at 0.3, and context-window summarisation firing on long runs and discarding different detail each time. Pin the model id, set temperature=0 for extraction and classification steps, and decide deliberately whether memory should be read-only during evaluation.

Managed variance: the rest. The instruments are a frozen input set of five to ten representative cases, a comparison of guardrail pass rates and token totals rather than text, and output_pydantic so the shape is constant even when the words are not. crewai test -n 3 scores repeated runs with a model and is useful as a before-and-after signal across prompt changes, not as an absolute quality number.

The sentence that lands well: "I do not assert on model text. I assert on schema, on guardrail outcomes, and on cost, and I keep the inputs frozen so exactly one thing changes at a time."

What strong answers cover that weak ones miss

The deprecations that date a candidate

Interviewers notice when an answer describes software that no longer exists. The ones most likely to come up: memory is unified behind one Memory class on LanceDB, and crewai reset-memories -l/-s/-e are deprecated in favour of -m. CodeInterpreterTool was removed in 1.14.0, with allow_code_execution and code_execution_mode as deprecated no-ops — the warning points at E2B and Modal. Agent.reasoning and max_reasoning_attempts are replaced by planning_config=PlanningConfig(...). function_calling_llm is deprecated. Agent.multimodal is marked for removal in v2.0. Task.max_retries gives way to guardrail_max_retries, default 3. Tools import from crewai.tools, not crewai_tools. verbose is a plain bool. And crewai flow kickoff is now crewai run.

Questions worth asking back

Ask what bounds the system today: what max_iter is set to, whether anyone has measured worst-case retries, where memory lives in production, and what the gate is before a crew's output reaches a customer. Ask whether evaluation runs in CI — crewai eval exits 0 only on a passing goal gate, so it works as one — and what happens when a run fails halfway. The answers tell you how mature the team is, and asking them demonstrates the same bounded-systems thinking the rest of the interview was testing.

Sixty-second self-test

Say these out loud: the default max_iter and what happens when it is exhausted; the two processes and what the second one costs; the four components that default to OpenAI; where memory, knowledge and checkpoints are stored; the current default for Crew.cache; the signature of a function guardrail; the difference between kickoff_async and akickoff; and one sentence on why a Flow belongs around a crew rather than inside one. Any you stumble on is the section of the detailed guide to re-read before the interview.

These are the errors, habits and recipes that separate a CrewAI project someone can hand over from one that only its author can run. All of it is against CrewAI 1.15.23, where several defaults differ from both the 0.x tutorials and, in a few places, from the current documentation pages.

The errors that cost the most time

The silent truncation. An agent that exhausts max_iter does not raise. It returns output beginning "You can't keep going, here is the best final answer you generated:". Nothing in CrewOutput marks it. The symptom is a pipeline that degrades over weeks as a tool slowly stops working; the cause is that a failure was shaped like a success. Fix it once, globally, with a guardrail on every task that rejects that string, and set max_iter explicitly because the default is 25 in the 1.15.23 source while the docs page still says 20.

Assuming tool caching is on. Crew.cache has defaulted to False since 1.14.3a1, and Agent.cache=True only means the agent participates when crew caching is enabled. The Crews docs table still prints True. The symptom is a bill or a latency profile that does not match a colleague's expectation; the fix is to write cache=True or cache=False explicitly in every crew so no reader has to guess which era the code was written in.

The hidden OpenAI call. Four components resolve their own model and all default to OpenAI: planning_llm, Memory(llm=…), the model used by crewai test, and the embedders for memory and knowledge. An agent with no llm falls back to gpt-4.1-mini after checking MODEL, MODEL_NAME and OPENAI_MODEL_NAME. The symptom is an OpenAI authentication error in a project that "does not use OpenAI"; the fix is the checklist in the detailed guide, verified by unsetting the key and running the real workload.

The embedder swap over a live store. Memory embeds with text-embedding-3-large (3072 dimensions), knowledge with text-embedding-3-small, and legacy stores used 1536. Change the embedder and you get a dimension-mismatch error that reads like a corruption bug. Reset first: crewai reset-memories -m for memory, -kn for knowledge, or delete ./.crewai/memory. The old -l, -s and -e flags are deprecated and print "All memory is now unified."

State on a container's local disk. Memory (LanceDB), knowledge (ChromaDB in the per-OS app-data directory), flow @persist (SQLite), checkpoints (./.checkpoints) and training data (a pickle) are all local files. Without CREWAI_STORAGE_DIR and a mounted volume, every restart is a fresh start; with a carelessly shared volume, two tenants read each other's memories.

Sync kickoff inside an event loop. Calling crew.kickoff() from FastAPI or a notebook raises an error that names the fix: use kickoff_async or, for the native coroutines, akickoff and akickoff_for_each.

The forward context reference. Task.context takes Task objects and must point backwards. A forward reference raises Task '<d>' has a context dependency on a future task '<d2>', which is not allowed. Related: a crew must end with at most one asynchronous task, and an async task cannot list an adjacent async task in its context.

Both output types at once. Only one output type can be set, either output_pydantic or output_json. And pass the class, not an instance — output_pydantic=Brief, never Brief(...), which produces a Pydantic error that looks like the model's fault.

crewai run hanging in CI. It prompts interactively for any {placeholder} with no default in inputs. In a pipeline that is a hang, not a prompt. Pass --inputs '{"topic":"…"}' or set defaults.

Upgrading the wrong install. The global CLI and the project venv are separate. crewai install runs uv sync and never bumps versions. To upgrade a project: uv add "crewai[tools]>=1.15.23", then crewai install, then uv pip show crewai to verify. The global CLI is uv tool install crewai --upgrade, and because crewai, crewai-core, crewai-cli and crewai-tools are version-locked, both must match.

The practices that pay off most, in order of leverage

1. Put a Flow around every crewDeterministic routing, typed state, persistence and human review for the price of one class. The official production guidance is Flow as the backbone with crews inside @listen steps, and it is the change that makes everything else on this list possible.
2. Set every bound explicitlymax_iter, max_execution_time, max_rpm, max_retry_limit and guardrail_max_retries, written out even where you agree with the default. Then compute the worst case: iterations × guardrail retries × tasks × inputs. That number belongs in your runbook.
3. Make every task output a Pydantic classoutput_pydantic turns a task into an interface. It also gives guardrails something structural to check, and since 1.15.23 the declared output format appears on task spans in tracing, so a schema the model ignored is visible in the trace.
4. Write tool descriptions as instructionsThe description is read by the model and is the only thing telling it when not to call the tool. Two sentences: what it returns, and when to use something else. Field descriptions in args_schema get the format example.
5. Let no exception escape a toolAn unhandled error becomes agent-visible text, and the model burns iterations working around it. Return a ToolFailure and choose the failure policy at the tool, task, agent or crew level.
6. Keep a frozen input setFive to ten representative cases, run on every prompt change with a pinned model id. Compare guardrail pass rates and token totals, never text.
7. Instrument before you need itA BaseEventListener on LLMCallCompletedEvent and ToolUsageErrorEvent is twenty lines and answers most production questions. Instantiate it at module import so it is not garbage-collected.
8. Decide where state lives on day oneCREWAI_STORAGE_DIR, a volume, and a per-tenant root_scope or storage path. Retrofitting tenant isolation after launch means a migration.
9. Pin versions and commit the lockfileMinor releases land roughly weekly and change defaults inside 1.x. uv.lock is also required for AMP deploys.
10. Prefer a router to a managerIf the branch is checkable in Python, branch in Python. Reserve Process.hierarchical for genuinely linguistic routing, and measure the premium before you keep it.

Practice cards

Operational recipes

Catch the silent truncation everywhere

from typing import Any
from crewai import TaskOutput

FORCED = "best final answer you generated"

def not_truncated(result: TaskOutput) -> tuple[bool, Any]:
    if FORCED in result.raw:
        return (False, "You ran out of steps. Answer concisely and finish.")
    return (True, result.raw)

# Put it first in every task's guardrails list: cheap, deterministic, runs before any LLM judge.

A crew that touches no OpenAI endpoint

from crewai import Crew, Memory, Process

MEM = Memory(
    llm="anthropic/claude-sonnet-4-6",
    embedder={"provider": "ollama",
              "config": {"model_name": "mxbai-embed-large", "url": "http://localhost:11434"}},
    root_scope="/tenant/riyadh",
)

crew = Crew(
    agents=[researcher, writer],          # each with an explicit llm=
    tasks=[research_task, write_task],
    process=Process.sequential,
    memory=MEM,
    planning=False,                       # planning_llm would default to an OpenAI model
    cache=False,
    embedder={"provider": "ollama",
              "config": {"model_name": "mxbai-embed-large", "url": "http://localhost:11434"}},
    output_log_file="logs/run.json",
)

Containerising without losing state

FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir uv
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen
COPY . .
ENV CREWAI_STORAGE_DIR=/data/crewai \
    CREWAI_DISABLE_TELEMETRY=true
VOLUME ["/data/crewai"]
CMD ["uv", "run", "python", "main.py"]

Python 3.10–3.13 only; 3.14 is not supported. Commit uv.lock, keep .env out of the image, and inject provider keys as environment variables at run time.

A CI gate that is not a rubber stamp

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pipx install uv
      - run: uv sync --frozen
      - run: uv run pytest tests/unit          # tools, guardrails, routers: no model calls
      - run: uv run crewai deploy validate
      - run: uv run crewai eval                # exits 0 only on a passing goal gate
        env:
          CREWAI_TRACING_ENABLED: "true"

The first step is where the coverage is. Tool _run methods, guardrail functions and flow routers are ordinary Python; test them like it. crewai eval evaluates the last traced run through AMP and exits non-zero when the goal gate fails, which is what makes it usable as a gate rather than a report.

Per-tenant isolation that holds up

from crewai import Memory

def memory_for(tenant: str) -> Memory:
    return Memory(storage=f"/data/crewai/{tenant}/memory",
                  root_scope=f"/tenant/{tenant}")

Separate storage paths plus a root scope, never a shared process-global directory. For a corpus several tenants may read but none may write, a MemorySlice with read_only is the right shape. Keep share_crew=False — the default — because True sends goals, backstories, inputs and task outputs to CrewAI, which is rarely acceptable for regulated data in Gulf or Egyptian deployments.

Reading a slow run

crewai run --inputs '{"topic":"AI in healthcare"}'   # never rely on the interactive prompt in CI
crewai log-tasks-outputs                             # task IDs from the latest kickoff
crewai replay -t <TASK_ID>                           # re-run from one task onward
crewai checkpoint prune --keep 20 --older-than 7d --dry-run
crewai reset-memories -m                             # unified memory; -kn for knowledge, -a for all

Cost control, concretely

Cost in CrewAI is agents × tasks × iterations × tool calls, and the levers sit in that order of size. Removing an agent removes a whole loop. Removing a task removes its iterations. Tightening max_iter caps the tail. Trimming tool schemas shrinks every prompt in every iteration.

The multipliers people forget are the optional subsystems. A hierarchical manager adds planning and validation calls above every agent loop. Crew(planning=True) adds a planner call per iteration. PlanningConfig(reasoning_effort="high") runs full observation with refinement after each step; "medium" replans only when a step fails, and "low" uses a heuristic with no observation calls at all — so that one field is a three-way cost dial. Memory adds LLM analysis on save and on deep recall, plus embedding calls. Context-window summarisation, which respect_context_window=True performs by default, is an extra call each time it fires.

Measure with the right object. CrewOutput.token_usage covers one crew. crew.usage_metrics is available after a run and includes cached-token breakdowns. flow.usage_metrics aggregates every call in a flow, which is the number to report when a crew runs inside one. Anthropic cache reads and writes are folded into prompt_tokens, so a cache-heavy workload looks prompt-heavy; do not read that as a prompt-bloat problem.

Then route models by role rather than globally. Give extraction, classification and formatting steps a cheap model with temperature=0; reserve the expensive one for the step whose judgement you are actually paying for. The 1.15.22 llm_overlay context variable routes agent roles to models without editing agent definitions, which makes "cheap in CI, expensive in production" a configuration change. For local analysis work, Memory(llm="ollama/…") keeps the memory subsystem off a metered endpoint entirely.

Finally, set max_rpm even when you are nowhere near a provider limit. Since 1.15.23 the LLM layer retries throttled calls itself, which means throttling now shows up as mysterious latency rather than an error — a per-agent rate limit turns that back into something you can see.

Errors you will meet exactly once

These are the ones that stop a run cold with a message precise enough to fix immediately, which is why they rarely cost more than a few minutes the second time.

The message "Attribute manager_llm or manager_agent is required when using hierarchical process" and its sibling Manager agent should not be included in agents list. are the two hierarchical setup errors. The second surprises people: the manager is configuration, not a crew member, and listing it twice is the usual first attempt.

Sequential process error: Agent is missing in the task with the following description: {description} means a task has no agent. Its sibling, The task '<desc>' has no agent assigned, therefore it can't be executed directly…, appears when you call task.execute_sync() directly.

Missing required template variable '<var>' in description means a {placeholder} has no matching input. In a JSONC crew, give it a default under inputs; in Python, pass it in kickoff(inputs=…).

The 'id' field cannot be set by the user. means something passed id= to a Crew or Task, usually code that round-trips objects through a dict.

Path traversal attempts are not allowed in output_file paths, along with the shell-expansion and shell-special-character variants, come from the 1.14 hardening. Use a plain relative path. Directory <dir> does not exist and create_directory is False is the companion, and create_directory=True is worth setting by default in containers.

The guardrail family — Guardrail function must accept exactly one parameter, Agent is required to use LLMGuardrail, Guardrail must be a callable or a string, and Task guardrail returned None as result. This is not allowed. — all come down to one signature: (TaskOutput) -> tuple[bool, Any], with an agent assigned whenever the guardrail is a string.

Invalid response from LLM call - None or empty. is the broadest of them. Check the provider/model-id form first, because a bare model name is the most common cause; then max_tokens, then provider content filters, then throttling.

And Refusing to resolve callback path '<p>': set CREWAI_DESERIALIZE_CALLBACKS=1 to allow. appears when restoring a checkpoint that references callables. Set that variable only for checkpoint files you produced yourself — a checkpoint from elsewhere is executable input.

One habit above the others

If you adopt only one thing from this page, adopt the frozen input set. Ten representative inputs in a file, with the tokens and guardrail outcomes from the last run committed next to them, converts every prompt edit from a guess into a measurement. It costs a few cents a run, it catches regressions a reviewer never would, and it is the artefact that lets you argue for or against a model change with evidence rather than impressions.

Habits worth keeping

Read the changelog on every upgrade, not just the version number. Defaults have changed inside 1.x more than once, and the changelog is the only place that says so. Write the version you verified against into your README, because a reader's biggest risk with CrewAI is applying 0.x advice.

Treat tool descriptions as production prose and review them like code. They are read by a model on every iteration, they are the largest lever on both accuracy and cost, and a careless edit is a behaviour change with no test to catch it.

Prefer one strong agent over three weak ones. Each additional agent multiplies iterations and tool schemas, and most two-agent crews outperform the five-agent version of the same workflow at a third of the cost. Add an agent when you can name the model call it saves.

Keep the non-deterministic surface small and labelled. Every place a model decides something should be a bounded step with an explicit limit, a validated output and a visible failure mode. That is the whole discipline, and it is what the detailed guide was building towards.

Checklist before a crew goes to production

Every agent has an explicit llm, max_iter and max_execution_time. Every task has output_pydantic or a documented reason not to. Every task's guardrails include the forced-answer check. planning_llm, Memory(llm=…, embedder=…), manager_llm and the knowledge embedder are all set deliberately. cache is written out. CREWAI_STORAGE_DIR points at a volume, and tenants are scoped. uv.lock is committed and the CLI matches the project version. An event listener records token usage and tool errors. output_log_file is on. A frozen input set and its last results are in the repository. And someone other than the author has run it end to end from a clean checkout, because the difference between a project and a prototype is whether a second person can reproduce it without asking a question.

This is part three of three. It stops asking how to make a crew work and starts asking what happens when thirty other engineers run crews you are responsible for. Everything from Beginner and Mid is assumed: agents, tasks, crews, processes, flows, tools, knowledge, memory, guardrails, the CLI. What changes here is the question. Not "did the crew produce a good report" but "what can this agent reach, what does it cost per run, where does its state live when the pod is replaced, and who gets paged when it loops".

CrewAI 1.15.23 is a library, not a server. That single fact shapes every section below. There is no scheduler, no queue, no tenant table, no control plane in the open-source package. The control plane is CrewAI AMP, the commercial Agent Management Platform, or it is something you build out of a web framework, a job runner and your own storage. Platform ownership of CrewAI means owning that gap deliberately rather than discovering it during an incident.

Where this picks up

I start with architecture, because every operational property below (durability, cost attribution, isolation, recoverability) is decided by the shape you choose before you write the first agent.

The architecture you are actually operating

A CrewAI deployment has four layers, and confusing them is the most common cause of surprise in production.

The first layer is orchestration: the Python objects that decide what runs next. This is a Flow, a Crew with a Process, or a single Agent. It is ordinary in-process Python. crew.kickoff() blocks the calling thread until every task finishes, and a Flow advances by calling the methods wired with @start(), @listen(...) and @router(...). Nothing here is distributed. If the process dies mid-run, the run is gone unless you arranged persistence.

The third layer is tools and context: your BaseTool subclasses, the built-ins from crewai-tools, MCP servers attached with Agent(mcps=[...]), AMP integrations attached with Agent(apps=[...]), knowledge sources, and unified memory. This layer is where almost all of your risk lives, because it is the only layer that touches systems outside the process.

The fourth layer is state: Flow persistence, checkpoints, memory, knowledge and the latest-kickoff task outputs that crewai replay reads. Every one defaults to local files. Flow @persist writes SQLite; checkpoints write JSON files or a WAL-mode SQLite file under ./.checkpoints; unified memory writes LanceDB under ./.crewai/memory; knowledge writes ChromaDB under the per-OS application data directory (~/Library/Application Support/CrewAI/{project}/knowledge/ on macOS, ~/.local/share/CrewAI/{project}/knowledge/ on Linux), overridable with CREWAI_STORAGE_DIR.

The official production guidance is to use a Flow as the backbone and call Crews or single Agents from Flow steps. That is worth restating as an engineering argument rather than a recommendation. A Crew with Process.sequential gives you a fixed task order but no branching; a Crew with Process.hierarchical gives you branching, but the branch decision is made by a manager LLM, which means the control flow of your application is itself non-deterministic and re-derived on every run. A Flow moves the branching into @router(...) methods that are plain Python returning plain strings. You keep autonomy where it earns its cost, which is usually inside a single step that has to read messy input and produce structured output, and you keep the shape of the run in code that you can unit-test without a model.

@persist and checkpoints hang off Flows and Crews, so a backbone gives you a place to resume from, typed state means a Pydantic model you can validate and diff, and @human_feedback gives you an approval gate at a known point rather than human_input=True scattered across tasks.

Which component breaks first

Scaling CrewAI is not about throughput in requests per second. It is about a product: agents × tasks × iterations × tool calls. Every agent step is an LLM call, so that product is simultaneously your cost, your latency and your failure surface. The components break in a predictable order, and knowing the order tells you what to instrument before you need it.

Provider rate limits break first. A crew with five agents, each allowed 25 iterations, can issue well over a hundred requests in a burst. CrewAI 1.15.23 does retry throttled provider calls at the LLM layer, which converts a hard failure into a long tail of latency, which is worse to debug. The lever is max_rpm, available on both Agent and Crew, and it is the first thing to set when several crews share one provider account. The second lever is routing: per-agent llm, or the llm_overlay context variable added in 1.15.22, which routes agent roles to different models without editing the agent definitions.

The context window breaks second. Agent.respect_context_window defaults to True, which means that on overflow CrewAI summarises the conversation to fit and logs Context length exceeded. Summarizing content to fit the model context window. Might take a while.... That is a safety net, not a design. It costs extra calls, it silently drops detail, and the drop is invisible in the final output. If you see that line in production logs, the fix is upstream: smaller task scopes, knowledge or RAG tools instead of pasting documents into descriptions, and output_pydantic so intermediate outputs are compact and typed rather than prose.

The iteration ceiling breaks third. When an agent hits max_iter (default 25 in the 1.15.23 source, even though the Agents docs page still says 20 — set it explicitly and do not rely on either number), CrewAI makes one final call instructing the model to give its best answer, and the output ends with You can't keep going, here is the best final answer you generated:. That string is the single most valuable thing to alert on. It means a run cost its full budget and returned something the agent itself considered incomplete. Nine times out of ten the root cause is a tool that fails in a way the agent keeps retrying, so the fix is in the tool's error message, not in a larger ceiling.

Local state breaks fourth, and most expensively. LanceDB memory, ChromaDB knowledge, SQLite flow persistence and JSON checkpoints are all files. Run two replicas of the same deployment without thinking about it and you get one of two outcomes: each pod has its own empty store, so memory and replay are useless, or both pods share a volume and two writers contend over the same embedded database. Neither is a crash you will enjoy diagnosing. The decision has to be explicit: set CREWAI_STORAGE_DIR per instance and accept that state is per-pod, mount a volume and pin to a single writer, or implement the FlowPersistence interface against Postgres and a custom StorageBackend for memory so state lives in a real database.

The event loop breaks last, but loudly. Calling a synchronous kickoff from inside a running event loop raises immediately: Agent execution was invoked synchronously from within a running event loop. Use agent.kickoff_async() / crew.kickoff_async() (or await agent.aexecute_task(...)) when calling from async code. Under FastAPI or in a notebook, use akickoff or kickoff_async. For fan-out, akickoff_for_each is the native async form and kickoff_for_each_async is the thread-based one; pick one per codebase so your concurrency limits mean something.

from crewai import Agent, Crew, Process, Task

# Every number here is a budget, not a default worth inheriting.
analyst = Agent(
    role="Claims analyst",
    goal="Decide whether claim {claim_id} needs manual review",
    backstory="Ten years in motor insurance adjudication.",
    llm="openai/gpt-4.1-mini",
    max_iter=8,                  # explicit: the code default is 25
    max_rpm=20,                  # bounds burst against the provider
    max_execution_time=120,      # seconds; raises TimeoutError past this
    max_retry_limit=2,
    respect_context_window=True, # safety net, not a plan
    allow_delegation=False,      # delegation multiplies calls
)

crew = Crew(
    agents=[analyst],
    tasks=[Task(
        description="Review claim {claim_id} against policy rules.",
        expected_output="A JSON object with fields decision and reason.",
        agent=analyst,
    )],
    process=Process.sequential,
    cache=False,                 # the code default since 1.14.3; the docs table is stale
    max_rpm=30,
)

result = crew.kickoff(inputs={"claim_id": "C-18422"})
print(crew.usage_metrics)        # prompt, completion and cached-token totals

The trust model, stated plainly

The security posture of a CrewAI deployment comes down to one sentence: an agent can call any tool you give it, and the LLM decides the arguments. Everything else follows.

That means a tool is not a convenience, it is a capability grant. FileWriterTool on an agent is write access to the filesystem with the path chosen by a language model that has read a web page. A database tool is query access with the predicate chosen the same way. The useful mental model is the one you would use for an API key: what is the smallest set of operations this role needs, and what is the worst thing that set can do if the arguments are chosen adversarially?

The adversary is usually not a person typing into your product. It is prompt injection arriving through content: a scraped page, a PDF in a knowledge source, a tool's own output, an email pulled in through an AMP app. Model output is data, and so is model input, and the moment a tool result is appended to a conversation it becomes instructions as far as the model is concerned. The mitigations CrewAI actually gives you are: least-privilege tools, guardrails on task output, output_pydantic so a malformed or hijacked response fails validation instead of flowing downstream, and a human gate via human_input=True on a task or @human_feedback on a flow step for anything irreversible.

Several things that used to be in scope no longer are, and this matters for anyone carrying a threat model written in 2025. CodeInterpreterTool was removed in 1.14.0. Agent.allow_code_execution and code_execution_mode are deprecated no-ops, and the deprecation warning says plainly: "CodeInterpreterTool is no longer available. Use dedicated sandbox services like E2B or Modal." There is no in-process code execution to sandbox any more. If your agents need to run generated code, that is an external service with its own isolation, network policy and budget, and the integration is yours to review.

The hardening CrewAI does ship, added through 1.14 and 1.15, is worth knowing by name because it changes what you need to build yourself. There are SSRF guards, with checks pinned to redirect hops and peer IP addresses since 1.15.17, so a tool fetching a URL cannot be trivially redirected to a metadata endpoint. There are path-traversal protections, including a FileWriterTool fix in 1.11.1. output_file is validated: paths containing .., ~, $ or shell metacharacters are rejected with messages like Path traversal attempts are not allowed in output_file paths. And checkpoints refuse to deserialise callbacks unless you opt in, which produces Refusing to resolve callback path '<p>': set CREWAI_DESERIALIZE_CALLBACKS=1 to allow. Only enable this for trusted checkpoint data. That environment variable should never be set in a deployment that restores checkpoints it did not write.

MCP deserves its own paragraph because it is the fastest way to hand an agent capabilities you have not reviewed. The docs are blunt: always ensure you trust an MCP server before using it. Attaching one with Agent(mcps=["https://mcp.example.com/api"]) means the tool list, the tool descriptions and the schemas all come from a remote party, and tool schemas are cached for about five minutes, so a server can also change what it offers between runs. Use create_static_tool_filter(allowed_tool_names=[...]) to pin exactly which tools are exposed, prefer MCPServerHTTP with explicit auth headers over SSE where you can, and for SSE remember the DNS-rebinding risk: validate Origin, do not bind local servers to 0.0.0.0, and require authentication.

from typing import Any
from crewai.tools import BaseTool
from pydantic import BaseModel, Field

ALLOWED_TABLES = {"policies", "policy_terms"}

class LookupInput(BaseModel):
    table: str = Field(..., description="One of: policies, policy_terms")
    policy_id: str = Field(..., description="Policy identifier, format P-00000")

class PolicyLookupTool(BaseTool):
    name: str = "Policy lookup"
    description: str = (
        "Read one policy row by id. Only the policies and policy_terms "
        "tables are available. Returns JSON. Never accepts free-form SQL."
    )
    args_schema: type[BaseModel] = LookupInput

    def _run(self, table: str, policy_id: str) -> Any:
        # The LLM chose both arguments. Validate as if they came from the internet.
        if table not in ALLOWED_TABLES:
            return f"Refused: table '{table}' is not readable by this role."
        if not policy_id.startswith("P-") or not policy_id[2:].isdigit():
            return f"Refused: '{policy_id}' is not a policy id."
        return self._read_row(table, policy_id)

That is the shape to insist on in review: a narrow args_schema, an allow-list inside _run, a refusal string rather than an exception for expected bad input, and a description precise enough that the model does not have to guess. When you want a failure to stop the run rather than be argued with, CrewAI's ToolFailure return type carries ignore, warn and raise policies at tool, task, agent or crew level.

Secrets and the supply chain

Local development uses .env, loaded through python-dotenv, and the scaffold's .gitignore already excludes it. That is adequate for a laptop and nothing else. In a deployment, provider keys arrive as environment variables injected from your secret manager. The variables that matter beyond the obvious provider keys are CREWAI_PLATFORM_INTEGRATION_TOKEN for AMP apps, CREWAI_USER_PAT for user-scoped platform access, SERPER_API_KEY if you use SerperDevTool, and CREWAI_STORAGE_DIR which is not a secret but decides where state lands.

In AMP, secrets are deployment environment variables, and crewai deploy create transfers .env values securely as part of the first deploy. RBAC can restrict access at entity level, so a team member can run an automation without being able to read the LLM connection behind it. If you are self-hosted on Factory, you are injecting from Vault, OCI Vault, AWS Secrets Manager or whichever manager you already run; if your organisation has data-residency obligations in the Gulf or in Egypt, this is also the place where the choice of provider region is enforced, because the provider base_url and the region in your Bedrock or Azure configuration are just environment variables like any other. Routing agent traffic to a regional endpoint is a configuration decision, and it belongs in the same review as the key itself.

The subtler problem is hidden OpenAI calls. CrewAI resolves a missing llm from MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME, and finally falls back to gpt-4.1-mini. Three auxiliary paths also default to OpenAI models in the 1.15.23 code, specifically gpt-5.4-mini: crew planning via planning_llm, the unified Memory(llm=...) analysis model, and crewai test -m. The memory embedder defaults to text-embedding-3-large and the knowledge embedder to text-embedding-3-small. A team that believes it runs entirely on Bedrock can therefore still be calling OpenAI, and the symptom is an authentication error about OPENAI_API_KEY from a crew whose agents all name a different provider, or Memory requires an embedder for vector search but initialization failed: ... followed by a hint to set OPENAI_API_KEY for the default embedder. The fix is to set the model on every agent plus planning_llm, manager_llm, Memory(llm=..., embedder=...) and the knowledge embedder. For a regulated workload, that enumeration is a compliance checklist, not a tuning exercise.

Supply chain has three strands. The first is the package graph: crewai depends on lock-stepped crewai-core==1.15.23 and crewai-cli==1.15.23, with crewai-tools pinned to the same version through the tools extra. Pin exact versions in production and commit uv.lock; AMP deploys require the lockfile, and a missing one shows up as dependency errors or "Module not found" at deploy time.

The second strand is the tool and skill repository. crewai tool install HANDLE, crewai skill install REF and crewai create template pull code into your project from AMP, and crewai uv add is a uv wrapper that injects tool-repository credentials. Code installed this way runs with your agents' privileges. Treat a published tool exactly as you treat any third-party dependency: pinned, reviewed on upgrade, and ideally mirrored. crewai tool publish --private is the right default when your team shares internal tools.

The third strand is declarative crews. Since 1.15.0 the default scaffold is JSON-first: crew.jsonc plus agents/*.jsonc, loaded through crewai.project.load_crew(path). JSONC looks like configuration, but it is not inert. A definition can reference Python with {"python": "module.attr"} and custom tools with "custom:<name>", which loads tools/<name>.py. The documentation's own guidance is to run JSON crews only from trusted sources. The platform consequence is concrete: if you let other teams submit crew definitions to a shared runner, those definitions are code submissions and need code review, signing or a loader that rejects the python and custom: forms entirely.

Multi-tenancy, because the framework has none

The open-source framework has no tenant concept. There is no tenant_id on a Crew, no per-tenant quota, no isolation between two crews running in the same process. If you serve more than one team or more than one customer, you build that boundary, and you build it at three levels.

Isolate state. Unified memory is the sharpest edge, because it is designed to be shared. Memories live under hierarchical scopes like /project/alpha, and recall uses a composite score of semantic 0.5, recency 0.3 and importance 0.2. Three mechanisms give you separation: distinct storage paths so tenants never share a LanceDB file, root_scope="/tenant/<id>" so every write and read is confined to a subtree, and MemoryScope or MemorySlice with read_only=True for a shared corpus that tenants may read but not pollute. The same applies to knowledge: separate collections and separate storage directories. Never share one process-global CREWAI_STORAGE_DIR across tenants without scoping, because a leaked memory is a data-residency incident dressed up as a relevance bug.

Isolate credentials. The cleanest boundary in a library-based system is the process. A worker that holds tenant A's keys and only tenant A's keys cannot leak into tenant B, regardless of what the agent decides to do. That is a stronger guarantee than any in-process check, and it is why per-tenant worker pools beat a single pool with a tenant argument, even though it costs more pods.

Isolate identity and quota. Flow state IDs should be namespaced per tenant, and the cost telemetry from flow.usage_metrics should be tagged with the tenant before it leaves the process. Rate limits via max_rpm are per agent or crew, not per tenant, so if you need a tenant quota you enforce it at admission in your own queue, before the crew starts.

AMP supplies what the library does not. Organisations are first-class with crewai org list, crewai org current and crewai org switch ID. RBAC ships predefined Owner and Member roles plus custom roles, with feature-level and entity-level permissions over automations, environment variables, LLM connections and Git repositories. User Bearer Tokens give user-scoped API access alongside the org-level Bearer Token for server-to-server calls. Enterprise adds SSO, workload identity, PII redaction and policies. Deployment comes in three shapes: Cloud at app.crewai.com, Factory which is self-hosted AMP and which you point the CLI at with crewai enterprise configure <url>, and Hybrid. For organisations with residency constraints, Factory is the conversation to have, and it is worth having before the first deployment rather than after.

from pathlib import Path
from crewai import Crew, Memory

def build_crew(tenant_id: str, agents, tasks) -> Crew:
    """One Crew per tenant, with state that cannot reach another tenant."""
    root = Path("/var/lib/agents") / tenant_id
    memory = Memory(
        llm="openai/gpt-5.4-mini",
        storage=str(root / "memory"),   # a distinct LanceDB per tenant
        root_scope=f"/tenant/{tenant_id}",
        recency_half_life_days=30,
    )
    return Crew(
        agents=agents,
        tasks=tasks,
        memory=memory,
        cache=False,
        output_log_file=str(root / "logs.json"),
    )

Cost, measured rather than estimated

Agent systems have a cost shape that surprises people who come from ordinary services: the marginal cost per request is large, variable, and decided at runtime by a model. A crew that usually finishes in six LLM calls can take thirty when a tool starts failing. Budgeting therefore means two separate things, a measured unit cost and an enforced ceiling, and you need both.

Measurement comes from three places. CrewOutput.token_usage covers the crew that produced it. crew.usage_metrics holds totals after a run, including cached-token breakdowns, which matters for Anthropic models where cache read and write tokens fold into prompt_tokens. flow.usage_metrics aggregates every LLM call in the flow, which is the number you want when a flow calls several crews — kickoff().token_usage only covers the final crew, so billing per run on it undercounts. Export all three tagged with tenant, flow name and version.

Then there are the calls that do not appear in the obvious places. Hierarchical process adds a manager LLM call for planning and delegation at every step, often on a larger model than the workers. Crew(planning=True) adds a planner call before each iteration. PlanningConfig(reasoning_effort="high") runs full observation with refinement and early goal detection after each step, where "medium" replans only when a step fails and "low" uses a heuristic with no observation calls at all — the single biggest cost dial in agent-level planning. Memory adds LLM analysis calls on save and deep recall plus embedding calls on every write, delegation adds a round trip per handoff, and respect_context_window=True adds summarisation calls precisely when prompts are already large.

Enforcement is a different set of levers. max_iter bounds the worst case per agent. max_execution_time bounds wall clock per task and raises Task '<d>' execution timed out after <n> seconds. Consider increasing max_execution_time or optimizing the task. max_rpm bounds burst. guardrail_max_retries, which defaults to 3, bounds how many times a failed guardrail re-runs a task — and note that Task.max_retries is deprecated in its favour, so a design document using the old name is working from stale information. Multiply them together for a given crew and you have a hard ceiling in calls, which multiplied by your per-model price gives you the number to put in the design review.

The cheapest structural saving is model routing: a summariser, a formatter and a classifier are usually fine on a small model, while the agent making the consequential judgement is not. Do it per agent with llm=, or at runtime with the llm_overlay context variable introduced in 1.15.22, which routes agent roles to models without touching the definitions. A second saving is simply fewer agents, since the extra ones often spend their tokens restating each other's work.

Upgrades and migrations

CrewAI ships minor releases roughly weekly, and defaults have moved inside 1.x. The cache default flipped in 1.14.3, max_iter moved to 25, the default agent model is gpt-4.1-mini and the auxiliary defaults are gpt-5.4-mini, the scaffold became JSON-first in 1.15.0, and conversational flows were promoted to stable in 1.15.18. The upgrade guide itself warns that defaults shift between releases. That cadence means an upgrade policy is not optional hygiene; it is the difference between a quiet platform and one that changes behaviour under its users.

The mechanical part has one trap. The global CLI and the project environment are separate installs. crewai install only runs a uv sync against the existing lockfile and never bumps versions. So the procedure is:

# 1. The global CLI (isolated uv tool environment)
uv tool install crewai --upgrade
crewai version --tools

# 2. The project: this edits pyproject.toml AND uv.lock
uv add "crewai[tools]>=1.15.23"
crewai install
uv pip show crewai          # confirm the venv actually moved

If someone has edited pyproject.toml by hand, run uv lock --upgrade-package crewai before crewai install, or the lockfile and the manifest will disagree. For a project still carrying a Poetry-era pyproject.toml, crewai update migrates it to uv. Keep the CLI and the project on the same version, because crewai, crewai-core, crewai-cli and crewai-tools are version-locked to each other.

Python support is >=3.10, <3.14. Python 3.14 is not supported at 1.15.23, which is the kind of detail that blocks a base-image bump three weeks after someone promised it.

Two migrations deserve a runbook of their own. The first is the embedder change. Memory's default embedder is text-embedding-3-large at 3072 dimensions, while legacy stores used 1536. Change embedder and the existing vectors are unreadable; the error is a dimension mismatch. The fix is crewai reset-memories -m for memory and -kn for knowledge, or deleting ./.crewai/memory and the knowledge directory outright. Plan it as a data migration with a re-ingest step, not as a configuration tweak, and schedule it when losing accumulated memory is acceptable.

The second is the memory API unification from around 1.10. A single Memory class backed by LanceDB replaced short-term, long-term, entity and external or Mem0 memory. The old reset flags survive as deprecated hidden flags that print Warning: --long is deprecated. Use --memory (-m) instead. All memory is now unified. Any code, runbook or tutorial that distinguishes short-term from long-term memory is describing a version you are not running. There is also a crewai memory TUI, with --storage-path, --embedder-provider, --embedder-model and --embedder-config options, which is the fastest way to inspect what a tenant's store actually contains.

The remaining deprecations still work today with a warning, which is exactly why they rot quietly: function_calling_llm, Agent.multimodal, Agent.reasoning and max_reasoning_attempts (use planning_config=PlanningConfig(...)), CrewAgentExecutor (agents now default to AgentExecutor, opt back in via executor_class), Task.max_retries, and the old CLI spellings — kebab-case --n-iterations, --task-id, --skip-provider, plus crewai create tool, crewai create skill, crewai create template, and crewai run in place of crewai flow kickoff. Also, verbose is a plain boolean now; an old verbose=2 is merely truthy.

Observability you can run an incident from

The primitive underneath everything observable in CrewAI is the event bus. Every lifecycle step emits a typed event on crewai_event_bus, and a BaseEventListener subscribes to them. The families are Crew (CrewKickoffStartedEvent, CrewKickoffCompletedEvent, CrewKickoffFailedEvent), Agent (AgentExecutionStartedEvent, AgentExecutionCompletedEvent, AgentExecutionErrorEvent), Task (TaskStartedEvent, TaskCompletedEvent, TaskFailedEvent), Tool (ToolUsageStartedEvent, ToolUsageFinishedEvent, ToolUsageErrorEvent), LLM (LLMCallStartedEvent, LLMCallCompletedEvent, LLMCallFailedEvent, LLMStreamChunkEvent), Flow (FlowStartedEvent, FlowFinishedEvent, FlowFailedEvent) and Memory (MemorySaveStartedEvent and friends). This is the correct integration point for a platform, because it gives you metrics that are yours, in your pipeline, independent of any vendor.

from crewai.events import (
    BaseEventListener,
    LLMCallCompletedEvent,
    ToolUsageErrorEvent,
    TaskCompletedEvent,
)

class PlatformMetrics(BaseEventListener):
    """Four counters that answer most incident questions."""

    def setup_listeners(self, crewai_event_bus):
        @crewai_event_bus.on(LLMCallCompletedEvent)
        def on_llm(source, event):
            metrics.increment("crewai.llm.calls", tags=self._tags(source))

        @crewai_event_bus.on(ToolUsageErrorEvent)
        def on_tool_error(source, event):
            metrics.increment("crewai.tool.errors", tags=self._tags(source))

        @crewai_event_bus.on(TaskCompletedEvent)
        def on_task(source, event):
            metrics.histogram("crewai.task.duration", event.timestamp, tags=self._tags(source))

    def _tags(self, source):
        return {"crew": getattr(source, "name", "unknown")}

# Instantiate at module import so the listener stays alive for the process.
listener = PlatformMetrics()

That last comment is load-bearing. A listener that is created inside a function and garbage-collected stops receiving events, and the failure mode is silence rather than an error.

Built-in tracing sits on top of the same bus and exports spans to AMP. Enable it with Crew(tracing=True) or Flow(tracing=True), with CREWAI_TRACING_ENABLED=true, or with crewai traces enable; crewai traces status and crewai traces disable complete the set, and crewai env view shows the relevant environment variables. Spans are buffered locally, with defaults of 1,000 spans and 8 MiB, tunable with CREWAI_EPHEMERAL_TRACE_MAX_SPANS and CREWAI_EPHEMERAL_TRACE_MAX_BYTES. Those buffer limits are worth raising before you debug a long flow, because a trace that drops its tail is worse than no trace. Traces upload only after an authenticated export or an explicitly approved anonymous upload, and since 1.15.22 human-feedback and pause events appear in tracing too, with task spans carrying the declared output format since 1.15.23.

If you already run an observability stack, the documented integrations cover most of it: Langfuse, Arize Phoenix, MLflow, OpenLIT, Langtrace, LangDB, Portkey, Opik, W&B Weave and Patronus AI, most through OpenTelemetry or OpenInference instrumentation. Keeping CrewAI inside your existing pipeline is usually worth more than a richer vendor UI, because your on-call engineer should not have to learn a second tool at three in the morning. The neighbouring guides cover the two most common choices: Langfuse for LLM-native tracing and evaluation, and OpenTelemetry if you want the spans in the same backend as the rest of your services. Arize Phoenix and Opik are the other two with first-class CrewAI support.

Finally, crewai eval evaluates the last traced run through AMP and exits 0 only when the goal gate passes, which makes it usable as a CI gate directly. crewai eval --run EXECUTION_ID targets a specific run. Combined with crewai test -n 3 -m gpt-5.4-mini, which scores N iterations with an LLM, you have a quality gate that runs on a pull request — with the caveat that crewai test currently accepts only OpenAI models for the scoring model, so budget for it.

Incident playbooks

Incidents in agent systems look different from service incidents: the system is usually up, and the output is wrong, slow or expensive. These are the five you will actually see.

A run never finishes. Check for the forced final answer string first; if it is absent, the run is still looping. The usual causes are a tool that errors on every call (the agent sees I encountered an error while trying to use the tool... and retries), a hallucinated tool name producing You tried to use the tool {tool}, but it doesn't exist. You must use one of the following tools, use one at time: {tools}., or malformed arguments producing Error: the Action Input is not a valid key, value dictionary. The immediate mitigation is max_execution_time on the task so the run dies with a clear timeout instead of burning budget. The fix is in the tool: a narrower args_schema, a clearer description, and a stronger or function-calling-capable model.

Costs doubled overnight with no deploy. Look for a changed model mapping, a provider defaulting differently, or a knowledge or memory store that grew enough to change prompt sizes. Then look for summarisation: Context length exceeded. Summarizing content to fit the model context window. in the logs means every affected run is paying an extra pass. Compare crew.usage_metrics for the same inputs against last week's recorded baseline, which is why recording a baseline is part of the platform's job.

Empty or truncated output. Invalid response from LLM call - None or empty. means the provider returned no content. The realistic causes are a bad model id, max_tokens set too low, content filtering, or rate limiting that exhausted retries. Reproduce with verbose=True or a trace, check the model id against the provider's current catalogue, and raise max_tokens before blaming the prompt.

State vanished after a deploy. Memory is empty, replay finds nothing, a flow cannot resume. The cause is almost always that ./.crewai/memory, ./.checkpoints or the knowledge directory was inside the container image rather than on a volume, or that CREWAI_STORAGE_DIR differs between the old and new pods. Recovery depends on having checkpoints somewhere durable, which is a decision made weeks earlier.

A crew produced a decision nobody can explain. This is the governance incident, and the only answer is evidence: the trace, the fingerprints from SecurityConfig, the output_log_file and the task outputs. A decision consequential enough to prompt that question needed a @human_feedback gate or human_input=True before it took effect, and flow.human_feedback_history holds the approval record.

Recovery mechanics are worth rehearsing before you need them. Checkpointing has been available since 1.14.0 and applies to Crew, Flow and Agent:

from crewai import Crew
from crewai.state import CheckpointConfig, SqliteProvider

crew = Crew(
    agents=agents,
    tasks=tasks,
    checkpoint=CheckpointConfig(
        location="/var/lib/agents/checkpoints",   # a volume, not the image
        on_events=["task_completed"],
        provider=SqliteProvider(),               # single WAL file; JsonProvider writes per-run files
        max_checkpoints=50,
    ),
)

# Resume after a crash
crew.kickoff(from_checkpoint=CheckpointConfig(restore_from="/var/lib/agents/checkpoints/<file>.json"))

# Or branch from a known-good point to test a fix
experiment = Crew.fork(CheckpointConfig(restore_from="..."), branch="experiment-a")

crewai checkpoint [--location PATH] opens a TUI, with list, info, resume, diff ID1 ID2 and prune [--keep N] [--older-than 7d] [--dry-run] subcommands. prune with --dry-run belongs in a cron job on day one, because checkpoint directories grow without limit. For Flows, resuming an existing run is kickoff(inputs={"id": "<uuid>"}) and forking one is kickoff(restore_from_state_id="<uuid>"), which gives the fork a fresh id. For crews, crewai log-tasks-outputs lists task IDs from the latest kickoff and crewai replay -t TASK_ID re-runs from that task onward — useful for debugging, but remember it reads locally stored outputs, so it only works where those files survive.

Governance that survives an audit

Governance for agent systems is mostly about three questions: what was the agent allowed to do, what did it actually do, and who approved the parts that mattered. CrewAI gives you a partial answer to each, and the platform's job is to close the gaps.

For what it was allowed to do, the artefact is the crew definition under version control, reviewed against a per-tenant tool allow-list. A JSONC definition is genuinely good for this, because the diff of a pull request shows exactly which tools, apps and MCP servers an agent gained. For what it actually did, the artefacts are the event stream, the trace, output_log_file (set it to a .json path so it is machine-readable rather than the default logs.txt), and the fingerprints that SecurityConfig attaches to agents, crews and tasks. For who approved, the artefacts are @human_feedback gates and flow.human_feedback_history, or human_input=True on the specific task.

Two patterns make this practical. Make irreversibility the trigger for a gate rather than seniority or sensitivity: anything that sends an email, moves money, writes to a customer-facing system or deletes data gets a human in the loop. And use structured output as the audit record — output_pydantic with a model carrying the decision, the reason and the evidence gives you a row you can query, where raw prose gives you something you have to read. Guardrails enforce the shape: a function with the signature (TaskOutput) -> tuple[bool, Any], or a string that becomes an LLMGuardrail using the agent's own model, retried up to guardrail_max_retries times.

Retention is the part teams forget. Traces, checkpoints, memory and knowledge all accumulate content derived from inputs, and if those inputs contain personal data then so do the stores. A deletion request has to reach ./.crewai/memory and the ChromaDB knowledge directory, not just your primary database. Memory.forget(scope=...) and Memory.reset() exist for exactly this, and Memory.tree() and Memory.info("/") let you see what is there before you decide. Write that path down before you need it.

Where CrewAI stops

A senior owner has to be able to say when the tool is the wrong one. CrewAI is excellent at role-based collaboration over text, and it is a pleasant way to get a deterministic backbone with pockets of autonomy. It is not a general workflow engine, and four boundaries are worth naming.

It is not a distributed scheduler. There is no built-in queue, no cron, no retry-with-backoff across machines, no fan-out across workers. Parallelism inside a run means async_execution tasks, akickoff or kickoff_for_each_async; parallelism across runs means your own processes behind a queue. If the workload is really a DAG of data transformations with a few LLM calls in it, Prefect or Airflow is the backbone and CrewAI is a task inside it, not the other way round.

It is not a graph runtime with arbitrary cycles. Flows give you routers, or_ and and_ combinators, persistence and typed state, which covers a great deal. If your problem genuinely needs fine-grained cyclic graphs, interrupt-and-resume at arbitrary nodes, or channel-level state reducers, LangGraph is built for that shape, and the CrewAI docs include a migration guide in the other direction for teams moving the opposite way.

It is not a tenant-aware platform on its own. That was the whole of the multi-tenancy section. If you need quotas, RBAC and per-tenant isolation out of the box, that is AMP, Factory, or your own control plane — and choosing to build it yourself is a legitimate answer as long as it is a decision rather than an omission.

The honest summary: pick CrewAI when the work is genuinely collaborative reasoning over text, when role framing helps the quality of the output, and when a deterministic Flow around it gives you the control you need. Pick something else when the orchestration problem is bigger than the reasoning problem.

Running it as a platform for other teams

Everything above becomes a service when you package it. A paved road for CrewAI has six pieces, and none of them are exotic.

A template comes first. crewai create template NAME [-o DIR] and crewai template list exist for this, and the template is where your defaults live: explicit max_iter, max_rpm and max_execution_time; cache=False unless justified; tracing=True; output_log_file pointing at a JSON path; share_crew=False; the event listener already wired; CREWAI_STORAGE_DIR read from the environment; and [tool.crewai] in pyproject.toml with type = "crew" or type = "flow", which is what crewai run and deploy read to decide what to execute. A team that starts from your template inherits your operational posture without a conversation.

A tool catalogue comes second. Publish reviewed internal tools with crewai tool publish --private and let teams install them with crewai tool install HANDLE; do the same for skills, which are reusable instruction packages attachable to agents or crews via Crew.skills. The point is not convenience, it is that a reviewed tool is a reviewed privilege, and a catalogue turns security review into something that happens once per tool rather than once per crew.

A runner comes third: the worker image, per-tenant credentials, the storage volume and the concurrency limits. A gate comes fourth: crewai eval on a traced run plus crewai test in CI, the structured-output assertions, and a cost-ceiling check. A budget comes fifth: usage metrics per tenant and per flow, alerting on unit cost rather than total spend, because total spend rises for good reasons and unit cost almost never does. An escape hatch comes sixth, so the paved road is a default rather than a cage.

If you deploy to AMP, the operational surface is a short list of commands. crewai login uses a device-code flow, crewai deploy create, validate, push, status, logs, list and remove manage the deployment, and each deployed crew gets a REST API: GET /inputs to discover required inputs, POST /kickoff taking {"inputs": {...}, "meta": {...}, "taskWebhookUrl": "...", "stepWebhookUrl": "...", "crewWebhookUrl": "..."} and returning {"kickoff_id": "<uuid>"}, then GET /status/{kickoff_id} to poll. Authentication is Authorization: Bearer <crew token>, with an org-level Bearer Token for server-to-server calls and a User Bearer Token for scoped user access. To redeploy from your own CI, POST https://app.crewai.com/crewai_plus/api/v1/crews/{UUID}/deploy with a Personal Access Token. Triggers connect the whole thing to the outside world: crewai triggers list and crewai triggers run PATH, with the payload arriving in a flow's crewai_trigger_payload argument.

The review checklist

Use this on pull requests and design documents. It is ordered by how often it catches something real.

The complete picture

Put the three levels side by side and the series has a single argument. Beginner taught the nouns: agents with a role, goal and backstory; tasks with a description and expected output; a crew with a process; kickoff returning a CrewOutput. Mid taught the machinery: flows, tools, knowledge, unified memory, guardrails, structured output, the CLI, and enough of the execution model to predict what a crew will do. Senior takes each of those and asks the ownership question.

The nouns become interfaces: a task's expected_output is a contract, and output_pydantic is how you enforce it. Tools become privileges. Memory becomes stateful infrastructure with residency and retention obligations. The process choice becomes an architectural decision about where non-determinism is allowed to live. The CLI becomes a deployment surface with a REST API and an RBAC model behind it. And max_iter, which looked like a loop guard in Beginner, turns out to be the ceiling on what a single bad run can cost you.

What does not change across the three levels is the discipline underneath. Decide what the agent is allowed to do, bound what it can spend, make its output checkable by something that is not a language model, and keep a record of what happened. CrewAI gives you a concrete mechanism for each: tools and MCP filters, the limit attributes, guardrails and Pydantic schemas, and the event bus with tracing.

Where the series leaves you

You can now design a CrewAI deployment that another team can use without your help, defend its cost and its blast radius in a review, recover it after a crash, upgrade it without surprising its users, and say clearly when the problem calls for a different tool. That is the whole of the ownership role.

Where to go next depends on which boundary you keep hitting. If orchestration is the constraint, read the LangGraph guide for a different model of the same problem, and the Prefect guide for the case where CrewAI is a step rather than the backbone. If observability is the constraint, Langfuse is the shortest path to traces you can reason about and OpenTelemetry the path to keeping them with the rest of your services. If evaluation is the constraint, RAGAS and DeepEval cover scoring agent and retrieval output properly. If tooling is the constraint, the MCP guide goes deeper into the protocol your agents are already speaking. And if the constraint is the platform underneath, Kubernetes and Vault are where the per-tenant workers and the credentials in this guide actually get implemented.

Sources

• CrewAI documentation
• Changelog
• Installation
• Upgrading CrewAI
• CLI reference
• Agents
• Tasks
• Crews
• Flows
• Processes
• LLMs
• Memory
• Knowledge
• Tools
• Planning
• Checkpointing
• Event listeners
• MCP overview
• Observability overview
• Tracing
• Telemetry
• Enterprise introduction
• RBAC
• Deploy to AMP
• API reference
• Kickoff endpoint
• crewai on PyPI
• crewAIInc/crewAI on GitHub

A senior CrewAI interview is not a quiz about decorators. The interviewer wants to know whether you can be handed a multi-agent system that other teams depend on and keep it cheap, safe, recoverable and explainable. The questions below are the three shapes that actually come up — system design, incident response, and arguing a trade-off upward — with what a strong answer covers and what a weak one leaves out.

Where this picks up

You already have the Mid-level answers: what a Crew is, how a Flow routes, what a guardrail does, how kickoff returns a CrewOutput. What changes at this level is that every answer has to name a budget, a blast radius, a place state lives, and a way to find out what happened. If an answer does not contain at least two of those four, it reads as junior no matter how fluent it sounds.

One framing to internalise before anything else: CrewAI 1.15.23 is a library, not a platform. There is no scheduler, no queue, no tenant model and no control plane in the open-source package. The control plane is CrewAI AMP, or Factory if it has to be self-hosted, or something you build from a web framework, a worker pool and your own storage. Candidates who say this early sound like owners. Candidates who assume a server exists get caught two questions later.

System-design prompts

"Design a document-review service for a bank. Twelve internal teams will use it."

Start with the shape, not the agents. The backbone is a Flow, because the branching has to be deterministic and testable; crews or single agents live inside @listen steps where the work is genuinely open-ended. Say why: a @router returning "escalate" or "auto" is plain Python you can unit-test, whereas the same decision made by a hierarchical manager LLM is re-derived on every run and costs a model call to make.

Then the per-team boundary. Twelve teams means twelve tenants, and the framework has no tenant concept, so you draw the line at the process: one worker pool per tenant, holding only that tenant's provider credentials. That is a stronger guarantee than any in-process check, because an agent cannot leak a key it does not have. State follows the same line — a separate Memory per tenant with its own storage path and root_scope="/tenant/<id>", separate knowledge collections, and CREWAI_STORAGE_DIR set per instance rather than shared.

Then durability. Memory is LanceDB files, knowledge is ChromaDB files, flow persistence is SQLite and checkpoints are JSON or SQLite under ./.checkpoints. All local by default. In containers that means a volume or a custom backend: implement FlowPersistence against Postgres and a StorageBackend for memory if you need shared durable state. Say out loud what happens if you skip this — either every pod has an empty store, or two pods contend over one embedded database.

Then budget. Explicit max_iter (the 1.15.23 code default is 25, not the 20 the docs page still prints), max_execution_time per task, max_rpm per agent and crew, max_retry_limit, guardrail_max_retries. Multiply them for a worst-case call count and price it. A senior answer gives the interviewer a number.

Finally the gate. Banking means irreversibility matters, so anything that writes to a customer-facing system sits behind human_input=True on the task or @human_feedback on the flow step, and flow.human_feedback_history is the approval record. Output is output_pydantic with a decision, a reason and evidence, validated by a guardrail, so downstream code checks a schema instead of parsing prose.

What a weak answer misses: it designs five collaborating agents and never mentions where state lives, never sets a limit, and assumes memory=True is isolated.

"Our agents need to run code the model writes. How do you sandbox that inside CrewAI?"

This is a trap question, and the correct answer is that you do not. CodeInterpreterTool was removed in 1.14.0; Agent.allow_code_execution and code_execution_mode are deprecated no-ops, and the warning says to use dedicated sandbox services such as E2B or Modal. So the design is an external execution service with its own network policy, its own filesystem isolation and its own timeout, exposed to the agent as a narrow tool. The agent gets "run this snippet, get stdout back", not shell access.

Say the second half too: whatever the sandbox returns comes back into the conversation as text, so it is untrusted input. Validate it, cap its size, and never let it choose the next tool call unmediated.

"Where would you put CrewAI in an existing data platform?"

The answer that scores is the one that identifies which problem is bigger. If the workload is mostly a DAG of transformations with a few reasoning steps, the orchestrator is Prefect or Airflow and CrewAI is a task inside it. If the workload is mostly open-ended reasoning with some plumbing, CrewAI's own Flow is the backbone and the data tools become CrewAI tools. If the control flow needs fine-grained cycles and interrupt-and-resume at arbitrary nodes, say that LangGraph is built for that shape — knowing where your tool stops is a senior signal, not a disloyalty.

"Expose a crew to another team as a service. What does the interface look like?"

Two routes. Deploy to AMP and you get a REST API per crew: GET /inputs to discover required inputs, POST /kickoff taking {"inputs": {...}, "meta": {...}} plus optional taskWebhookUrl, stepWebhookUrl and crewWebhookUrl, returning {"kickoff_id": "<uuid>"}, then GET /status/{kickoff_id} to poll. Auth is Authorization: Bearer <crew token>, with an org-level Bearer Token for server-to-server calls and a User Bearer Token for scoped user access. Or build it yourself behind FastAPI, in which case the thing to get right is that crew.kickoff() is synchronous and raises inside a running event loop; you use akickoff or kickoff_async, and you put the run on a worker rather than in the request.

Either way, the interface is the inputs schema and the output schema, not the crew. Teams should never depend on which agents exist, because that is the part you will change.

"Other teams want to submit their own crew definitions to a shared runner. Design that."

The reflex answer is "great, the default scaffold is JSON-first since 1.15.0, so they just send us a crew.jsonc". That answer fails. A JSONC definition is not inert: it can reference Python with {"python": "module.attr"} and custom tools with "custom:<name>", which loads tools/<name>.py. The documentation's own guidance is to run JSON crews only from trusted sources. So a submitted definition is a code submission, and it needs the same handling: it arrives by pull request, it is reviewed against that tenant's tool allow-list, and the loader either rejects the python and custom: forms outright for untrusted input or the runner holds only that tenant's credentials.

Say what you gain by keeping it declarative anyway: the diff of a pull request shows exactly which tools, apps and MCP servers an agent gained, which makes the security review mechanical. That is a genuine advantage over reviewing a Python class, and it is the reason to keep the format rather than abandon it.

Finish with MCP, because it is the same problem one layer out. Agent(mcps=[...]) takes a remote server's tool list, descriptions and schemas on trust, and schemas are cached for about five minutes, so a server can change what it offers between runs. Pin the surface with create_static_tool_filter(allowed_tool_names=[...]), prefer MCPServerHTTP with explicit auth headers, and for SSE validate Origin and never bind a local server to 0.0.0.0.

Incident-response questions

"A crew has been running for forty minutes and has spent two hundred dollars. What do you do?"

Containment first: kill the run, then check whether max_execution_time was set at all. If it was not, that is the first fix, because a timeout turns an open-ended spend into a bounded one.

Diagnosis second, and name the evidence. Look for the forced-final-answer string — output ending You can't keep going, here is the best final answer you generated: means max_iter was reached, so the agent looped rather than progressed. Look for Context length exceeded. Summarizing content to fit the model context window., which means every affected run is paying an extra pass. Look at the tool-error events on crewai_event_bus, because the usual root cause is a tool that fails identically every time and an agent that keeps retrying it. Two specific messages point straight at the tool layer: You tried to use the tool {tool}, but it doesn't exist. You must use one of the following tools, use one at time: {tools}. and Error: the Action Input is not a valid key, value dictionary.

Fix third, in the right place. A larger max_iter is not a fix. A clearer tool description, a narrower args_schema, a refusal string the agent can act on, or a stronger function-calling model is a fix. Then add the alert: count runs containing the forced-final-answer string, weekly.

"After a deploy, memory is empty and nobody can replay a run. What happened?"

Almost certainly storage. ./.crewai/memory, ./.checkpoints and the knowledge directory were inside the container image or on an ephemeral filesystem, or CREWAI_STORAGE_DIR differs between the old and new pods. The replay symptom is diagnostic on its own: crewai log-tasks-outputs and crewai replay -t TASK_ID read locally stored outputs from the latest kickoff, so they only work where those files survive.

The recovery answer separates owners from users. If checkpoints were written somewhere durable, you resume with crew.kickoff(from_checkpoint=CheckpointConfig(restore_from=...)), or for a flow with kickoff(inputs={"id": "<uuid>"}). If they were not, the runs are gone, and the honest statement is that the recovery objective was never actually set. Mention the TUI too — crewai checkpoint with list, info, resume, diff and prune [--keep N] [--older-than 7d] [--dry-run] — and that prune belongs in a cron job because checkpoint directories grow without limit.

"Costs doubled with no deploy. Where do you look?"

Three places, in order. First, hidden model calls: a missing llm resolves from MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME, and finally falls back to gpt-4.1-mini, while crew planning, the unified Memory(llm=...) and crewai test -m all default to gpt-5.4-mini in the 1.15.23 code. A crew everyone believes runs on Bedrock can still be calling OpenAI. Second, prompt growth: a knowledge or memory store that grew enough to change prompt sizes, visible as new summarisation lines in the logs. Third, a default that moved under you — minor releases ship roughly weekly and defaults have changed inside 1.x, cache and max_iter among them.

Then say how you would have caught it: crew.usage_metrics and flow.usage_metrics exported per tenant and per flow, with the alert on unit cost per run rather than total spend. Total spend rises for good reasons; unit cost almost never does.

"Someone reports that a crew leaked another customer's data into its answer."

Treat it as a data incident, not a quality bug. The most likely mechanism is shared unified memory: scopes are hierarchical, recall scores semantically with recency and importance weighting, and a single store with no root_scope will happily return another tenant's fact. Second most likely is a shared CREWAI_STORAGE_DIR. Third is Crew(share_crew=True), which transmits goals, backstories, task outputs and inputs to CrewAI and must stay off for anything containing customer data — distinct from telemetry, which is anonymous, carries no prompts and is on by default, with CREWAI_DISABLE_TELEMETRY=true or OTEL_SDK_DISABLED=true to opt out.

Containment is to stop the affected workers and separate the stores. Then the deletion path: Memory.forget(scope=...) and Memory.reset(), plus the knowledge directory, because a deletion request has to reach the vector stores and not only your primary database. In the Gulf and in Egypt that last sentence is often the whole conversation — residency obligations apply to embeddings as much as to rows, and Factory, the self-hosted AMP, is the answer worth naming when a cloud control plane is not acceptable.

Arguing a trade-off to leadership

Three arguments come up repeatedly, and the skill being tested is translating an engineering position into a business one without overclaiming.

"Why not let the manager agent decide everything? It's more autonomous." The business framing is predictability and cost. A hierarchical process puts your application's control flow inside a model, which means the same input can take a different path on two consecutive runs, and each decision costs a manager call on what is usually your most expensive model. A Flow moves that decision into code that is free to execute, reviewable in a pull request and testable without a provider key. You keep autonomy where it earns its cost — inside a step that reads messy input — and you spend nothing on autonomy you did not need. Offer the measurement rather than the opinion: run both for a week, compare flow.usage_metrics and the variance in output shape.

"The vendor platform costs money. Can't we self-host everything?" Be concrete about what you would be rebuilding. AMP supplies organisations, RBAC with predefined Owner and Member roles plus custom roles, entity-level permissions over automations, environment variables, LLM connections and repositories, User Bearer Tokens, deployment with secret transfer, traces and a tool repository. Self-hosting means building or substituting each of those, and Factory exists as the middle option — self-hosted AMP, pointed at with crewai enterprise configure <url>. Frame it as a build-versus-buy with a named list, and let leadership price the list. Do not frame it as a preference.

"Can we ship next week without the human-approval step?" The useful move is to narrow the ask rather than refuse it. Gate on irreversibility, not on importance: anything that sends, pays, writes to a customer system or deletes keeps the gate, and everything else ships now. That usually turns out to be a small fraction of the actions, which converts a blocking argument into a scoping decision. Then make the gate cheap — @human_feedback at one flow step with a clear message is a few lines, not a project.

Across all three, the pattern that reads as senior is the same: name the mechanism, name the number, and offer a measurement that would settle the disagreement. The pattern that reads as junior is naming a best practice.

Sixty-second self-test

Answer these out loud. Any hesitation marks the section of the detailed guide to reread.

Where does unified memory write by default, and what breaks when you run two replicas? What is the max_iter default in the 1.15.23 code, and why should you set it explicitly anyway? Which four model-calling paths default to OpenAI even when your agents name another provider? What does Crew.cache default to, and why does the docs table disagree? What exactly does CodeInterpreterTool do today? Which two log strings tell you a run wasted its budget? What does flow.usage_metrics include that kickoff().token_usage does not? What makes a crew.jsonc file code rather than configuration? Which command gates CI on a traced run, and what does it exit on? And what is the one-sentence reason to put a Flow in front of a hierarchical crew?

These are the things that do not show up on a laptop. A crew that works perfectly for one engineer develops a personality once thirty people run it on shared infrastructure with real credentials and a real invoice, and the failures are rarely crashes. They are silent cost, silent data mixing, silent quality drift and state that quietly disappears.

Where this picks up

Mid-level tips were about making crews reliable. These are about what breaks when you are not the only user, plus the habits that make you a tolerable owner and the checklists that catch the rest in review.

Production traps that only appear at scale

The cache default flipped and your cost model did not. Crew.cache has defaulted to False since 1.14.3a1. Agent.cache=True only means that agent participates when caching is enabled on the crew. The Crews documentation table still prints True, so a design written from the docs assumes repeated tool calls are free when they are not. Set cache explicitly, and only turn it on for genuinely idempotent tools — a search is, a ticket-creation call is not, and caching the second one will eventually create one ticket where the business expected five.

Four hidden paths call OpenAI even when your agents do not. An agent with no llm resolves from MODEL, then MODEL_NAME, then OPENAI_MODEL_NAME, and finally falls back to gpt-4.1-mini. Crew planning via planning_llm, the unified Memory(llm=...) analysis model and crewai test -m all default to gpt-5.4-mini in the 1.15.23 code even though the docs pages still say gpt-4o-mini. The memory embedder defaults to text-embedding-3-large and the knowledge embedder to text-embedding-3-small. The symptom is an authentication error about OPENAI_API_KEY from a crew whose agents all name Bedrock, or Memory requires an embedder for vector search but initialization failed: .... For a workload with residency obligations this is not a billing annoyance; it is traffic leaving a region you promised it would not.

Two replicas, one embedded database. Memory is LanceDB files under ./.crewai/memory, knowledge is ChromaDB under the per-OS app-data directory, flow persistence is SQLite and checkpoints are JSON or SQLite under ./.checkpoints. Scale a deployment to two pods without deciding this and you get either two empty stores or two writers on one file. Decide explicitly: per-instance CREWAI_STORAGE_DIR and accept per-pod state, a volume with a single writer, or a real backend through FlowPersistence and a custom StorageBackend.

Unified memory is designed to be shared, which is the problem. Scopes are hierarchical like /project/alpha, and recall scores semantically with recency and importance weighting (0.5, 0.3, 0.2). One store with no root_scope will return one tenant's fact to another tenant's agent, and it will look like a relevance quirk rather than a leak. Give each tenant its own storage path and root_scope="/tenant/<id>", and use MemoryScope or MemorySlice with read_only=True for shared corpora.

respect_context_window hides a growing problem. It defaults to True and summarises on overflow, logging Context length exceeded. Summarizing content to fit the model context window. Might take a while.... Detail is silently dropped and extra calls are silently paid for. Treat that log line as an alert, not as reassurance.

The forced final answer is your most useful signal and nobody watches it. When max_iter is reached, CrewAI makes one last call and the output ends You can't keep going, here is the best final answer you generated:. That means a run spent its whole budget and returned something the agent itself considered incomplete. Count it weekly. Also remember the default is 25 in the 1.15.23 source while the Agents docs page still says 20 — set it yourself so neither number matters.

Checkpoint directories grow forever. max_checkpoints on CheckpointConfig bounds one crew; it does not clean up across runs and branches. crewai checkpoint prune [--keep N] [--older-than 7d] [--dry-run] belongs in a cron job on day one, with --dry-run the first time.

Changing the embedder invalidates every store. Memory's default is 3072 dimensions; legacy stores used 1536. A switch produces a dimension-mismatch error and the only fixes are crewai reset-memories -m (and -kn for knowledge) or deleting the directories. Plan it as a data migration with a re-ingest step, scheduled when losing accumulated memory is acceptable.

crew.jsonc is code wearing a configuration costume. A definition can reference Python with {"python": "module.attr"} and custom tools with "custom:<name>", loading tools/<name>.py. The docs say to run JSON crews only from trusted sources. If a shared runner accepts submitted definitions, it is accepting code.

MCP tool surfaces drift under you. Schemas are cached for about five minutes and connections open lazily, so a remote server can change what it offers between runs. Pin with create_static_tool_filter(allowed_tool_names=[...]), prefer MCPServerHTTP with explicit auth headers, and for SSE validate Origin and never bind a local server to 0.0.0.0.

Sync kickoff inside an event loop raises, and people discover it in production. Under FastAPI or in a notebook you get Agent execution was invoked synchronously from within a running event loop. Use agent.kickoff_async() / crew.kickoff_async() .... Pick one concurrency form per codebase — native akickoff and akickoff_for_each, or thread-based kickoff_async and kickoff_for_each_async — so your limits mean something.

Delegation multiplies calls invisibly. allow_delegation defaults to False on an Agent, which is the right default, and every time someone flips it to True the worst-case call count for the crew changes without any other line of the diff showing it. A hierarchical process does the same thing structurally: the manager spends a call per step on planning, delegation and validation, usually on a larger model than the workers. Both are legitimate; both need the new ceiling written down.

Agent-level planning has a cost dial most people never read. planning_config=PlanningConfig(reasoning_effort=...) behaves very differently at each setting: "low" makes no observation calls and uses a heuristic, "medium" replans only when a step fails, and "high" runs full observation with refinement and early goal detection after every step. Nothing in the agent definition makes that obvious at a glance. Note also that the older Agent.reasoning and max_reasoning_attempts are deprecated in favour of this, so a crew still using them is working from a pattern that has moved.

Async task rules bite late. A crew must end with at most one asynchronous task, or you get The crew must end with at most one asynchronous task.; an async task cannot have an adjacent async task in its context; and context cannot reference a later task, which raises Task '<d>' has a context dependency on a future task '<d2>', which is not allowed. These all pass review easily and fail the first time someone reorders tasks to parallelise a slow step. Add a synchronous aggregator task at the end as a matter of habit.

output_file is validated, which surprises people writing dynamic paths. Paths containing .., ~, $ or shell metacharacters are refused with messages such as Path traversal attempts are not allowed in output_file paths, and a missing directory raises Directory <dir> does not exist and create_directory is False. Use a plain relative path with create_directory=True, and never interpolate model output into it.

Platform-ownership habits

Put a Flow in front of everything, even the simple cases. A @router is free to execute, reviewable in a diff and testable without a provider key. A hierarchical manager making the same decision costs a model call and re-derives your control flow on every run. Starting with a Flow also means @persist, checkpoints, @human_feedback and flow.usage_metrics are already available when you need them, rather than being a refactor.

Make every number explicit in the template, not in a wiki. max_iter, max_rpm, max_execution_time, max_retry_limit, guardrail_max_retries, cache, and a named model on every agent plus manager_llm, planning_llm, Memory(llm=...) and both embedders. Ship this as a crewai create template so teams inherit your posture without a conversation.

Instrument from the event bus, not from the vendor UI. A BaseEventListener on crewai_event_bus gives you four counters that answer most incident questions: LLM calls, tool errors, task durations and crew failures. Instantiate the listener at module import so it stays alive — a listener created inside a function is garbage-collected and stops receiving events, and the failure mode is silence rather than an error. Then layer built-in tracing on top with Crew(tracing=True) or crewai traces enable, and raise CREWAI_EPHEMERAL_TRACE_MAX_SPANS and CREWAI_EPHEMERAL_TRACE_MAX_BYTES above their 1,000-span and 8 MiB defaults before debugging a long flow.

Alert on unit cost, never on total spend. Total spend rises for good reasons. Cost per run rising is almost always a bug: a changed default, a grown store, a looping tool. Export crew.usage_metrics and flow.usage_metrics tagged with tenant, flow name and version. Use flow.usage_metrics for anything billed, because kickoff().token_usage only covers the final crew.

Gate on irreversibility, not on importance. Sends, payments, writes to customer systems and deletes get human_input=True or @human_feedback; everything else runs unattended. This is both the right security line and the argument that gets you shipped, because it is a small fraction of actions.

Make output checkable by something that is not a model. output_pydantic with the class — never an instance, and never alongside output_json, which raises Only one output type can be set, either output_pydantic or output_json. — plus a guardrail with the signature (TaskOutput) -> tuple[bool, Any]. A typed decision with a reason and evidence is an audit row; prose is something a human has to read.

Treat the tool catalogue as the security boundary. Publish reviewed internal tools with crewai tool publish --private and let teams install with crewai tool install HANDLE. Review once per tool instead of once per crew.

Keep the CLI and the project on the same version, deliberately. They are separate installs and crewai install never bumps anything: uv tool install crewai --upgrade for the CLI, then uv add "crewai[tools]>=1.15.23" and crewai install for the project, then uv pip show crewai to confirm. Pin exact versions, commit uv.lock (AMP deploys require it), log crewai.__version__ at worker boot, and set CREWAI_DISABLE_VERSION_CHECK in containers and CI so no build depends on an outbound call. Python support is >=3.10, <3.14.

Run a weekly upgrade canary. Minor releases ship roughly weekly and defaults have moved inside 1.x. A CI job that runs your smallest representative crew at the newest release against recorded inputs and diffs the structured output will catch a moved default before a user does. Pair it with crewai eval, which evaluates the last traced run through AMP and exits 0 only on a passing goal gate.

Keep a recorded baseline for every crew you own. A fixed input set, the structured output it produced, and the usage_metrics it cost, committed next to the code. Almost every diagnosis in this guide starts with "compare against last week", and that comparison is impossible unless somebody wrote the baseline down. It is also what makes the upgrade canary meaningful rather than decorative.

Rehearse recovery once per quarter. Kill a worker mid-run and resume from a checkpoint with no code changes: crew.kickoff(from_checkpoint=CheckpointConfig(restore_from=...)) for a crew, kickoff(inputs={"id": "<uuid>"}) for a flow, or kickoff(restore_from_state_id="<uuid>") to fork one with a fresh id. Time it. That number is the only honest answer to "what happens if a crew crashes", and it is usually much worse than people assume the first time.

Give each team an inputs schema and an output schema, not a crew. The agents are the part you will change, so if another team's code depends on which agents exist, you cannot refactor. On AMP, GET /inputs already makes the input contract discoverable; keep the output half just as explicit with a Pydantic model you version.

Write the per-tenant onboarding as a script, not a document. Creating a tenant means a storage directory, a Memory with its own root_scope, a credential set in your secret manager, a worker pool entry, a quota in the admission queue and a dashboard row. Six steps done by hand is six steps done inconsistently, and the inconsistent one is the one that leaks. If you are running the workers on Kubernetes with secrets from Vault, that script is a short manifest template and a policy, and it is the single highest-leverage thing you will write as the owner.

Read the changelog weekly, and keep a note of what moved. Releases ship roughly weekly, and the entries that matter to an owner are rarely features. Recent examples worth knowing: conversational flows became stable in 1.15.18, the llm_overlay context variable for routing agent roles to models arrived in 1.15.22, and 1.15.23 added provider-throttling retries at the LLM layer plus crewai eval against the last traced run. Each of those changes an operational assumption somebody on your platform holds.

Review checklist for a pull request

Review checklist for a design document

The through-line of all of it: decide what the agent may do, bound what it may spend, make its output checkable by something that is not a language model, and keep a record of what happened. Everything above is one of those four wearing a CrewAI-shaped hat.