The Complete Weaviate Guide

This is part one of three. It covers everything you need to start doing real work with Weaviate, not a teaser. By the end you can run a Weaviate server on your laptop, create a collection, load objects into it, search them by meaning, by keyword and by both at once, filter the results, generate answers from them, and read the common error messages without panic. Mid-level and Senior take the same topics further; nothing here is thrown away.

The guide targets Weaviate 1.39 (the stable line in August 2026, with patch releases continuing since) and the Python client v4. Weaviate also has clients for TypeScript, Go, Java and C#, and the ideas transfer directly, but every code sample here is Python.

Each section ends with a Try it task. Do them as you go. They take a few minutes each, and the ideas only stick once you have watched your own data go in and come back out.

What Weaviate is, and the problem it solves

Weaviate is an open-source vector database written in Go. It stores your data as objects, and next to each object it can store one or more vectors: long lists of numbers that capture what the object means. It then lets you search by meaning, by keyword, or by a blend of the two, and it can hand the results to a language model to produce an answer.

To see why that is useful, start with what came before it.

A traditional database is excellent at exact questions. "Give me the row where id = 42." "Give me every order placed after 1 March." If you want to find an article about getting a refund, a SQL LIKE '%refund%' query finds only the articles that contain that exact string. It misses the one titled "How to get your money back", even though a person would say it is the same topic. Keyword search engines improved on this with stemming and ranking, but they still match words, not ideas.

The change came from embedding models. An embedding model reads a piece of text (or an image, or audio) and returns a vector, for example a list of 768 floating-point numbers. The important property is that things with similar meaning get vectors that sit close together in that space. "Refund" and "money back" land near each other. "Refund" and "volcano" do not. Once your data is vectors, "find things similar in meaning to this query" becomes "find the vectors nearest to this query vector".

That nearest-neighbour question is expensive if you answer it naively. Comparing a query against ten million vectors one by one is far too slow. Vector databases exist to answer it fast, using an approximate nearest neighbour (ANN) index that trades a tiny amount of accuracy for a huge speed-up. They also do the unglamorous work around it: storing the original objects, filtering on ordinary fields, keeping data safe on disk, and scaling across machines.

Weaviate's own description is that it stores objects together with their vectors and serves vector, keyword (BM25), hybrid, filtered and generative search. The pieces you will meet in this guide are:

The most common reason people reach for Weaviate is retrieval-augmented generation (RAG): you want a language model to answer questions about your own documents. You cannot paste ten thousand pages into a prompt. Instead you store the pages in a vector database, search for the few passages most relevant to the question, and give only those to the model. Weaviate does the storing and searching, and can even call the model for you. Other uses are semantic site search, recommendations ("more like this"), deduplication, and searching images by a text description.

A note on where Weaviate sits among its neighbours. It is not the only vector database: Qdrant and Pinecone are comparable dedicated systems, and pgvector adds vector search to PostgreSQL. Weaviate's character is that it is a full database with a schema, built-in hybrid search and optional built-in integrations with embedding and generation providers.

The mental model: four nouns

Weaviate has more features than you will learn in a week, but the vocabulary for everything you do as a beginner is four words. Learn these and the rest of the documentation becomes readable.

Collection. A collection is a named set of objects that share a schema, much like a table in a relational database. You might have a collection called Article and another called Product. You will see the older word class in some documentation and in REST paths such as /v1/schema/{className}. It means the same thing as a collection. Collection names must start with an uppercase letter, so Article works and article does not.

Object. An object is one record in a collection. Each object has a UUID (a unique identifier, which Weaviate generates for you unless you supply one), a set of properties, zero or more vectors, and creation and update timestamps. If a collection is a table, an object is a row.

Property. A property is a typed field on an object, like a column. Common types are text, int, number (a floating-point number), boolean, date, uuid and arrays such as text[]. Dates use RFC 3339 format, for example 1985-04-12T23:20:50.52Z. Property names start with a lowercase letter, and Weaviate lowercases the first character for you.

Vector. A vector is the list of numbers that represents an object's meaning. Where it comes from is the most important choice you make, and there are two options. Either you configure a vectorizer (a model provider integration such as text2vec-openai or text2vec-ollama) and Weaviate calls the embedding model for you whenever you insert an object or run a text query. Or you bring your own vectors: you compute them yourself and pass them in. Weaviate calls that second option self-provided vectors.

Two further words appear constantly and are worth defining now.

A vector index is the data structure that makes nearest-neighbour search fast. The default is HNSW (Hierarchical Navigable Small World), a graph held in memory. You do not need to understand its internals to begin. You only need to know it is what Weaviate builds so that a search over a million objects takes milliseconds, and that it lives in RAM, which is why memory is Weaviate's main cost at scale.

A distance metric defines "close". The default is cosine, which compares the direction of two vectors and ignores their length. Search results report a distance, and lower means closer. Newcomers often expect a "score" where higher is better, so get used to reading distance the other way round.

Weaviate talks to you through three interfaces, and knowing which does what prevents a lot of confusion later. REST on port 8080 handles creating collections, inserting and fetching objects, and health checks. gRPC on port 50051 is the fast path the v4 Python client uses for searching and batch imports, and the v4 client requires it. GraphQL at /v1/graphql is the older query interface. As a beginner using the Python client you will rarely touch GraphQL directly.

Installing Weaviate and checking the setup

Weaviate ships as a Linux container image and as a Go binary. The easiest path on every operating system is Docker, so this guide uses it. If Docker is new to you, read the Docker guide first; the beginner part is enough. On Linux you need Docker Engine and the Compose plugin. On macOS, Docker Desktop, Colima or OrbStack all work, and Apple Silicon is supported through the arm64 image. On Windows use Docker Desktop with the WSL 2 backend.

There are other routes you should know exist. Weaviate also runs on Kubernetes through a Helm chart (weaviate/weaviate), which is the production route covered at the Mid and Senior levels. Weaviate Cloud is the managed service, where you create a cluster in a browser and receive a URL and an API key. And there is an embedded mode for Python that starts a Weaviate process from inside your script. Embedded mode is experimental and works only on Linux and macOS, so it is not a good teaching path for a mixed class.

Start the server

Create a folder for the project and put this in a file named docker-compose.yml. It is a single-node setup with data stored in a named volume:

services:
  weaviate:
    command: [--host, 0.0.0.0, --port, '8080', --scheme, http]
    image: cr.weaviate.io/semitechnologies/weaviate:1.39.7
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      CLUSTER_HOSTNAME: 'node1'
volumes:
  weaviate_data:

Read it line by line, because every line is something you will eventually need to change. The image pins an exact version, which is good practice: 1.39.7 was the latest 1.39 patch when this guide was verified, and you should prefer the newest patch of the minor you choose. The two ports lines publish the REST port 8080 and the gRPC port 50051 to your machine. Publishing 50051 is not optional, because the Python client checks it at startup. The volumes entry stores data in a Docker volume mounted at /var/lib/weaviate, which is where Weaviate writes. AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true' lets you connect without a key, which is convenient on a laptop and wrong for anything reachable from a network. QUERY_DEFAULTS_LIMIT sets how many results a query returns when you do not say.

Start it:

docker compose up -d

If you only want a quick throwaway run without a compose file, this also works:

docker run -p 8080:8080 -p 50051:50051 cr.weaviate.io/semitechnologies/weaviate:1.39.7

Without a volume, the data disappears when that container is removed. That is fine for a first experiment and a surprise if you forget it.

Check that it is alive

Weaviate has no core command-line tool. You check it with ordinary HTTP requests. The first tells you the version and the enabled modules:

curl http://localhost:8080/v1/meta

You get a JSON document that includes the version string and a list of modules. The readiness probe is the one to use in scripts:

curl -i http://localhost:8080/v1/.well-known/ready

A 200 OK means the node is ready to serve. There is also /v1/.well-known/live, which returns 200 as soon as the process is alive, even if it is still starting up. Right after docker compose up -d you may see a connection refused or a non-200 reply for a few seconds. Wait and retry; it is not an error.

Install the Python client

Weaviate's current Python client is weaviate-client, version 4.23 at the time of verification. It needs Python 3.10 or newer. Create a virtual environment and install it:

python -m venv .venv
source .venv/bin/activate
pip install -U weaviate-client

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1 instead. Now confirm Python can reach the server:

import weaviate

client = weaviate.connect_to_local()
try:
    print(client.is_ready())
    print(client.get_meta()["version"])
finally:
    client.close()

You should see True and then a version such as 1.39.7. The try/finally is not decoration. The v4 client holds open connections, and you must close it. The tidier pattern is a with block, which closes the client for you:

import weaviate

with weaviate.connect_to_local() as client:
    print(client.is_ready())

Connecting from Python

Before building anything, spend a few minutes on the connection, because most beginner problems show up here. A connection function returns a client object, and everything else is a method on it.

The simplest is weaviate.connect_to_local(). It assumes Weaviate is at localhost with REST on 8080 and gRPC on 50051, which is what the compose file above provides.

When your setup differs, use connect_to_custom and spell out both channels:

import weaviate

client = weaviate.connect_to_custom(
    http_host="localhost", http_port=8080, http_secure=False,
    grpc_host="localhost", grpc_port=50051, grpc_secure=False,
)

There are two channels because the client talks over REST for some operations and gRPC for others. That is why a half-working setup is possible: REST reachable and gRPC blocked produces the most common connection error, covered in the errors section.

For Weaviate Cloud you pass the cluster URL and an API key. Read them from environment variables rather than writing them into a file you might commit:

import os
import weaviate
from weaviate.classes.init import Auth

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)

If you later turn on API-key authentication on your own server, the same Auth.api_key(...) object goes into connect_to_local through its auth_credentials argument.

Many embedding and generation providers need their own key, for example an OpenAI key. You hand it to Weaviate as a request header named X-<Provider>-Api-Key, and the client has a headers argument for it:

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ["OPENAI_API_KEY"]}
)

The alternative is to set the server's own environment variable (OPENAI_APIKEY), in which case every client shares one key. If both are present, the header wins. Provider keys are never stored in the collection configuration, which is a good thing: a backup of your database does not contain them.

Two last connection habits. Always close the client, either with with or client.close(); forgetting produces a ResourceWarning, and using a client after closing raises WeaviateClosedClientError. And if you are building a web service, create one client at startup and reuse it rather than connecting on every request.

Your first collection and your first objects

Time to build something. We will start with the path that needs no API key and no extra services: bringing our own vectors. It is the best way to learn because nothing is hidden. Weaviate does not call any model. You see exactly what goes in and what comes out.

Real embedding vectors have hundreds of dimensions, which are unpleasant to type. For learning, we will use tiny three-dimensional vectors that we invent, where the three numbers loosely mean "fruit-ness", "vehicle-ness" and "animal-ness". The behaviour is identical to the real thing.

import weaviate
from weaviate.classes.config import Configure, Property, DataType

with weaviate.connect_to_local() as client:
    if client.collections.exists("Thing"):
        client.collections.delete("Thing")

    things = client.collections.create(
        "Thing",
        vector_config=Configure.Vectors.self_provided(),
        properties=[
            Property(name="name", data_type=DataType.TEXT),
            Property(name="kind", data_type=DataType.TEXT),
            Property(name="weight_kg", data_type=DataType.NUMBER),
        ],
    )

    things.data.insert(
        properties={"name": "apple", "kind": "fruit", "weight_kg": 0.2},
        vector=[0.9, 0.1, 0.0],
    )
    things.data.insert(
        properties={"name": "banana", "kind": "fruit", "weight_kg": 0.12},
        vector=[0.8, 0.0, 0.1],
    )
    things.data.insert(
        properties={"name": "truck", "kind": "vehicle", "weight_kg": 9000},
        vector=[0.0, 0.95, 0.05],
    )
    things.data.insert(
        properties={"name": "horse", "kind": "animal", "weight_kg": 500},
        vector=[0.1, 0.1, 0.9],
    )

    print(things.aggregate.over_all(total_count=True).total_count)

Take it apart. client.collections.create("Thing", ...) creates a collection and returns a handle to it. The name starts with an uppercase letter, as required. vector_config=Configure.Vectors.self_provided() says "I will give you the vectors; do not call any model". The properties list declares three typed fields. data.insert(properties=..., vector=...) adds one object and returns its UUID. The last line asks the collection how many objects it holds; it should print 4.

Notice the guard at the top: if client.collections.exists("Thing"). Creating a collection that already exists fails with an error such as class name Thing already exists (HTTP 422). Making scripts re-runnable like this saves you from the most boring error in the guide.

Now search. Because we supplied vectors, we search with a vector too, using near_vector:

import weaviate
from weaviate.classes.query import MetadataQuery

with weaviate.connect_to_local() as client:
    things = client.collections.use("Thing")

    result = things.query.near_vector(
        near_vector=[0.85, 0.05, 0.05],   # "fruit-ish"
        limit=2,
        return_metadata=MetadataQuery(distance=True),
    )

    for obj in result.objects:
        print(obj.properties["name"], round(obj.metadata.distance, 4))

Expected output is the two fruits, apple and banana, with small distances, and the closer one first. client.collections.use("Thing") gets a handle to an existing collection. (You may meet client.collections.get("Thing") in older v4 examples. It was replaced by use, so prefer use.) The return_metadata argument asks for the distance; without it Weaviate returns only the properties. Each result object has .uuid, .properties and .metadata.

Look at the distances. They are small for the fruits and would be large for the truck. That is the entire idea of a vector database in one result: the query had no word in common with anything, and the right objects still came back, because the numbers were close.

Letting Weaviate create the vectors

Typing vectors by hand teaches the idea, but in real projects you do not do it. You let an embedding model produce them. Weaviate supports this through modules (also called model provider integrations): you name a vectorizer when you create the collection, and from then on Weaviate calls the model whenever it needs a vector, both when you insert an object and when you run a text query.

The advantage is that your application code stays text-only. You insert {"title": "...", "body": "..."} and later search with near_text(query="..."). You never see a vector.

There are many providers: OpenAI, Cohere, Google, AWS, Mistral, Voyage AI, Hugging Face, Weaviate's own hosted embeddings (text2vec-weaviate) and more. For a course, a local model is attractive because it needs no account and no per-call cost. Weaviate's official local quickstart uses Ollama, a tool that runs open models on your own machine (see the Ollama guide for more).

The setup adds an Ollama container to the compose file and turns on two Weaviate modules, one for embeddings and one for generation. Replace docker-compose.yml with:

services:
  weaviate:
    command: [--host, 0.0.0.0, --port, '8080', --scheme, http]
    image: cr.weaviate.io/semitechnologies/weaviate:1.39.7
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      ENABLE_MODULES: 'text2vec-ollama,generative-ollama'
      CLUSTER_HOSTNAME: 'node1'
  ollama:
    image: ollama/ollama:0.12.9
    ports:
    - 11434:11434
    volumes:
    - ollama_data:/root/.ollama
volumes:
  weaviate_data:
  ollama_data:

Apply it and download two models: an embedding model, nomic-embed-text, and a small chat model, llama3.2, which we use later for generation:

docker compose up -d
docker compose exec ollama ollama pull nomic-embed-text
docker compose exec ollama ollama pull llama3.2

The downloads are a few gigabytes in total, so do this on a good connection. Now the part that trips almost everyone, so read it twice.

Create a collection that uses Ollama to vectorize text:

import weaviate
from weaviate.classes.config import Configure, Property, DataType

with weaviate.connect_to_local() as client:
    if client.collections.exists("Article"):
        client.collections.delete("Article")

    client.collections.create(
        "Article",
        vector_config=Configure.Vectors.text2vec_ollama(
            api_endpoint="http://ollama:11434",
            model="nomic-embed-text",
        ),
        generative_config=Configure.Generative.ollama(
            api_endpoint="http://ollama:11434",
            model="llama3.2",
        ),
        properties=[
            Property(name="title", data_type=DataType.TEXT),
            Property(name="body", data_type=DataType.TEXT),
        ],
    )

Two new arguments appear. vector_config=Configure.Vectors.text2vec_ollama(...) is the vectorizer: it names the endpoint and the embedding model. generative_config names the model Weaviate will call when you ask it to generate text. Both point at http://ollama:11434, as the warning explained.

A note on naming that will save you time. Older tutorials write vectorizer_config=Configure.Vectorizer.text2vec_openai() or Configure.NamedVectors.... Those forms are deprecated. The current form is vector_config=Configure.Vectors.<provider>(...), as above. If you prefer a hosted provider, swap in Configure.Vectors.text2vec_openai() and pass your key as a header, as shown in the connection section.

By default, Weaviate builds the vector from all text properties of the object, joined together. You can narrow it with source_properties, which the Mid-level guide covers. For now the default is sensible.

Loading data: single inserts and imports

With a vectorizer in place, loading data is just handing over properties. Weaviate calls the embedding model, stores the vector and indexes it.

import weaviate

articles_data = [
    {"title": "How to get your money back",
     "body": "Steps for requesting a refund after a purchase, and how long it takes."},
    {"title": "Training a neural network on a laptop",
     "body": "Tips for fitting small models into limited memory and keeping GPUs cool."},
    {"title": "A beginner's guide to sourdough",
     "body": "Flour, water, salt and patience: how to keep a starter alive."},
    {"title": "Reading a bank statement",
     "body": "What each line means, and how to spot a charge you did not make."},
]

with weaviate.connect_to_local() as client:
    articles = client.collections.use("Article")

    first_id = articles.data.insert(articles_data[0])
    print("inserted", first_id)

    articles.data.ingest(articles_data[1:])
    print(articles.aggregate.over_all(total_count=True).total_count)

Two methods, two purposes. data.insert(properties) adds one object and returns its UUID. It is fine for a handful of objects or for an application saving one record at a time. data.ingest(list_of_objects) is the bulk path. It streams a whole list to the server using server-side batching, so you do not tune batch sizes yourself. It is the one to use for anything beyond a few dozen objects. The line at the end should print 4.

Why does bulk loading deserve its own method? Every object needs an embedding, and embedding is the slow, expensive step. Sending objects one network call at a time is slow, and sending them all in one giant request can be rejected for being too large. The gRPC channel has a message size cap (GRPC_MAX_MESSAGE_SIZE, about 100 MB by default), and a huge payload fails with Sent message larger than max. ingest avoids both problems. You may see an older method insert_many in tutorials. It works for small lists, but it is the one that triggers that size error on big ones.

Three details about objects are worth knowing from day one.

First, UUIDs. If you do not supply an ID, Weaviate generates a random one. Running the loading script twice then gives you duplicates, because every run creates new random IDs. If you want re-runs to be safe, derive the ID from the content with the helper generate_uuid5:

from weaviate.util import generate_uuid5

obj_id = generate_uuid5(articles_data[0])
articles.data.insert(articles_data[0], uuid=obj_id)

The same content always yields the same UUID, so inserting it again fails with an "already exists" error instead of silently adding a duplicate. That is a feature: it turns a blind duplicate into a visible message.

Second, autoschema. If you insert a property that the collection does not define, Weaviate by default creates it for you, guessing the type from the value. That is AUTOSCHEMA_ENABLED, on by default. It is handy while exploring, but it also means a typo in a property name quietly creates a new property. Declare your properties explicitly, as we did, and turn autoschema off on servers that matter.

Third, errors in bulk loads do not always raise. A batch can partly succeed. After a batched load, check for failures rather than assuming success. Use the batch.stream() context manager when you want finer control and look at articles.batch.failed_objects afterwards. The Mid-level guide covers this in depth. For the beginner, the habit is: after importing, count the objects and compare with what you sent.

Searching: by meaning, by keyword, and by both

This is the payoff. Weaviate offers several ways to search, and choosing between them is the main skill of a beginner. All of them are methods on collection.query.

Search by meaning with near_text

near_text sends your query text to the vectorizer, gets a vector, and finds the nearest objects:

import weaviate
from weaviate.classes.query import MetadataQuery

with weaviate.connect_to_local() as client:
    articles = client.collections.use("Article")

    result = articles.query.near_text(
        query="I want to be reimbursed",
        limit=2,
        return_metadata=MetadataQuery(distance=True),
    )

    for obj in result.objects:
        print(round(obj.metadata.distance, 3), obj.properties["title"])

The query contains none of the words "refund", "money" or "back", and the refund article should still come first. That is semantic search. Exact distances depend on the embedding model, so your numbers will differ from anyone else's. What matters is the order.

Search by keyword with bm25

bm25 is classic keyword search. It scores objects by how often the query's words appear, weighting rare words more heavily. No embedding model is involved:

result = articles.query.bm25(query="sourdough starter", limit=3)
for obj in result.objects:
    print(obj.properties["title"])

Keyword search wins when the user types something specific that must appear literally: a product code, an error message, a person's name. Vector search can blur such exact tokens. If the query is "ERR_4021", you want BM25.

Combine them with hybrid

hybrid runs both a vector search and a BM25 search and fuses the two rankings:

result = articles.query.hybrid(query="refund policy", alpha=0.5, limit=3)
for obj in result.objects:
    print(obj.properties["title"])

The alpha parameter sets the balance: 1 means pure vector search, 0 means pure keyword search, and 0.5 weighs them equally. By default the two result lists are fused with relative score fusion, the method Weaviate has used for hybrid search since version 1.24. The other option is rank-based fusion (HybridFusion.RANKED), a topic for later.

Which should a beginner choose? A reasonable default for text search in real applications is hybrid, because it handles both the vague question and the exact term. Start with alpha=0.5 and adjust after looking at real results. Use pure near_text when queries are always natural language, and pure bm25 when they are always exact tokens.

Search is not always about text

You have already used near_vector with your own vectors. Weaviate also has near_object (find objects similar to a given object, which is the "more like this" feature) and, with a multimodal module, near_image. They all follow the same shape: a query, a limit, and optional metadata.

Reading results

Every search returns an object with an .objects list. Each item has:

• .uuid, the object's identifier;
• .properties, a dictionary of the stored fields;
• .metadata, which holds distance, score or other values only if you asked for them with return_metadata=MetadataQuery(...).

For bm25 and hybrid, the relevant metadata is score (higher is better), requested with MetadataQuery(score=True). For vector searches it is distance (lower is better). Mixing these up is the classic beginner mistake of reading a distance as a score.

A last note about limit. If you omit it, the server applies its default (QUERY_DEFAULTS_LIMIT, which our compose file set to 25). Always set limit explicitly so your code does not depend on server configuration.

Filtering: combining meaning with exact conditions

Real questions mix fuzzy and exact: "articles about refunds, written this year, in English". Vector similarity cannot express "this year". For exact conditions you use filters. A filter limits which objects are even considered, and you attach it to any search.

Filters use the Filter class. Our Article collection has only text properties, so to demonstrate, add a richer collection. This one uses self-provided vectors again, so it works with no models running:

import weaviate
from weaviate.classes.config import Configure, Property, DataType
from weaviate.classes.query import Filter

with weaviate.connect_to_local() as client:
    if client.collections.exists("Product"):
        client.collections.delete("Product")

    products = client.collections.create(
        "Product",
        vector_config=Configure.Vectors.self_provided(),
        properties=[
            Property(name="name", data_type=DataType.TEXT),
            Property(name="category", data_type=DataType.TEXT),
            Property(name="price", data_type=DataType.NUMBER),
            Property(name="in_stock", data_type=DataType.BOOL),
        ],
    )

    products.data.insert({"name": "Trail shoes", "category": "outdoor",
                          "price": 89.0, "in_stock": True}, vector=[0.9, 0.1])
    products.data.insert({"name": "Hiking boots", "category": "outdoor",
                          "price": 140.0, "in_stock": False}, vector=[0.85, 0.15])
    products.data.insert({"name": "Office chair", "category": "furniture",
                          "price": 210.0, "in_stock": True}, vector=[0.1, 0.9])

    cheap_outdoor = products.query.near_vector(
        near_vector=[0.9, 0.1],
        limit=5,
        filters=(
            Filter.by_property("category").equal("outdoor")
            & Filter.by_property("price").less_than(100)
        ),
    )
    for obj in cheap_outdoor.objects:
        print(obj.properties["name"], obj.properties["price"])

The boolean type is spelled DataType.BOOL in the Python client. Only the trail shoes pass both conditions. The pieces: Filter.by_property("category") selects a property, and an operator method compares it. The available operators include equal, not_equal, less_than, less_or_equal, greater_than, greater_or_equal, like (with * wildcards, for example like("*shoe*")), and for array properties contains_any and contains_all. Combine conditions with & (and) and | (or), putting each condition in parentheses when you chain several.

You can also use a filter without any vector search, with fetch_objects. That is Weaviate acting like an ordinary database:

result = products.query.fetch_objects(
    filters=Filter.by_property("in_stock").equal(True),
    limit=10,
)
for obj in result.objects:
    print(obj.properties["name"])

There is also fetch_object_by_id(uuid) to get a single object when you know its ID.

One subtlety trips beginners with text. Filters and keyword matching work on tokens, not on the whole string. By default text is tokenized by word, which splits it into lowercase words. So an equal("outdoor") match on a multi-word text property matches objects that contain that word, and a filter on a full sentence behaves differently from what you might expect. When you need an exact whole-value match, such as an ID stored as text, create the property with tokenization=Tokenization.FIELD (imported from weaviate.classes.config). The Mid-level guide returns to this.

Generative search: asking a model about your results

So far Weaviate found objects. Generative search goes one step further: it finds objects and then asks a language model to do something with them. This is RAG in a single call. It uses the generate namespace, and it needs the generative_config we gave the Article collection.

import weaviate

with weaviate.connect_to_local() as client:
    articles = client.collections.use("Article")

    response = articles.generate.near_text(
        query="problems with my bank account",
        limit=2,
        single_prompt="Summarise this article in one sentence: {body}",
        grouped_task="In two sentences, say what these articles have in common.",
    )

    for obj in response.objects:
        print(obj.properties["title"])
        print("  ->", obj.generative.text)

    print("Overall:", response.generative.text)

There are two ways to prompt and you can use either or both.

single_prompt runs the model once per retrieved object. Curly braces interpolate that object's properties, so {body} is replaced with each article's body in turn. The result for each object is at obj.generative.text.

grouped_task runs the model once over all the results together, and the answer is at response.generative.text. Use it for summaries and questions across the whole result set.

The first call is slow, because the local model has to load into memory. If the model name is wrong, or Ollama is not reachable at http://ollama:11434, you get an error from Weaviate about the generative module, not from Python.

You can override the generation model for a single query without changing the collection. Version 1.30 added that:

from weaviate.classes.generate import GenerativeConfig

response = articles.generate.near_text(
    query="problems with my bank account",
    limit=2,
    grouped_task="Name the main theme.",
    generative_provider=GenerativeConfig.ollama(
        api_endpoint="http://ollama:11434",
        model="llama3.2",
    ),
)
print(response.generative.text)

The same generate namespace exists for bm25, hybrid and near_vector, so you can retrieve with whichever search suits the data and then generate.

Keep expectations honest. Generation quality depends on the model you choose and on how good the retrieved objects are. A small local model will be weaker than a large hosted one. And if retrieval returns the wrong passages, the answer will be wrong, however good the model. Most "the RAG gives bad answers" problems are search problems, which is why the quality of your search is worth more attention than the prompt. The RAGAS guide covers how to measure that.

Everyday operations: update, delete, inspect

Once data is in, you need to change it, remove it and look around. These are the operations you will use constantly.

Update an object. data.update merges the properties you give into the existing object, leaving other properties alone. data.replace overwrites the object entirely:

articles.data.update(uuid=first_id, properties={"title": "How to get a refund"})

When the text of a vectorized property changes, Weaviate recomputes the vector. A merge that changes a vectorized property therefore costs another embedding call.

Delete an object. By ID:

articles.data.delete_by_id(first_id)

Delete many objects by filter uses delete_many:

from weaviate.classes.query import Filter

articles.data.delete_many(where=Filter.by_property("title").like("*sourdough*"))

Be careful: a delete filter that matches more than you intended deletes it all. Run the same filter through fetch_objects first and look at what it returns. Weaviate does reject a batch delete that omits its match condition, returning 422 Unprocessable Entity since 1.39 (it used to return a 500).

Read everything. For a full pass over a collection, use the iterator, which pages through objects with a cursor and has no result cap:

for obj in articles.iterator():
    print(obj.uuid, obj.properties["title"])

Ordinary queries are limited by QUERY_MAXIMUM_RESULTS (10,000 by default), so a search with a huge limit is not the way to export data.

Inspect collections.

print(client.collections.list_all())
print(client.collections.exists("Article"))

Delete a collection. This deletes the collection and all its objects:

client.collections.delete("Article")

There is no undo and no confirmation prompt. Treat it like DROP TABLE.

Add a property later. You can add a new property to an existing collection:

from weaviate.classes.config import Property, DataType

articles.config.add_property(Property(name="author", data_type=DataType.TEXT))

What you generally cannot do afterwards is change the settings that define the collection: a property's data type, the distance metric, the vectorizer. For those, you create a new collection and copy the data across. This is the strongest argument for the "think before you create" advice. A beginner who wants to change the embedding model is not stuck, but they are going to rebuild the collection, and re-embedding a big dataset takes time and money.

The REST equivalents. Everything above also exists over plain HTTP, which is handy for debugging with curl. GET /v1/schema lists collections and their configuration, GET /v1/objects lists objects, POST /v1/batch/objects batch-loads them and DELETE /v1/objects/{className}/{id} deletes one. You will use GET /v1/schema often to confirm what a collection really looks like:

curl http://localhost:8080/v1/schema

Configuration you should know about

Weaviate is configured almost entirely through environment variables set on the server, which in our setup means the environment: block of the compose file. You do not need to memorise them. You need to know the handful that matter early and where to find the rest: the environment-variable reference in the official docs.

The ones that show up in a beginner's life:

A word on modules. A module is an optional plug-in that adds a capability: a vectorizer, a generative model, a reranker, a backup target. Since version 1.33, modules that call hosted API providers are enabled by default, and the variable API_BASED_MODULES_DISABLED=true turns them off. Modules that run locally, like text2vec-ollama, you list in ENABLE_MODULES. The /v1/meta endpoint shows which are active.

Authentication deserves a clear statement, because the quickstart configuration hides it. The setup in this guide has anonymous access on: anyone who can reach port 8080 can read, change and delete everything. That is acceptable on localhost. The moment a server is reachable by anyone else, enable API-key authentication and authorization instead. The relevant variables are AUTHENTICATION_APIKEY_ENABLED, AUTHENTICATION_APIKEY_ALLOWED_KEYS and AUTHENTICATION_APIKEY_USERS, with role-based access control through AUTHORIZATION_RBAC_ENABLED and AUTHORIZATION_RBAC_ROOT_USERS. If you turn RBAC on without naming at least one root user, Weaviate refuses to start. This is a topic for the Mid-level guide. The beginner's rule is simply: never put an anonymous Weaviate on a public address.

Persistence is the other thing to get right. The volume at /var/lib/weaviate is where your data lives. docker compose down stops and removes containers but keeps the volume, so your data survives. docker compose down -v also deletes the volumes, and with them all your data. The -v is easy to type from muscle memory.

Reading common errors

Most Weaviate errors fall into a small set. Learn to recognise them by their wording.

WeaviateGRPCUnavailableError: gRPC health check could not be completed. The client reached the REST port but not the gRPC port. Almost always, port 50051 is not published, is blocked, or the client is pointed at the wrong gRPC host or grpc_secure setting. In Docker, make sure the compose file has 50051:50051. In Kubernetes, the gRPC service must be enabled in the Helm values. Use connect_to_custom to set the gRPC host and port explicitly. There is a skip_init_checks=True option, but treat it as a temporary diagnostic, not a fix.

class name Article already exists (HTTP 422). You tried to create a collection that is already there. Check client.collections.exists(...) first, or delete it.

401 Unauthorized, or a message about anonymous access not being enabled. The server requires authentication and your client sent nothing. Pass Auth.api_key(...) or enable anonymous access on a local dev server.

403 Forbidden. You authenticated, but your user's role lacks permission for that action. This is RBAC at work.

Vectorizer returns 401 or 429 on insert. The model provider rejected the call: a missing or wrong key (401), or you are sending requests faster than the provider allows (429). Check the X-<Provider>-Api-Key header or the server's <PROVIDER>_APIKEY variable, and slow your imports.

Ollama "connection refused". Weaviate is trying localhost from inside its container. Use http://ollama:11434.

Sent message larger than max. One request exceeded the gRPC size cap. Load with data.ingest(...) instead of one huge insert_many.

WeaviateClosedClientError, or a ResourceWarning about unclosed connections. You used a client after closing it, or never closed it. Use a with block.

The server will not accept writes and shards show READONLY. The disk or memory crossed a safety threshold (by default 90 percent disk use makes shards read-only). Free space and restore the shard's status. You will meet this in real use, rarely on a laptop.

All errors raised by the Python client inherit from weaviate.exceptions.WeaviateBaseError, with more specific classes such as WeaviateConnectionError, WeaviateQueryError and WeaviateBatchError. A try/except on the specific class lets you handle a connection failure differently from a bad query.

When an error is unclear, two sources of truth help. The container logs, from docker compose logs weaviate, show the server's side of the story, including module and vectorizer failures that the client reports only vaguely. And curl http://localhost:8080/v1/meta confirms the version and modules you are actually running, which settles many "but I configured it" disputes.

Putting it all together

Let us finish with one small end-to-end project that uses everything: a searchable help centre with a question-answering endpoint. It has three steps and one file each.

First, the setup script creates the collection with explicit properties, including one for filtering:

import weaviate
from weaviate.classes.config import Configure, Property, DataType

with weaviate.connect_to_local() as client:
    if client.collections.exists("HelpArticle"):
        client.collections.delete("HelpArticle")

    client.collections.create(
        "HelpArticle",
        vector_config=Configure.Vectors.text2vec_ollama(
            api_endpoint="http://ollama:11434", model="nomic-embed-text"),
        generative_config=Configure.Generative.ollama(
            api_endpoint="http://ollama:11434", model="llama3.2"),
        properties=[
            Property(name="title", data_type=DataType.TEXT),
            Property(name="body", data_type=DataType.TEXT),
            Property(name="topic", data_type=DataType.TEXT),
        ],
    )

Second, the loader reads articles from a JSON file named articles.json (a list of objects with title, body and topic), derives stable IDs so re-runs do not duplicate, and verifies the count:

import json
import weaviate
from weaviate.classes.data import DataObject
from weaviate.util import generate_uuid5

with open("articles.json", encoding="utf-8") as f:
    rows = json.load(f)

with weaviate.connect_to_local() as client:
    col = client.collections.use("HelpArticle")
    col.data.insert_many([
        DataObject(properties=r, uuid=generate_uuid5(r["title"])) for r in rows
    ])
    print("expected", len(rows), "stored",
          col.aggregate.over_all(total_count=True).total_count)

For a few dozen articles insert_many is fine, and DataObject lets us attach a stable UUID to each. For thousands, switch to the batching approach from the Mid-level guide. Notice the last two lines: the loader compares what it sent with what is stored.

Third, the question-answering script retrieves with hybrid search, optionally limited to a topic, and has the model answer from the retrieved articles:

import sys
import weaviate
from weaviate.classes.query import Filter, MetadataQuery

question = " ".join(sys.argv[1:]) or "How do I get a refund?"

with weaviate.connect_to_local() as client:
    col = client.collections.use("HelpArticle")

    result = col.generate.hybrid(
        query=question,
        alpha=0.5,
        limit=3,
        filters=Filter.by_property("topic").equal("billing"),
        return_metadata=MetadataQuery(score=True),
        grouped_task=f"Answer this question using only the articles provided: {question}",
    )

    print("Question:", question)
    for obj in result.objects:
        print(f"  {obj.metadata.score:.3f}  {obj.properties['title']}")
    print("Answer:", result.generative.text)

Run it with python ask.py "How long does a refund take?". Each step maps to a concept from this guide. The collection and properties are the schema. The vectorizer turns text into vectors automatically. hybrid combines meaning and keywords. The Filter restricts results with an exact condition. generate with grouped_task turns the retrieved objects into an answer, and the sources list above it shows which articles that answer rests on.

If you want to take it further, wrap ask.py in a small FastAPI service, or use a framework such as LangChain to orchestrate it. The database layer you have just learned stays the same.

What you can now do, and what comes next

You can now run Weaviate in Docker, check that it is healthy, and connect to it from Python with the v4 client. You can create collections with typed properties, choose between bringing your own vectors and letting a module like Ollama create them, and load data with insert, ingest and stable UUIDs. You can search by meaning, keyword or both, narrow results with filters, and have a model generate answers over what was retrieved. You can update, delete and inspect data, read the main configuration variables, and diagnose the errors beginners meet most.

What you have not yet touched is what separates a demo from a service. Mid-level goes into batching with error handling, named vectors (several vectors per object), tuning hybrid search, filters on references and metadata, multi-tenancy for serving many customers from one collection, quantization to shrink memory, schema changes and aliases, and the async client. Senior covers replication and consistency, sharding, backups and restores, authentication with OIDC and RBAC, upgrades, monitoring with Prometheus, cost and capacity planning, and when Weaviate is the wrong tool.

Two practical pointers before you go. Weaviate releases often, and a new minor version appears every couple of months; only the latest three minor versions receive fixes, so check the release notes before you start a project and pin the exact version in your compose file. And look at how Weaviate compares with its neighbours in this catalogue, Qdrant, Milvus, Chroma and pgvector, so you can explain why you chose it.

For readers working with Gulf or Egyptian employers, one more practical point: where the vectors are computed matters for data residency. With a local model such as Ollama, document text never leaves your infrastructure. With a hosted embedding provider, the text of every object is sent to that provider's API, which may be an issue for regulated data. Weaviate lets you choose either, per collection, and that is worth raising early in a design discussion.

Sources

• Weaviate documentation
• Weaviate release notes
• Weaviate on GitHub: releases
• Local quickstart
• Docker installation guide
• Python client library
• Environment variables reference
• Collection operations
• Importing data
• Hybrid search
• Filters
• Generative search
• Model providers
• Data types
• Authentication

Part one of three. A twenty-five-minute review of the Beginner Detailed track, then the questions interviewers ask. Mid-level reviews this plus its own material; Senior reviews all three. The guide targets Weaviate 1.39 and the Python client v4.

The thirty-second answer

Weaviate is an open-source vector database written in Go. It stores objects with their vectors, so you can search by meaning, by keyword (BM25) or by a hybrid of both, filter on ordinary properties, and optionally pass the results to a language model for generative search. Data is organised into collections of typed objects, and an approximate nearest neighbour index, HNSW by default, keeps vector search fast.

Then the line that shows you have used it: "the vectors come either from a vectorizer module that Weaviate calls for me, or from my own embeddings, and several collection settings, such as the vectorizer and distance metric, are fixed at creation, so I decide them before I load data."

The model

Three ports and interfaces to keep straight: REST on 8080, gRPC on 50051 (required by the v4 clients), and GraphQL at /v1/graphql, which is the legacy query interface.

If the interviewer asks "why a vector database at all", the model answer is a short story. Keyword search matches words, so it misses paraphrase. Embedding models place texts with similar meaning near each other as vectors. Comparing a query to every vector is too slow at scale, so a vector database uses an approximate index to find near neighbours quickly, and also stores the original objects and supports filters.

The commands to have ready

You are rarely asked to type these from memory, but you should be able to say what each does and write the shape of it.

Start the server and check it:

docker compose up -d
curl http://localhost:8080/v1/meta
curl -i http://localhost:8080/v1/.well-known/ready

Connect and create a collection with the v4 Python client:

import weaviate
from weaviate.classes.config import Configure, Property, DataType

with weaviate.connect_to_local() as client:
    client.collections.create(
        "Article",
        vector_config=Configure.Vectors.self_provided(),
        properties=[Property(name="title", data_type=DataType.TEXT)],
    )

Insert, search and filter:

articles = client.collections.use("Article")
articles.data.insert({"title": "Hello"}, vector=[0.1, 0.2, 0.3])
articles.query.near_vector(near_vector=[0.1, 0.2, 0.3], limit=3)
articles.query.bm25(query="hello", limit=3)
articles.query.hybrid(query="hello", alpha=0.5, limit=3)

from weaviate.classes.query import Filter

articles.query.fetch_objects(
    filters=Filter.by_property("title").equal("Hello"), limit=10
)

Generative search, when the collection has a generative module configured:

articles.generate.near_text(
    query="refunds", limit=3,
    single_prompt="Summarise: {body}",
    grouped_task="What do these have in common?",
)

A few operational one-liners round it out. articles.aggregate.over_all(total_count=True) counts objects. articles.data.update(uuid=..., properties=...) merges changes, while replace overwrites. articles.data.delete_by_id(uuid) removes one object. client.collections.delete("Article") removes the collection and all its objects. curl http://localhost:8080/v1/schema shows how a collection is really configured.

The classic traps and why they happen

The gRPC port is not published. The Python v4 client talks over both REST and gRPC. If you publish only 8080, the REST check passes and the client then fails with WeaviateGRPCUnavailableError: gRPC health check could not be completed. The fix is to publish 50051, or to set the gRPC host, port and secure flag explicitly with connect_to_custom. Interviewers love this one because it is the first thing everyone hits.

localhost inside a container. When Weaviate runs in Docker and the embedding model runs in another container such as Ollama, Weaviate must reach it by the compose service name, http://ollama:11434. From inside Weaviate's container, localhost is Weaviate itself. The symptom is "connection refused" from the module.

Following version 3 tutorials. weaviate.Client(...), client.schema.create_class and client.query.get(...) belong to the v3 client, which is no longer supported. The current API is weaviate.connect_to_*() and client.collections.*. Using the old names signals that your knowledge is out of date.

Deprecated v4 names. Within v4 some forms changed too. client.collections.get was replaced by client.collections.use, and vectorizer_config=Configure.Vectorizer... was replaced by vector_config=Configure.Vectors.... Mentioning that you use the current forms reads well.

Reading distance as a score. Vector searches return a distance, where lower is closer. BM25 and hybrid return a score, where higher is better. Each appears in metadata only if you request it.

Anonymous access in the quickstart. The sample configuration enables anonymous access. That is fine on a laptop and dangerous on a reachable server. The production baseline is API keys or OIDC, plus RBAC.

docker compose down -v deletes your data. Data lives in a volume mounted at /var/lib/weaviate, and -v removes the volumes. Without any volume, removing the container loses the data too.

Settings you cannot change later. The vectorizer, the distance metric and property data types are set at creation. Changing them means creating a new collection and copying data across, re-embedding if the model changed. Know this before you recommend a model choice.

Random UUIDs and duplicates. Without an ID, Weaviate generates a random UUID, so re-running an import duplicates everything. Deterministic IDs from generate_uuid5 make imports repeatable.

Autoschema hides typos. With autoschema on (the default), inserting a misspelled property silently creates a new property instead of failing.

Forgetting to close the client. The v4 client holds connections. Use with or client.close(). A closed client raises WeaviateClosedClientError if you reuse it.

Common interview questions

What is a vector database, and why not just use a regular database? A vector database stores embeddings and answers nearest-neighbour queries quickly using an approximate index. A regular database answers exact questions well but cannot rank by semantic similarity. You can bolt vector search onto a relational database, for example with pgvector, and that is a perfectly good choice at smaller scale or when the data already lives there. A dedicated system such as Weaviate adds features built around search: hybrid search, built-in model integrations, multi-tenancy and replication.

What is the difference between a collection, an object and a property? A collection is a named set of objects that share a schema, comparable to a table. An object is one record, with a UUID, properties and optional vectors, comparable to a row. A property is a typed field, comparable to a column.

How do vectors get into Weaviate? Two ways. A vectorizer module, such as text2vec-openai or text2vec-ollama, is configured on the collection, and Weaviate calls the embedding model on insert and on text queries. Or I bring my own vectors, configure the collection with Configure.Vectors.self_provided(), and pass vector= on insert. The first keeps application code text-only. The second gives me full control, for example when my embeddings come from a pipeline I already run.

What does alpha do in hybrid search? It balances the two ranked lists: 1 means pure vector search and 0 means pure keyword search. The default fusion is relative score fusion. I would start at 0.5 and tune against real queries, because the right value depends on how exact the queries tend to be.

When would you prefer BM25 over vector search? When queries contain exact tokens that must match literally, such as error codes, SKUs and names. Vector search can blur those. Hybrid search covers both cases, which is why it is a sound default for text search.

What does HNSW do and what is its main cost? HNSW is a layered graph that lets a search jump quickly toward the nearest vectors instead of scanning all of them. It is approximate, trading a little recall for large speed-ups. Its main cost is memory, since the index lives in RAM. At scale you reduce that with compression such as rotational quantization, or choose a different index. That is mid-level and senior territory, but saying it shows you understand the trade.

Explain generative search. It is retrieval and generation in one call. Weaviate retrieves objects with a normal search, then sends them to a configured generative module. single_prompt runs the model once for each object, with {property} placeholders. grouped_task runs it once over all results. The quality of the answer is bounded by what retrieval returned.

Why would a v4 client fail even though the REST port works? Because the client also needs gRPC on 50051. Either the port is not published, a firewall blocks it, or on Kubernetes the gRPC service is not enabled. The error message mentions the gRPC health check.

How do you keep data when the container is replaced? Mount a volume at /var/lib/weaviate. The data survives docker compose down and container upgrades, but not docker compose down -v. For real protection you also take backups, which is a later topic.

How do you make imports idempotent? Derive the UUID from the content, with generate_uuid5, so the same record always gets the same ID. A second run then fails visibly instead of adding a duplicate.

What would you check if search results look wrong? First, am I using the metric I think I am, and reading distance or score correctly? Second, did the import fully succeed, so I should compare the count against what I sent? Third, which properties were vectorized, and is the model suited to the language of the data? Fourth, would hybrid search help because exact terms matter? And fifth, is a filter excluding objects I expected?

How do you authenticate to Weaviate? Anonymous access is allowed in the quickstart. For anything shared I enable API keys or OIDC, with AUTHENTICATION_APIKEY_ENABLED and related variables, and use RBAC for permissions. In Python the key goes into Auth.api_key(...) and is passed as auth_credentials. Provider keys for embedding models are separate: they go in request headers like X-OpenAI-Api-Key, and are not stored in the collection configuration.

How would you let Weaviate use a local model? Run Ollama alongside it, enable text2vec-ollama and generative-ollama through ENABLE_MODULES, pull models such as nomic-embed-text and llama3.2, and point the collection configuration at http://ollama:11434. Text then never leaves my infrastructure, which matters for data residency.

What changed recently that I should know? Mention the Python v3 client being unsupported, the v4 naming changes, and that the 1.39 line is current with only the latest three minor versions receiving patches. If pressed, the Boost API and MMR diversity selection became generally available in 1.39. Do not go deeper than you can defend.

What a strong answer covers that a weak one misses

A weak answer defines a vector database as "a database for AI" and stops. A strong answer names the problem (keyword search misses meaning), the mechanism (embeddings plus approximate nearest neighbour search), and the trade (a little accuracy for a lot of speed, paid for in memory).

A weak answer lists search types. A strong answer says when each one wins: vector for paraphrase, BM25 for exact tokens, hybrid when you cannot predict which the user will type, filters when the condition is exact.

A weak answer shows the code from a tutorial. A strong answer mentions the failure modes: the gRPC port, the localhost trap in Docker, immutable collection settings, duplicate imports and distance versus score.

A weak answer says "I would use Weaviate" for everything. A strong answer can name an alternative and the reason to choose it: pgvector when the data is already in PostgreSQL and the scale is modest, Qdrant or Milvus as other dedicated engines, or a managed service when the team does not want to operate a database.

How to talk about experience you do not have

If you have only done the quickstart, say so, then show you understand it. "I ran Weaviate in Docker with Ollama, loaded a few hundred documents, and compared near_text, bm25 and hybrid on my own queries. I did not run a cluster." That is credible and specific. Claiming production experience that you cannot back with details is the fastest way to lose an interview, because the next question will be about a failure you have never seen.

If you are asked about something outside this level, such as multi-tenancy, replication or quantization, a good beginner answer is to give the one-line definition and say you have not used it. For example: "Multi-tenancy gives each tenant its own shard inside a collection so that tenants are isolated and a tenant can be removed cheaply. I have read about it but not run it." Honesty plus the correct definition scores better than a confident guess.

Sixty-second self-test

Answer each aloud, without notes. If you hesitate on more than two, reread the matching section of the detailed track.

1. In two sentences, what is Weaviate and what problem does it solve?
2. What are the three things a collection, an object and a property correspond to in a relational database?
3. Which two ports must be published when running Weaviate in Docker, and what uses each?
4. Which Python function connects to a local server, and which block guarantees the client is closed?
5. What is the difference between a vectorizer module and self-provided vectors?
6. From inside Weaviate's container, how do you reach an Ollama container?
7. Which search would you use for the query "ERR_4021", and which for "how do I get my money back"?
8. What does alpha=0 mean in a hybrid query?
9. Does a vector search return distance or score, and which direction is better?
10. Name two collection settings you cannot change after creation.
11. What does docker compose down -v do that docker compose down does not?
12. How do you stop a re-run of an import from duplicating objects?
13. What is the difference between single_prompt and grouped_task?
14. Why should you not follow a tutorial that calls weaviate.Client(...)?

Answers. (1) An open-source vector database that stores objects with vectors and supports vector, keyword, hybrid and generative search, so you can search by meaning. (2) A table, a row and a column. (3) 8080 for REST and 50051 for gRPC, which the v4 clients need. (4) weaviate.connect_to_local(), and a with block. (5) A vectorizer calls an embedding model for you; self-provided means you pass in your own vectors. (6) By the service name, http://ollama:11434. (7) BM25 for the code, near_text or hybrid for the question. (8) Pure keyword search. (9) Distance, and lower is closer. (10) Any two of the vectorizer, the distance metric and a property's data type. (11) It also deletes the volumes, and so your data. (12) Derive the UUID with generate_uuid5. (13) The first runs the model per object, the second once over all results. (14) That is the v3 client, which is no longer supported.

Mid-level takes each of these further. Expect questions on batching with error handling, named vectors, tuning hybrid and filters, multi-tenancy, quantization and schema changes. Senior then adds replication and consistency, sharding, backups, security, upgrades and capacity planning. Read the Docker guide too if container networking was the shaky part, and the Ollama guide for the local model side.

Part one of three. Three mistakes produce most of the beginner errors below: you treat Weaviate as a single port when it is two, you forget that some collection settings are fixed at creation, or you read search output in the wrong direction. Start with the table, then the longer explanations, the habits and the checklist. This guide targets Weaviate 1.39 and the Python client v4.

Common errors at this level

Mistakes in detail

Mistake 1: publishing only port 8080

Symptom. curl http://localhost:8080/v1/meta works and the browser is happy, but Python raises WeaviateGRPCUnavailableError, with wording about gRPC "not enabled or incorrectly configured on the server or the client".

Cause. Weaviate serves REST on 8080 and gRPC on 50051, and the v4 Python client uses both. Searches and batch imports go over gRPC. A compose file with only 8080:8080 leaves the client half connected, and its start-up health check catches it.

Fix. Add 50051:50051 under ports, recreate the container, and reconnect. If your gRPC endpoint is on a different host or port, or uses TLS while REST does not, use connect_to_custom and set grpc_host, grpc_port and grpc_secure yourself. The client has a skip_init_checks=True option that suppresses the check. Use it only to find out whether everything else works. It does not fix the missing port, and searches will still fail.

Mistake 2: using localhost from inside a container

Symptom. Inserting an object into a collection with text2vec-ollama fails with a connection error mentioning port 11434, although ollama is clearly running and curl localhost:11434 works on your laptop.

Cause. Two different localhosts. Your laptop's localhost reaches Ollama through the published port. Weaviate's localhost is the Weaviate container, where nothing is listening.

Fix. In the collection's api_endpoint, use the compose service name, http://ollama:11434. This works because Docker Compose puts both services on one network where service names resolve as hostnames. On Docker Desktop, host.docker.internal is an alternative when Ollama runs directly on the host. Your Python script keeps using localhost for Weaviate, because the script runs on your laptop.

Mistake 3: choosing the vectorizer carelessly

Symptom. A month into a project you discover that a better embedding model exists, or that the one you picked handles Arabic poorly, and you cannot switch in place.

Cause. The vectorizer, like the distance metric and property data types, is part of the collection's definition and cannot be changed afterwards. Vectors produced by two different models are not comparable anyway, so mixing them in one collection would give nonsense.

Fix. Before loading real data, test two or three candidate models on a small sample of your own text and queries. If you are working with Arabic or mixed Arabic and English content, test with real Arabic examples, because a model's quality varies a lot across languages. When you must change later, create a new collection, copy the objects across and re-embed. Plan for that: keep the original text in your source system, or at least be able to read every object back with the iterator().

Mistake 4: reading the wrong metadata

Symptom. You print obj.metadata.score after a near_text search and get None, or you sort descending by distance and the worst results come first.

Cause. Metadata is returned only when requested, and the relevant field depends on the search. Vector searches have distance, where lower is closer. BM25 and hybrid have score, where higher is better.

Fix. Ask explicitly:

from weaviate.classes.query import MetadataQuery

r = articles.query.near_text(query="refund", limit=3,
                             return_metadata=MetadataQuery(distance=True))
h = articles.query.hybrid(query="refund", limit=3,
                          return_metadata=MetadataQuery(score=True))

Results already arrive in the right order, so you rarely need to sort them yourself.

Mistake 5: relying on the default limit

Symptom. A query returns 25 results on your machine and 10 on a colleague's, or code that worked in one environment returns a different number in another.

Cause. When you omit limit, the server uses QUERY_DEFAULTS_LIMIT, which is set by whoever configured the server. The compose file in the guide sets it to 25. The built-in default is 10.

Fix. Always pass limit= explicitly. For reading everything, do not raise the limit to a huge number, because there is a hard cap (QUERY_MAXIMUM_RESULTS, 10,000 by default). Use collection.iterator(), which pages with a cursor.

Mistake 6: duplicate objects from re-running a script

Symptom. Your collection holds 2,000 objects after loading 1,000 articles twice. Search results show the same article twice in a row.

Cause. Without an explicit ID, Weaviate generates a random UUID for every insert. Same content, different IDs, so it is a new object each time.

Fix. Create a deterministic ID from the content that identifies the record:

from weaviate.util import generate_uuid5

obj_id = generate_uuid5({"title": row["title"]})
articles.data.insert(row, uuid=obj_id)

Now an identical re-insert is rejected visibly rather than silently duplicated. To clean up existing duplicates, delete the collection and reload.

Mistake 7: huge single requests

Symptom. Loading a big list fails with Sent message larger than max, or the process stalls for ages and then times out.

Cause. An insert_many call sends everything at once. A large payload exceeds the gRPC message limit (about 100 MB by default). With a vectorizer module, the server also has to embed every object, which takes time.

Fix. Use articles.data.ingest(list_of_objects), which streams the data using server-side batching. Check the result by counting objects afterwards. If the provider is rate-limiting you (HTTP 429), slow the import down rather than retrying harder.

Mistake 8: leaving anonymous access on

Symptom. No visible symptom, which is the danger. A teammate or a scanner on the same network deletes a collection.

Cause. The quickstart configuration sets AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true' so that nothing needs configuring. That setting is meant for a laptop.

Fix. Bind to localhost, or better, switch to API-key authentication before the server is reachable from anywhere else. Provider API keys belong in environment variables on your side and are sent as headers, never committed to a repository. If you accidentally commit a key, revoke it at the provider, because deleting the commit does not remove it from history.

Mistake 9: typing a typo into autoschema

Symptom. A property that should hold values is empty on most objects, and the collection has a strange new property named like your typo.

Cause. AUTOSCHEMA_ENABLED is on by default, so inserting an unknown property name makes a new property rather than an error.

Fix. Declare properties when you create the collection, and inspect the real schema from time to time with curl http://localhost:8080/v1/schema. On shared servers, turn autoschema off so typos fail loudly.

Mistake 10: running docker compose down -v

Symptom. Everything vanishes, the collection is gone, and nothing in the logs says why.

Cause. -v removes named volumes, including the one holding Weaviate's data. The same loss happens if the compose file never defined a volume and the container is removed.

Fix. Use docker compose down to stop and remove containers while keeping data. Make sure volumes: maps weaviate_data to /var/lib/weaviate. Keep your loading scripts, so rebuilding is a command and not a crisis, and practise the rebuild once on purpose.

Habits worth adopting on day one

Pin the version. Write cr.weaviate.io/semitechnologies/weaviate:1.39.7, not a floating tag, and write down which client version you used. Weaviate ships a new minor version every couple of months, and only the latest three minors receive fixes. A pinned version makes a bug reproducible and an upgrade a deliberate act. When you do upgrade, go one minor version at a time, never skip.

Write setup as code. Put collection creation in a script with an exists guard, not in a notebook cell you ran once. Anyone, including you in three weeks, can recreate the same schema with one command.

Declare your schema. List properties and types explicitly. It documents the data and stops typos from becoming columns.

Count after every import. aggregate.over_all(total_count=True) against len(your_list) catches silent partial failures early.

Keep the original text outside Weaviate too. Treat the database as an index you can rebuild, not as the only copy. When you change embedding models you will need the source data.

Close your clients. One with block per script, one long-lived client per service.

Keep secrets out of code. Read keys from environment variables. Use .env files for local work and add them to .gitignore.

Set limit every time. It makes behaviour independent of server configuration.

Try hybrid before you tune anything. For text search it is the best default. Tune alpha against ten real queries before you touch index settings.

Evaluate retrieval, not just answers. If a RAG answer is bad, look at the retrieved objects first. Print them. Most of the time the search returned the wrong passages. The RAGAS guide shows how to measure this systematically.

Look at the logs. docker compose logs weaviate is the first stop for module and vectorizer errors, which the Python client often reports vaguely.

Read the release notes before upgrading. The deprecation table in the docs is short and saves hours. Older tutorials referring to removed names are the commonest source of confusing errors.

Practice cards

Work these in order, and write the answer before checking.

Card 1. You run docker compose up -d, and curl localhost:8080/v1/.well-known/ready returns a connection error, but a few seconds later returns 200. Is something wrong? No. The node needed a moment to start. Poll the readiness endpoint instead of assuming failure.

Card 2. Your script runs client.collections.create("article", ...) and fails. What is wrong? Collection names must start with an uppercase letter. Use Article.

Card 3. near_text returns objects with distance of 0.12 and 0.45. Which is the better match? 0.12, because lower distance is closer.

Card 4. A search with bm25 finds nothing for "reimbursement", though an article says "refund". Why, and what would you try? BM25 matches the literal words. Use near_text or hybrid, which can match by meaning.

Card 5. You changed the embedding model name in your script and ran the create again. What happens? If the collection exists, you get an "already exists" error. The vectorizer cannot be changed on an existing collection, so you create a new collection and copy the data.

Card 6. You import 5,000 objects in a loop of data.insert calls and it is very slow. What do you change? Use data.ingest, which uses server-side batching.

Card 7. A filter Filter.by_property("status").equal("on hold") returns objects with the status "hold". Why? Text is tokenized into words by default, so matching is on tokens. Create the property with Tokenization.FIELD if you need whole-value matching.

Card 8. The Weaviate container logs a connection error to localhost:11434. Fix it. Change the collection's api_endpoint to http://ollama:11434. The collection will need to be recreated, since the vectorizer configuration is fixed at creation.

Card 9. A coworker's tutorial code starts with client = weaviate.Client("http://localhost:8080"). What do you tell them? That is the removed v3 client. Use weaviate.connect_to_local() and client.collections.use(...).

Card 10. You want to remove every article tagged draft. What is the safe procedure? Run the same filter through fetch_objects to see what matches, then call delete_many(where=...).

A short debugging routine

When something fails and you do not know why, work through the layers in order. It takes five minutes and finds the problem most of the time.

1. Is the server up? docker compose ps, then curl -i http://localhost:8080/v1/.well-known/ready. Not 200 means look at docker compose logs weaviate.
2. Which version and modules? curl http://localhost:8080/v1/meta. Confirm the version matches what you expect and that the module you use is listed.
3. Can Python reach both ports? client.is_ready() after connect_to_local(). A gRPC error here is the port problem.
4. Does the collection look right? curl http://localhost:8080/v1/schema. Check the property names, types and vectorizer.
5. Is the data there? aggregate.over_all(total_count=True) and a fetch_objects(limit=3) to look at real objects.
6. Does a simple search work? Try bm25 first, which needs no model. If it works and near_text fails, the problem is the vectorizer connection.
7. Is the model reachable from Weaviate? For Ollama, docker compose exec weaviate has few tools in it, so test from the logs instead: a failed embedding call is logged with the endpoint it tried.

Splitting the problem this way tells you whether you have a server problem, a client problem, a schema problem or a model problem. Most beginners lose time by changing things in all four at once.

Habits for working with embeddings

A few points that are about embeddings rather than Weaviate, because they decide whether your search feels good.

Keep chunks a sensible size. A vector summarises the text it was made from. A whole 40-page document in one object gives one blurry vector; a single sentence gives a vector with too little context. Splitting long documents into paragraph-sized objects, each with a reference to its source, usually retrieves better.

Vectorize what people search for. By default all text properties are combined into the vector. If an object has a long boilerplate property, the signal is diluted. Later you can restrict which properties feed the vector, but even now you can keep boilerplate out of the object.

Test with your own queries. Write ten queries that real users would type, and note what you expect to come first. Run them after every change. This tiny test set beats any intuition about which setting is better.

Mind the language. Multilingual content, including Arabic alongside English, needs an embedding model that actually handles both. A model trained mostly on English can retrieve poorly in Arabic and still look fine in a demo that uses English queries.

Remember data residency. Hosted embedding providers receive the text of every object you insert and every query you run. For personal or regulated data, check your employer's rules first, or use a local model like Ollama so text stays inside your own infrastructure.

Checklist

Before you call a first Weaviate project finished, check each item.

• The compose file pins an exact Weaviate version and publishes both 8080 and 50051.
• A volume is mounted at /var/lib/weaviate, and nobody runs down -v by habit.
• Anonymous access is acceptable only because the server is on localhost.
• Collection creation lives in a script with an exists guard.
• Properties are declared explicitly, with sensible types.
• You chose the vectorizer deliberately and tested it on your own text.
• Ollama, or any other module, is reached by service name from inside Docker.
• Provider keys come from environment variables or headers, not source code.
• Imports use ingest or batching, with deterministic UUIDs, and you check the count afterwards.
• Every query sets limit, and you request the metadata you read.
• You know whether you are reading distance or score.
• The client is closed with a with block or close().
• You can explain, in one sentence, why you chose vector, keyword or hybrid search.
• You looked at the retrieved objects before blaming the language model.
• You have a way to rebuild the collection from source data.

Mid-level takes these further. Expect batching with error handling and rate limits, named vectors, hybrid and filter tuning, multi-tenancy, quantization, aliases for zero-downtime reindexing and the async client. For the container side, revisit the Docker guide, and for a local model server the Ollama guide.

This is part two of three. It picks up where Beginner ended and takes every topic from there further, then adds the machinery you have not met yet. The guide targets Weaviate 1.39.x and the Python client v4.23. Where you already know the basics, we go straight to the depth.

Where this picks up

Each section starts with the problem it solves, and most end with a Try it you can do on a real project in a few minutes. If you have not read the Beginner guide, start there. The neighbouring guides in this catalogue cover the same ground from other angles: Qdrant, Milvus, Pinecone and pgvector are the usual alternatives, and LlamaIndex and LangChain are the usual callers.

What is on disk: shards, stores and indexes

Most Weaviate surprises at work come from a wrong picture of where data lives. At Beginner you thought in collections and objects. To predict memory, disk and latency you need one more layer: the shard.

A collection is split into one or more shards, and each shard lives on one node. A shard is a self-contained database. It holds three things: an object store (the properties and metadata of every object), an inverted index (the keyword and filter structure), and a vector index. The object store and the inverted index are built on an LSM tree, a write-optimised structure in which new data lands in a memory table and a write-ahead log, is flushed to immutable segment files, and is periodically merged by compaction. By default these segment files are memory-mapped (PERSISTENCE_LSM_ACCESS_STRATEGY defaults to mmap), which means the operating system decides how much of them stays in RAM.

The vector index is the expensive part. With the default HNSW index, the graph lives in memory, and so, by default, do the vectors needed to navigate it. That is why the sizing rule of thumb is about memory: a 768-dimension float32 vector is about 3 KB before graph overhead, so a million of them is already several gigabytes of RAM. The docs put HNSW at roughly 2 to 12 GB per million vectors depending on dimensions and settings. Everything else in this guide about compression, HFresh and tenant offloading is a way of changing that bill.

Two consequences follow from the shard layout.

The shard count is fixed at creation. You set it with sharding_config=Configure.sharding(desired_count=N). If you do not, it defaults to the node count, or to DEFAULT_SHARDING_COUNT if the cluster sets one. There is no in-place resharding. If you got it wrong, the fix is a new collection and a copy, which is exactly what the alias section below is for.

Disk and memory are watched per node. When a node crosses DISK_USE_WARNING_PERCENTAGE (80) it warns, and at DISK_USE_READONLY_PERCENTAGE (90) its shards turn READONLY and writes fail. The memory equivalents are MEMORY_WARNING_PERCENTAGE and MEMORY_READONLY_PERCENTAGE. A READONLY shard is not corruption. After you free or add disk, you set the shard status back to READY with PUT /v1/schema/{class}/shards/{shard} and the body {"status":"READY"}.

You can see all of this from the outside. GET /v1/nodes reports each node's status and shard count, and a verbose form (it needs the Permissions.Nodes.verbose permission under RBAC) lists shards and object counts per collection.

curl -s http://localhost:8080/v1/nodes | python -m json.tool | head -40
curl -s "http://localhost:8080/v1/nodes?output=verbose" | python -m json.tool | head -60

The HNSW index in practice

HNSW (hierarchical navigable small world) is a layered graph. Each vector is a node linked to its near neighbours, and a search starts at the top sparse layer and walks greedily downward, so the work grows roughly logarithmically with the number of objects. Three parameters control the trade between recall (the fraction of true nearest neighbours you actually find), speed and memory.

maxConnections (default 32) is how many neighbours each node keeps. More links mean better recall and a bigger graph. efConstruction (default 128) is how wide the search is while the graph is being built. Higher values build a better graph and slow imports. ef is how wide the search is at query time. Its default is -1, which means Weaviate chooses it dynamically: ef = min(max(dynamicEfMin, limit * dynamicEfFactor), dynamicEfMax) with defaults of 100, 8 and 500.

That last formula explains a symptom that confuses people: recall changes when you change limit. With ef=-1, asking for 10 results searches with an ef of 100, while asking for 100 results searches with an ef of 500. If an evaluation harness fetches 10 and production fetches 50, they are testing different search effort. For repeatable behaviour set a fixed ef.

Which of these can you change after the fact? The docs split them cleanly. maxConnections, efConstruction and the distance metric are fixed at creation, because they shape the graph that already exists. ef, the dynamicEf* values, flatSearchCutoff, vectorCacheMaxObjects, cleanupIntervalSeconds and filterStrategy can be updated, and so can turning quantization on. You update them through collection.config.update(...):

from weaviate.classes.config import Reconfigure

articles = client.collections.use("Article")
articles.config.update(
    vector_config=Reconfigure.Vectors.update(
        name="default",
        vector_index_config=Reconfigure.VectorIndex.hnsw(ef=128),
    )
)

Weaviate offers three other index types worth knowing. The flat index is brute force: no graph, tiny memory footprint, exact results, and it suits small collections and per-tenant data where each tenant has few objects. The dynamic index starts flat and switches to HNSW once a collection passes a threshold (10,000 objects by default). It is still experimental, it needs ASYNC_INDEXING=true, and the switch is one-way. HFresh, generally available since 1.38, is a disk-based index inspired by SPFresh: the postings live on disk and a small in-memory HNSW index covers the centroids. It has built-in 1-bit RQ and supports only cosine and l2-squared. Its documented defaults are maxPostingSizeKB 48, replicas 4 (immutable) and searchProbe 256, which 1.39 raised from an earlier default. Reach for HFresh when the HNSW memory bill is the problem and you can accept a different latency profile, and test recall on your own data first.

Quantization: paying less for memory

When the memory bill from the last section is the constraint, compression is the lever. Weaviate supports four quantizers: rotational (RQ), product (PQ), binary (BQ) and scalar (SQ). The docs recommend RQ as the default choice, and the reasons are practical. It needs no training step, because it rotates the vector and then quantizes each component, so it works from the first object. At 8 bits it gives about 4x compression at a documented 98 to 99 percent recall. At 1 bit it gets close to 32x. In 1.39 a 4-bit variant is in preview (about 8x), HNSW only: a flat index rejects it with the error RQ bits must be either 1 or 8.

You choose a quantizer in the index configuration of a vector:

from weaviate.classes.config import Configure, Property, DataType, VectorDistances

client.collections.create(
    "Doc",
    vector_config=Configure.Vectors.text2vec_openai(
        vector_index_config=Configure.VectorIndex.hnsw(
            distance_metric=VectorDistances.COSINE,
            quantizer=Configure.VectorIndex.Quantizer.rq(),   # 8-bit by default
        ),
    ),
    properties=[Property(name="body", data_type=DataType.TEXT)],
)

Use rq(bits=1) for the aggressive option. A new collection has no quantization unless you configure it or the cluster sets DEFAULT_QUANTIZATION (for example rq-8, rq-1 or rq-4; the default is none). Setting it cluster-wide is a good platform default because it means nobody forgets.

Two rules matter operationally. First, you can turn quantization on for an existing collection (with Reconfigure.VectorIndex.Quantizer.rq() inside Reconfigure.Vectors.update(...)), but once a collection is quantized, you cannot turn it off. Treat enabling it as a one-way change and rehearse it on a copy. Second, compression is lossy, so measure recall on your data before and after, using the baseline from the previous section. Higher compression usually wants a rescoring step: a candidate pool is found with compressed vectors and re-ranked with fuller precision. The 4-bit option exposes this as rescore_limit.

Ingesting data at scale

Single inserts are fine for tests. Real loads need batching, and the way Weaviate batches changed recently, so this is a place where old tutorials mislead.

The current way is server-side batching, generally available since 1.36 and supported by the Python client from 4.20. The client opens a stream, you add objects, and the server tells the client how fast to go, so you no longer hand-tune batch sizes against a variable-speed vectorizer. In the collection API it looks like this:

from weaviate.util import generate_uuid5

articles = client.collections.use("Article")

with articles.batch.stream() as batch:
    for row in rows:
        batch.add_object(
            properties=row,
            uuid=generate_uuid5(row["external_id"]),
        )
        if batch.number_errors > 10:
            print("too many errors, stopping")
            break

failed = articles.batch.failed_objects
print(f"{len(failed)} objects failed")

The older client-level helpers (client.batch.fixed_size(batch_size=200), client.batch.dynamic() and client.batch.rate_limit(requests_per_minute=600)) still exist. client.batch.experimental() is deprecated since 4.20, so remove it from any code you inherit. For a one-shot import of a list you already hold in memory, collection.data.ingest(...) does the same server-side batching in a single call.

Four practices separate a reliable import from a flaky one.

Use deterministic IDs. generate_uuid5 turns a stable key into a UUID, so re-running the same import overwrites rather than duplicates. Idempotency is what lets you retry a failed job safely. Without it, a crashed import that you re-run leaves you with two copies of the first half.

Check errors; the batch does not raise. Individual objects can fail (a type mismatch, a vectorizer error) while the batch carries on. Inspect batch.number_errors during the run and failed_objects afterwards, and fail your job if the count is not zero. Silent partial imports are the most common data quality bug in these pipelines.

Respect the vectorizer's rate limit. If Weaviate calls an embedding provider for each object, the provider's limit is your real throughput ceiling, and a 429 response is the sign you hit it. client.batch.rate_limit(requests_per_minute=...) exists for exactly this. Note that unchanged objects are not re-vectorized (REVECTORIZE_CHECK_DISABLED defaults to false), which makes re-imports cheaper.

One batch per thread. The batch API is not thread-safe. If you parallelize, give each worker its own batch context, not a shared one.

For large payloads there is a message-size ceiling: GRPC_MAX_MESSAGE_SIZE defaults to about 100 MB. If you send a huge insert_many you may see Sent message larger than max. The fix is to use ingest or batch.stream(), or to raise the limit on the server. For CPU-bound imports, ASYNC_INDEXING=true lets the server accept writes faster and build the vector index in the background.

Designing the collection: properties, tokenization and named vectors

Schema decisions are cheap on day one and expensive on day 200, because several of them are immutable. A few deserve deliberate thought.

Property flags decide which indexes exist. Each property can have indexFilterable (for filtering), indexSearchable (for BM25) and indexRangeFilters (for range queries on numbers and dates). Every index you leave on costs disk and write time. A large text field you only ever display, such as a raw HTML body that you do not search, should have its indexes off. The docs also allow dropping an inverted index later (generally available since 1.36), so you are not locked in, but turning them off at creation is cheaper.

Tokenization decides what "match" means. For text properties the default is word: text is lowercased and split on non-alphanumeric characters. That is right for prose and wrong for identifiers. A SKU like AB-123/x becomes several tokens, so an equality filter matches more loosely than you expect. For exact matching of codes, emails and IDs use field, which indexes the entire value as one token. Other choices are lowercase, whitespace, trigram (useful for substring-like matching) and language-specific tokenizers (gse, kagome_ja, kagome_kr). Tokenization is set per property at creation and is immutable in the stable API (reindexing a property is still in preview), which is the strongest reason to decide it carefully. For Arabic content, test your tokenization on real text before committing: the default tokenizer splits on word boundaries, but the effect on morphology-rich text is something to measure with queries, not assume.

Named vectors let one object carry several embeddings. A product might have a vector of its title, another of its description and a third self-provided from an image model. In 1.39 syntax:

from weaviate.classes.config import Configure, Property, DataType, VectorDistances

client.collections.create(
    "Product",
    vector_config=[
        Configure.Vectors.text2vec_openai(
            name="title",
            source_properties=["title"],
            vector_index_config=Configure.VectorIndex.hnsw(
                distance_metric=VectorDistances.COSINE,
                quantizer=Configure.VectorIndex.Quantizer.rq(),
            ),
        ),
        Configure.Vectors.self_provided(
            name="custom",
            vector_index_config=Configure.VectorIndex.flat(),
        ),
    ],
    properties=[
        Property(name="title", data_type=DataType.TEXT),
        Property(name="sku", data_type=DataType.TEXT, tokenization="field"),
    ],
)

products = client.collections.use("Product")
res = products.query.near_text(query="red running shoes", target_vector="title", limit=5)

Once a collection has named vectors, queries must say which one to use with target_vector=. Each named vector has its own index configuration, so you can compress one and leave another exact. Multi-vector (ColBERT-style late interaction) embeddings, where one object has many vectors for a single embedding, use Configure.MultiVectors.*.

Notice the API names. If you are reading older code you will see vectorizer_config=, Configure.Vectorizer.*, Configure.NamedVectors.* and client.collections.get(...). These are deprecated since client 4.16. The current forms are vector_config=, Configure.Vectors.*, Configure.MultiVectors.* and client.collections.use(...). A bring-your-own-vector collection is Configure.Vectors.self_provided(). Python client v3 syntax (weaviate.Client(...), client.query.get(...)) is no longer supported, so any snippet that uses it is dead code.

Auto-schema is on by default. AUTOSCHEMA_ENABLED defaults to true, which means a typo in a property name at insert time silently creates a new property. In production, turn it off so that schema changes happen only through deliberate code.

Hybrid search and score control

Vector search finds meaning, keyword search finds exact terms, and users type both kinds of query. Hybrid search runs both and fuses the results. You already know alpha: 1 is pure vector, 0 is pure keyword, 0.5 is the middle. The mid-level questions are how the fusion works and how to steer it.

The default fusion is RELATIVE_SCORE, which has been the default since 1.24. It normalizes the BM25 scores and the vector scores onto a common scale, then combines them with the alpha weights. The alternative is RANKED, which ignores score magnitudes and combines by rank position. Relative score is usually better because it preserves how much better one result is than the next, while ranked fusion can be more stable when the two score distributions are very different. Neither is correct in general; you choose by testing on your queries.

from weaviate.classes.query import HybridFusion, MetadataQuery, BM25Operator

res = articles.query.hybrid(
    query="postgres connection pool",
    alpha=0.6,
    fusion_type=HybridFusion.RELATIVE_SCORE,
    query_properties=["title^2", "body"],
    max_vector_distance=0.4,
    bm25_operator=BM25Operator.or_(minimum_match=2),
    limit=10,
    return_metadata=MetadataQuery(score=True, explain_score=True),
)
for o in res.objects:
    print(o.uuid, o.metadata.score)
    print(o.metadata.explain_score)

Each knob has a purpose. query_properties restricts the keyword half to chosen properties and lets you boost one (title^2 doubles its weight; since 1.38 fractional boosts like title^2.5 work). max_vector_distance drops vector candidates that are too far away, which stops weak semantic matches from padding the list. bm25_operator controls how multiple query words combine: and_() requires every term, or_(minimum_match=2) requires at least two, and 1.39 adds and_cross(), a cross-property AND. explain_score returns a breakdown of where the score came from, and it is the first thing to reach for when someone asks why a result ranks where it does.

Filters: strategy, cost and the empty-result trap

A filter restricts which objects a search may return. It looks trivial and hides the most subtle behaviour in the system, because the filter and the vector index are two different structures that must cooperate.

The naive approaches both fail at the extremes. Filtering after the vector search (post-filtering) can return fewer than limit results, even none, if the top candidates all fail the filter. Filtering before and then searching only the matches is exact but slow when most objects match. Weaviate's HNSW chooses between strategies. If the filter matches very few objects (below flatSearchCutoff, default 40,000), it scans them directly with brute force. Otherwise it searches the graph while applying the filter. The graph-walking behaviour is set by filterStrategy, whose default has been acorn since 1.34. ACORN explores the graph with filter awareness so it can reach matches that sit behind non-matching neighbours; sweeping is the older alternative.

You rarely need to change this, but the knowledge pays off in diagnosis. A very restrictive filter combined with a vector search is fast and exact because of the cutoff. A medium-selectivity filter exercises the graph path. If recall looks poor only on filtered queries, that is where to look.

from weaviate.classes.query import Filter

f = (
    Filter.by_property("category").equal("billing")
    & Filter.by_property("published").greater_or_equal("2026-01-01T00:00:00Z")
    & Filter.by_property("tags").contains_any(["refund", "invoice"])
)
res = articles.query.near_text(query="late payment fees", filters=f, limit=5)

Combine conditions with & and |, or with Filter.all_of([...]), Filter.any_of([...]) and Filter.not_(...). Besides property filters there are metadata filters (Filter.by_id(), Filter.by_creation_time(), Filter.by_update_time()), reference filters (Filter.by_ref("link").by_property(...)) and property-length filters (Filter.by_property(p, length=True)). Dates are RFC 3339 strings. Null checks need the property's null state indexed.

Why does a filter sometimes return nothing when you are sure the data exists? Four causes cover almost every case. The property's tokenization splits your value differently than you assumed (the SKU example from the schema section). The property has indexFilterable turned off. You are comparing a date as a plain string without a timezone. Or you are in a multi-tenant collection and forgot with_tenant(...), so you are querying an empty or different tenant. Check them in that order.

Filters also apply to deletes. A filtered batch delete matches objects by condition, and two behaviours matter. A delete request whose match fields are missing returns 422 Unprocessable Entity as of 1.39 (it used to be a 500). And as of the 1.39.7 patch release, one filtered delete resolves at most QUERY_MAXIMUM_RESULTS objects per request, so a very large delete-by-filter may need several passes. Always run the same filter as a query with a count first, so you know what you are about to delete.

articles.data.delete_many(where=Filter.by_property("category").equal("draft"))

Reranking, diversity, boosting and grouping

Retrieval gets candidates; these features shape the final list. Four of them matter at work, and they apply in a fixed order, so it helps to know what runs when.

Reranking reorders the retrieved results with a model that reads query and document together, which is more accurate than comparing precomputed vectors and more expensive per document. It needs a reranker module on the collection (reranker_config=Configure.Reranker.cohere() is the documented example; Jina, NVIDIA and Voyage rerankers also exist) and is applied per query with rerank=Rerank(prop="body", query="..."). The standard pattern is to retrieve generously (say 50), rerank, and show 5. The cost scales with the candidate count, so do not rerank 500.

Boost, which graduated to general availability in 1.39 after a gRPC-only preview in 1.38, rescores results at query time using signals other than text similarity: a filter match, a numeric property, a time decay (favour recent documents) or a numeric decay. Conditions combine with Boost.blend. The documented scoring is (1 - weight) * primary + weight * boost, where the outer weight defaults to 0.5, each condition's weight defaults to 1.0, and depth, the number of top candidates considered, defaults to 100 (QUERY_BOOST_DEFAULT_DEPTH). It works across hybrid, bm25 and the near_* searches, runs before rerankers, and accepts at most 20 conditions per query. Because the Python signatures for boost are detailed, take them from the current docs page rather than from memory. The idea is what to carry: "freshness" and "popularity" should be expressed as boosts, not baked into embeddings.

MMR diversity selection, generally available in 1.39, solves the case where the top 10 are ten near-duplicates. It selects results that are relevant but different from each other:

from weaviate.classes.query import Diversity

res = articles.query.near_text(
    query="how to rotate API keys",
    limit=10,
    diversity_selection=Diversity.mmr(limit=4, balance=0.3),
)

balance runs from 0.0 (most diverse, the default) to 1.0 (pure relevance). It needs Python client 4.23.0 or newer, works only on HNSW, and is not available for bm25 or multi-vector collections. Be careful with older blog posts: the 1.37 preview used a different spelling (selection=Diversity.MMR(...)) that is outdated.

Group by returns results organized under a property, for instance at most 2 passages from each of 3 documents:

from weaviate.classes.query import GroupBy

res = articles.query.near_text(
    query="data retention policy",
    limit=20,
    group_by=GroupBy(prop="source_doc", objects_per_group=2, number_of_groups=3),
)
for name, group in res.groups.items():
    print(name, [o.uuid for o in group.objects])

Generative search and model-provider keys

Weaviate can pass retrieved objects to a language model in the same call, which turns a query into a RAG step. You met single_prompt (one generation per object, with {property} interpolation) and grouped_task (one generation across all results). At mid level the questions are about configuration and control.

You set a collection default with generative_config=Configure.Generative.openai() (and equivalents for Anthropic, Google, Cohere, Mistral, DeepSeek and Ollama among others), and since 1.30 you can override it per query:

from weaviate.classes.generate import GenerativeConfig

res = articles.generate.near_text(
    query="refund policy for annual plans",
    limit=4,
    grouped_task="Answer the question using only these passages. Cite titles.",
    generative_provider=GenerativeConfig.ollama(
        api_endpoint="http://ollama:11434",
        model="llama3.2",
    ),
)
print(res.generative.text)
for o in res.objects:
    print(o.properties["title"])

The per-object output is in o.generative.text and the grouped output in res.generative.text. The override matters operationally: you can run an experiment with a different model without changing the collection, and you can route cheap queries to a small model.

Where the API keys live is a security decision. A model-provider key travels in a request header (X-OpenAI-Api-Key, in general X-<Provider>-Api-Key) or sits in a server environment variable (OPENAI_APIKEY). If both exist, the header wins. The key is never stored in the collection configuration, which is good: exporting or backing up a collection does not leak it. In a service that talks to Weaviate on behalf of many users, put the key in the server's environment (from your secret manager) rather than shipping it from every client. For a client that must send it per request, pass headers when connecting:

import os
import weaviate
from weaviate.classes.init import Auth

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
    headers={"X-OpenAI-Api-Key": os.environ["OPENAI_API_KEY"]},
)

A few failure patterns recur. A 401 from the vectorizer means the provider key is missing or wrong. A 429 means the provider is throttling you, and the cure is to slow imports with rate_limit. "Connection refused" to Ollama from inside Docker nearly always means you wrote localhost, which inside the Weaviate container is the container itself; use the Compose service name (http://ollama:11434) or host.docker.internal. If your data is regulated, remember that an API-based module sends text to a third party. Since 1.33 API-based modules are enabled by default, and API_BASED_MODULES_DISABLED=true turns them off, which is a useful guard for Gulf or Egyptian employers with strict residency rules. Pair it with MODULES_VALIDATE_BASE_URL=true to limit which endpoints modules may call, and consider a local provider such as Ollama when data may not leave your network. Teams that route many providers behind one gateway often put LiteLLM in front of generative calls.

Multi-tenancy: one shard per tenant

If you build a product where each customer's data must be separate, you have two bad options and one good one. One collection per customer explodes the schema and the Raft metadata. One shared collection with a customer_id filter is simple but gives no isolation, and a forgotten filter leaks data. Multi-tenancy gives each tenant its own shard inside a single collection, so isolation is structural, deletion of a tenant is cheap, and a cold tenant can leave memory.

You enable it at creation and then address tenants explicitly:

from weaviate.classes.config import Configure
from weaviate.classes.tenants import Tenant, TenantActivityStatus

docs = client.collections.create(
    "Docs",
    multi_tenancy_config=Configure.multi_tenancy(
        enabled=True,
        auto_tenant_creation=True,
        auto_tenant_activation=True,
    ),
)

docs.tenants.create([Tenant(name="acme"), Tenant(name="globex")])

acme = docs.with_tenant("acme")
acme.data.insert({"title": "Q3 plan"})
hits = acme.query.near_text(query="plan", limit=3)

docs.tenants.update([Tenant(name="globex", activity_status=TenantActivityStatus.INACTIVE)])
docs.tenants.remove(["globex"])   # deletes that tenant's data

Tenant names are 4 to 64 characters of letters, digits, underscore and hyphen. Every read and write in a multi-tenant collection must go through with_tenant(...), which is the structural protection: a query cannot cross tenants by accident.

A tenant is in one of three states, renamed in 1.26. ACTIVE is loaded and queryable. INACTIVE is on local disk but not loaded, so it costs no memory and cannot be queried until activated. OFFLOADED is moved to cloud storage and needs the offload-s3 module (configured with OFFLOAD_S3_BUCKET and related variables). The old names HOT, COLD and FROZEN are deprecated, so translate them when you read older material. The cost story follows directly: a SaaS with ten thousand tenants of which five percent are active in a given week keeps ninety-five percent of them out of memory.

Two convenience features carry caveats. Auto tenant creation creates a missing tenant, but only on batch insert (available since 1.25), so a single insert to a tenant that does not exist still fails. Auto activation wakes an inactive tenant on access, so a job that touches every tenant undoes your memory savings.

For access control, RBAC (below) has tenant-level permissions with wildcards, such as Permissions.tenants(collection="Docs", tenant="acme*", read=True), so a service identity for one customer's integration can be limited to that customer's tenants. Namespaces, a preview in 1.38 and 1.39 with general availability planned for 1.40, will add isolation between users of a shared cluster one level above tenants; until they are stable, do not build on them.

Changing a live schema: updates, aliases and blue/green reindexing

Production schemas change. What you may change in place is limited, so you need a routine for the rest.

In place, you can add a property (collection.config.add_property(Property(...))), update the mutable index settings shown earlier, change BM25 parameters, adjust the description, and turn quantization on. You cannot change the distance metric, efConstruction, maxConnections, the vectorizer, a property's data type or tokenization, or the shard count. Changing the replication factor through the config is also not the path; use replica movement. Note that adding a property does not backfill it for old objects, so they simply have no value until you update them.

For everything immutable, the pattern is blue/green reindexing with an alias. An alias, available since 1.32, is a second name for a collection. Your application always queries the alias, so the physical collection behind it can change without a deploy:

# 1. Create the new collection with the corrected settings
client.collections.create(
    "Articles_v2",
    vector_config=Configure.Vectors.text2vec_openai(
        vector_index_config=Configure.VectorIndex.hnsw(
            quantizer=Configure.VectorIndex.Quantizer.rq()
        )
    ),
    sharding_config=Configure.sharding(desired_count=3),
    properties=[Property(name="title", data_type=DataType.TEXT)],
)

# 2. Backfill from the old collection
old = client.collections.use("Articles_v1")
new = client.collections.use("Articles_v2")
with new.batch.stream() as batch:
    for obj in old.iterator(include_vector=False):
        batch.add_object(properties=obj.properties, uuid=obj.uuid)

# 3. Point the alias at the new collection, atomically from the app's view
client.alias.update(alias_name="Articles", new_target_collection="Articles_v2")

The very first time, create the alias with client.alias.create(alias_name="Articles", target_collection="Articles_v1"), and later use client.alias.list_all() and client.alias.delete(...) to inspect and remove aliases. Two details. Deleting a collection does not delete aliases that point at it, so a stale alias can point at nothing; clean up as part of the cutover. And the backfill above lets the new collection's vectorizer re-embed text. If you want to copy existing vectors instead (for example when the vectorizer has not changed and you want to avoid embedding cost), fetch them with include_vector=True and pass them through, taking care that the vector names and dimensions match.

This is also the answer to a cluster-level mistake, such as the wrong shard count, and to adopting a new embedding model, since vectors from different models are not comparable. Treat "embedding model change" as a reindex, always.

Authentication, RBAC and secrets

Out of the box Weaviate allows anonymous access. That is right for a laptop and wrong anywhere reachable by anything else. The production baseline is: turn anonymous access off, require an identity, and authorize with roles.

The simplest identity is an API key. In a Docker Compose environment block the baseline looks like this:

      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'false'
      AUTHENTICATION_APIKEY_ENABLED: 'true'
      AUTHENTICATION_APIKEY_ALLOWED_KEYS: 'admin-key,app-key'
      AUTHENTICATION_APIKEY_USERS: 'admin,app'
      AUTHORIZATION_RBAC_ENABLED: 'true'
      AUTHORIZATION_RBAC_ROOT_USERS: 'admin'
      AUTHENTICATION_DB_USERS_ENABLED: 'true'

The keys and users lists are matched by position. With RBAC enabled you must name at least one root user, or Weaviate refuses to start; that is a deliberate safeguard so a cluster cannot boot into a state where nobody can administer it. The variable is AUTHORIZATION_RBAC_ENABLED. The older AdminList scheme (AUTHORIZATION_ADMINLIST_*) still exists, but RBAC is the modern path and the one to teach a new colleague.

Static keys in environment variables are fine for bootstrapping and poor for operating, because rotating one means restarting the cluster. DB users, available since 1.30 once you set AUTHENTICATION_DB_USERS_ENABLED=true, are created and rotated at runtime through the API:

from weaviate.classes.rbac import Permissions

# connected as a root user
api_key = client.users.db.create(user_id="search-service")   # returns the key once
client.roles.create(
    role_name="article-reader",
    permissions=[
        Permissions.collections(collection="Article*", read_config=True),
        Permissions.data(collection="Article*", read=True),
    ],
)
client.users.db.assign_roles(user_id="search-service", role_names=["article-reader"])
# later
new_key = client.users.db.rotate_key(user_id="search-service")

Store the returned key in your secret manager the moment you receive it; Weaviate does not show it again. Two built-in roles exist, root and viewer, and neither can be modified. Everything else is a role you define from permission builders: collections, data, tenants, backup, cluster, nodes, roles, users, replicate, aliases and OIDC groups. Roles can also be changed with add_permissions and remove_permissions, inspected with get, list_all and get_user_assignments, and removed with delete. Collection names in permissions accept wildcards, which is how you scope a service to Article* or a tenant pattern like acme*.

Secrets handling follows from this. Keep admin keys, provider keys and backup credentials in a manager such as Vault or your cloud's secret store, inject them as environment variables or Kubernetes Secrets, and never commit them to a Compose file in git. Cloud backup credentials should come from workload identity or an assumed role (S3 role assumption arrived in 1.37) rather than long-lived keys.

Running it: Docker Compose, Helm and health probes

Development uses Docker Compose; production usually means Kubernetes. The configuration surface is the same, which is one of the advantages of an environment-variable driven server.

For a single-node Compose deployment, the essentials are pinned versions, both ports, a persistent volume and explicit settings:

services:
  weaviate:
    command: [--host, 0.0.0.0, --port, '8080', --scheme, http]
    image: cr.weaviate.io/semitechnologies/weaviate:1.39.7
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      CLUSTER_HOSTNAME: 'node1'
      AUTOSCHEMA_ENABLED: 'false'
      PROMETHEUS_MONITORING_ENABLED: 'true'
volumes:
  weaviate_data:

Pin the image to an exact patch, never latest, so a restart cannot upgrade you. The volume at /var/lib/weaviate is the database; docker compose down keeps it and docker compose down -v deletes it, a mistake that has cost people real data. Port 50051 matters as much as 8080: the v4 clients require gRPC, and the classic connection error gRPC health check could not be completed almost always means that port is not published, or grpc_host and grpc_secure are wrong, or on Kubernetes the gRPC service is not enabled. Using skip_init_checks=True hides the symptom and should only be a temporary diagnostic. If you have not used Docker in depth, the Docker guide covers health checks and volumes.

On Kubernetes the deployment is the official Helm chart:

helm repo add weaviate https://weaviate.github.io/weaviate-helm
helm repo update
helm show values weaviate/weaviate > values.yaml
helm upgrade --install weaviate weaviate/weaviate \
  --namespace weaviate --create-namespace --values ./values.yaml

It needs Kubernetes 1.23 or newer, Helm 3, and a persistent volume provisioner that supports ReadWriteOnce (the docs advise avoiding file.csi.azure.com on Azure). In values.yaml, set grpcService.enabled: true, choose its type for how clients reach it, set replicas, storage class, resources, authentication and the env block. Keep that file in git and apply it with helm upgrade --install, which is idempotent. For templating and chart conventions see the Helm and Kubernetes guides; if your platform team provisions clusters with code, the Terraform guide is the neighbour.

The deployment is a StatefulSet. Wire the Kubernetes probes to the documented endpoints, using ready for readiness and live for liveness, and give startup generous time so a node loading many shards is not killed mid-load.

curl -i http://localhost:8080/v1/.well-known/live     # 200 while the process is alive
curl -i http://localhost:8080/v1/.well-known/ready    # 200 when the node can serve
curl -s http://localhost:8080/v1/meta                 # version and enabled modules

Replication, consistency and backups

A single node is a single point of failure. Two mechanisms matter for availability, and a third for recovery.

Replication keeps copies of each shard on several nodes. You set the factor at collection creation:

from weaviate.classes.config import Configure, ReplicationDeletionStrategy

docs = client.collections.create(
    "Orders",
    replication_config=Configure.replication(
        factor=3,
        async_enabled=True,
        deletion_strategy=ReplicationDeletionStrategy.TIME_BASED_RESOLUTION,
    ),
)

Cluster metadata uses Raft, but object data uses leaderless, Dynamo-style replication: any node can coordinate a read or write, and you choose the consistency level per request, ONE, QUORUM (the default) or ALL. The rule to remember is that reads are strongly consistent when the read and write replica counts sum to more than the replication factor. With three replicas, quorum reads plus quorum writes give you that. ONE is faster and can return stale data right after a write on another node. You select it with collection.with_consistency_level(...).

Replicas that fall out of sync are repaired by two mechanisms: repair-on-read, and asynchronous replication, which compares Merkle hash trees between replicas. Since 1.38 async replication is cluster-wide, generally available and on by default when the factor is above one. The per-collection asyncConfig settings of 1.36 were removed, and the cluster-level knobs are ASYNC_REPLICATION_SCHEDULER_WORKERS (default 10) and ASYNC_REPLICATION_DISABLED. If you copied configuration from 1.36-era material, strip the removed variables. The default conflict strategy for deletions is TimeBasedResolution since 1.36. You can enforce a floor with REPLICATION_MINIMUM_FACTOR so that nobody creates an unreplicated production collection by accident.

Backups are the recovery path, and replication is not a backup: a replicated delete is replicated. Enable a backup module in ENABLE_MODULES (backup-s3, backup-gcs, backup-azure or backup-filesystem) and configure its bucket variables. The filesystem module works on a single node only and is for development and testing. Then:

client.backup.create(
    backup_id="nightly-2026-10-02",
    backend="s3",
    include_collections=["Orders", "Docs"],
    wait_for_completion=True,
)
status = client.backup.get_create_status(backup_id="nightly-2026-10-02", backend="s3")

# restore, on a cluster with the same node count and node names
client.backup.restore(backup_id="nightly-2026-10-02", backend="s3", wait_for_completion=True)

Incremental backups, generally available since 1.37, use incremental_base_backup_id= on create so you do not copy unchanged data every night, but the base backup must stay available. Know the restore constraints before you need them: the target needs the same node count and node names as the source, and RBAC roles and users are not restored by default. Inactive tenants are included in backups, offloaded ones are not. A restore cannot be cancelled once it reaches the FINALIZING stage.

Observability: metrics, logs and the slow query

You cannot tune what you cannot see. Weaviate exposes Prometheus metrics when PROMETHEUS_MONITORING_ENABLED=true, served on port 2112 at /metrics by default (PROMETHEUS_MONITORING_PORT). Setting PROMETHEUS_MONITORING_GROUP=true aggregates metrics across shards, which keeps cardinality sane when you have many tenants, and PROMETHEUS_MONITORING_METRIC_NAMESPACE adds a prefix. A Prometheus scrape of that endpoint and a Grafana dashboard (the weaviate/weaviate-examples repository includes a monitoring example) is the usual setup.

For individual slow queries, three tools help. The slow query log (QUERY_SLOW_LOG_ENABLED, with QUERY_SLOW_LOG_THRESHOLD, default 2 seconds) writes the offenders to the log. Query profiling, a preview since 1.37, is requested per query with MetadataQuery(query_profile=True) and breaks down where time went. explain_score explains ranking rather than latency. Logs are JSON by default (LOG_FORMAT), controlled by LOG_LEVEL, which means a log pipeline can filter them without regular expressions. If a process hangs, sending it SIGQUIT makes the Go runtime dump all goroutine stacks, which is the evidence a bug report needs. The pprof port is 6060.

Upgrades and staying current

Weaviate supports the latest three minor versions: at the time of writing 1.39, 1.38 and 1.37. Older minors stop receiving patches, so a cluster that sits on an old version is accumulating risk. The rules for upgrading are short and strict.

Take a backup first. Upgrade one minor version at a time, to the latest patch of each; skipping minors is the documented cause of failed upgrades and metadata drift. From 1.26 onward, check /v1/cluster/statistics before upgrading to confirm the Raft metadata is in sync across nodes. Read the migration notes for the version you cross. Historic ones worth knowing: 1.25 introduced Raft and needed manual migration on Kubernetes, and chart version 17.0.0 and later means deleting the StatefulSet before upgrading from earlier versions; 1.30 made BlockMax WAND the default and collections created earlier need a migration to benefit.

For the 1.38 to 1.39 step, specific cleanups apply. Automatic HNSW snapshots are generally available in 1.39, so remove the five deprecated environment variables (PERSISTENCE_HNSW_DISABLE_SNAPSHOTS, PERSISTENCE_HNSW_SNAPSHOT_INTERVAL_SECONDS, PERSISTENCE_HNSW_SNAPSHOT_ON_STARTUP, PERSISTENCE_HNSW_SNAPSHOT_MIN_DELTA_COMMITLOGS_NUMBER and PERSISTENCE_HNSW_SNAPSHOT_MIN_DELTA_COMMITLOGS_SIZE_PERCENTAGE); they are ignored and produce a warning. PERSISTENCE_HNSW_MAX_LOG_SIZE (default 500 MiB) still applies. The Search REST API is experimental; on 1.39.0 to 1.39.6 it needs EXPERIMENTAL_REST_SEARCH_ENABLED=true, and the 1.39.7 patch notes suggest the flag is being dropped, but check the docs for your exact version before relying on either. Also note that gRPC-Web is enabled by default under /v1/grpc-web/ on the REST port, and can be disabled with the runtime-config key grpc_web_enabled=false, and the GraphQL API can now be switched off with DISABLE_GRAPHQL.

Putting it all together

Here is the shape of a well-run Weaviate deployment for a support-search feature serving many customers.

The application queries an alias, Docs, never a physical collection. The physical collection, Docs_v3, is multi-tenant with auto tenant creation, so each customer is a tenant and a query cannot cross customers. Its vector configuration uses RQ 8-bit, because the team measured recall against an exact baseline and lost under one point for a quarter of the memory. The ticket_id property uses field tokenization so exact filters behave, display-only properties have their inverted indexes off, and auto-schema is disabled.

The ingestion job uses batch.stream() with UUIDs derived from the ticket ID, so a retry is harmless, and it fails the pipeline if failed_objects is not empty or the final count disagrees with the source. Provider keys come from the secret manager. The search API calls hybrid search at an alpha chosen by a sweep over labelled queries, trims weak matches with max_vector_distance, and uses MMR so the top five are not five copies of one answer.

The cluster runs from Helm with a pinned image, gRPC enabled, three replicas, anonymous access off, RBAC on and DB users per service. Prometheus scrapes the metrics, and alerts fire on latency, errors and disk at 80 percent. Nightly incremental backups go to object storage, and a weekly job restores one into a scratch cluster and runs the labelled queries. Every immutable change goes through the alias cutover, with the old collection kept for a week.

None of that is exotic. Each piece answers a failure that actually happens: the OOM kill, the duplicate import, the leaked tenant, the silent recall loss, the backup that would not restore.

Where you are now

You can explain what a shard holds and use that to estimate memory and disk. You can tune HNSW without guessing, compress with RQ and measure the cost, and import reliably with server-side batching and deterministic IDs. You can steer ranking with hybrid fusion, boosts, diversity and reranking, isolate customers with multi-tenancy, change a live schema through aliases, lock a cluster down with RBAC, and run it with probes, replication and restorable backups.

The senior level takes the platform view: which component breaks first at scale, cost, security, running Weaviate for other teams, and when to choose something else. Before you go, compare notes with Qdrant, Milvus, Pinecone, pgvector and FAISS, because explaining a trade-off against another system is the clearest sign you understand your own.

Sources

• Weaviate documentation
• Release notes
• Weaviate GitHub releases
• Weaviate 1.39 release
• Weaviate 1.38 release
• Weaviate 1.37 release
• Docker installation
• Kubernetes installation
• Environment variables
• Authentication
• Configuring RBAC
• Backups
• Monitoring
• Migration guides
• Python client
• Python client notes and best practices
• Collection operations
• Multi-tenancy
• Collection aliases
• Importing data
• Hybrid search
• Filters
• Generative search
• Vector index concepts
• Vector index configuration reference
• Compression
• Consistency
• Cluster architecture
• Model providers

Part two of three. At this level an interviewer is not checking whether you can create a collection. They want to know whether you can run Weaviate when it misbehaves, defend the choices you made, and say what you would measure before changing anything. The questions below are scenario and trade-off questions, each with a reasoned answer and a note on what separates a strong answer from a weak one. Everything targets Weaviate 1.39 and the Python client v4.

How to answer scenario questions

Interviewers at this level listen for a method more than a fact. A reliable shape is: restate the symptom, name the two or three most likely causes, say what you would check first and why, and say what you would do once you know. Strong candidates also volunteer what they would not change until they had evidence. Weak candidates start changing settings.

Three habits recur in every good answer here. Separate retrieval quality from system health, because "the search is bad" and "the search is slow" have different causes. Reach for a measurement: an exact-search baseline, a labelled query set, a metric. Name the immutable things: if the fix needs a setting that cannot be changed in place, say so and describe the alias-based reindex.

Scenario questions

Search results got worse after we turned on compression. What do you do?

Start by confirming it is real. Compare the compressed collection with an uncompressed copy, or with a flat exact index, on a labelled set of queries, and compute the overlap of the top-k or the hit rate. A feeling that results are worse is not evidence.

If recall did drop, look at which compression was chosen. Rotational quantization at 8 bits is documented at roughly 98 to 99 percent recall and about 4x memory saving, so a large drop there would be surprising and worth investigating elsewhere (did ef change, did the filter strategy matter, did data change?). A 1-bit setting gives close to 32x and is expected to cost more recall. Options are to move back up to 8 bits, to raise the rescoring candidate pool where the setting exposes one, or to keep the compression and accept the loss if it meets the target.

The catch is that quantization cannot be switched off once enabled on a collection. So the real remedy is to create a new collection without it, backfill, and cut over through an alias. A strong answer mentions that last step and says the enabling should have been rehearsed on a copy. A weak answer says "turn it off".

Recall is different in production from our test harness. Why?

The most common cause is ef. With the default ef=-1, Weaviate computes the search width from the requested limit: roughly the larger of 100 and eight times the limit, capped at 500. If the harness asks for 10 results and production asks for 50, they run searches of different effort. Set a fixed ef in both and compare again.

Other causes follow. Production has filters the harness lacks, and filtering interacts with the index strategy. Production data is larger, so the same parameters give lower recall. The harness uses different vectors, perhaps a different embedding model, or the production collection is quantized and the test one is not. A strong answer lists these and says how to isolate each, changing one at a time.

An import job finishes with no error, but a thousand documents are missing. How?

The batch does not raise on individual object failures. Objects can fail on a type mismatch or a vectorizer error while the job carries on, so the job reported success. The fix is to check batch.number_errors during the run, inspect failed_objects afterwards, compare the final object count with the source count, and fail the job on any mismatch.

Then ask why they failed. A 429 from an embedding provider means the import outran its rate limit, and the answer is a rate limiter or the server-side batching stream. A "message larger than max" error means a payload exceeded GRPC_MAX_MESSAGE_SIZE, and the answer is ingest or batch.stream() instead of one giant insert_many. Mention deterministic UUIDs, because they make the re-run of the failed part safe. A strong answer treats the silent partial import as a pipeline defect, not a one-off.

Writes started failing and the logs say shards are read-only. What happened?

Weaviate protects itself. When disk use crosses DISK_USE_READONLY_PERCENTAGE (default 90 percent), or memory crosses its equivalent, the node's shards turn READONLY and writes fail. This is protective, not corruption. First, free or add disk and find out why it filled: tombstones and compaction, a large import, backups written locally, or unbounded growth. Then set the shard status back to READY through PUT /v1/schema/{class}/shards/{shard} with {"status":"READY"}.

The deeper answer is prevention: alert at the 80 percent warning threshold, track growth, and size disks for compaction headroom. A strong candidate also notes that compression, HFresh or tenant offloading change the storage curve and that capacity planning belongs before the incident.

The client says gRPC health check could not be completed. Where do you look?

The v4 clients require gRPC on port 50051, separate from REST on 8080, so a REST curl that works proves nothing. Check in order: is 50051 published by Docker or exposed by the Kubernetes service; on Helm, is grpcService.enabled true; are grpc_host, grpc_port and grpc_secure correct in connect_to_custom; does a proxy or load balancer in front of it speak HTTP/2 correctly; and on managed cloud, did the gRPC URL change after a cluster update.

skip_init_checks=True removes the error and the symptom, so it is a temporary diagnostic and never a fix. A strong answer explains why REST works while gRPC does not, and does not suggest the skip flag as a solution.

Pods are being OOM-killed. What is your sequence?

HNSW holds its graph and, by default, its vectors in memory, so memory is the primary cost. Work through the levers in order of leverage and least disruption: enable compression such as RQ 8-bit for roughly a quarter of the vector memory; set GOMEMLIMIT below the container limit so the Go runtime collects before the kernel kills the process; lower vectorCacheMaxObjects if the vector cache is large; use flat or HFresh for data that suits them; and add nodes or shards. Check docker stats or kubectl top against the metrics for vector_index_size and tombstones to see whether growth is data or garbage.

A strong answer also asks whether the OOM correlates with an import (then ASYNC_INDEXING and batch behaviour matter) or with query load, and notes that adding memory without compression only postpones the problem.

A tenant's data appeared in another customer's results. How did this happen and how do you prevent it?

If the data is in one shared collection with a customer_id filter, one missing filter causes exactly this, and the prevention is structural. Use multi-tenancy, where each tenant is a separate shard and every query must go through with_tenant(...). Add RBAC tenant permissions with wildcards so a service identity can touch only its own tenants. Then test it: an automated test that inserts into tenant A and asserts that a query on tenant B returns nothing.

Strong answers go beyond the feature and mention a regression test and a code review rule, because the failure was a human one.

The vectorizer is returning 401 or 429. How do you tell them apart and fix each?

A 401 means the model provider rejected the credential: the X-<Provider>-Api-Key header or the server's <PROVIDER>_APIKEY environment variable is missing or wrong, and the header wins when both exist. A 429 means the provider is throttling you. Slow imports with client.batch.rate_limit(requests_per_minute=...) or the server-driven batching stream, and remember unchanged objects are not re-vectorized, so re-runs are cheaper. If the error is a connection refused to a local Ollama, the cause is nearly always localhost inside the container; use the Compose service name.

Trade-off questions

Hybrid or pure vector search for our product?

It depends on what users type. Embeddings capture meaning and blur rare exact tokens, so identifiers, error codes and product names are poorly served by pure vector search. Keyword search is exact and blind to paraphrase. Hybrid covers both, at a cost: tuning alpha, choosing the fusion type, and a slightly heavier query. The way to decide is to build a labelled query set that includes both conceptual and identifier queries, sweep alpha, and look at the hit rate.

A strong answer mentions the default fusion (relative score since 1.24, versus ranked), explain_score for diagnosing a ranking, and max_vector_distance to remove weak semantic matches. A weak answer says hybrid is always better.

HNSW, flat, dynamic or HFresh?

HNSW is the default and the best general choice for in-memory, low-latency search, at a memory cost. Flat is brute-force and exact with a tiny footprint, ideal for small collections or per-tenant data. Dynamic starts flat and moves to HNSW past a threshold, but it is still experimental, needs ASYNC_INDEXING, and the switch is one-way. HFresh, generally available since 1.38, is disk-based with in-memory centroids and built-in 1-bit RQ, supports only cosine and L2-squared, and is the choice when HNSW memory is the constraint and you can accept a different latency profile.

The senior-sounding move is to say you would test recall and latency on your own data for the two candidates rather than choose from a blog.

One collection per customer, or multi-tenancy?

One collection per customer multiplies schema and cluster metadata, which is replicated through Raft, so it does not scale to many customers. Multi-tenancy puts one shard per tenant into a single collection: structural isolation, cheap deletion, inactive tenants that leave memory, and offloading to object storage. The cost is that every call must carry the tenant, and that schema is shared across tenants. If tenants need different schemas, or a handful of very large customers need dedicated infrastructure, separate collections or clusters may be right.

Named vectors or separate collections?

Named vectors suit one object with several views: title and body embeddings, or text plus an image vector. Each has its own index config, and you pick one with target_vector. Separate collections are better when the objects are really different things or when lifecycle and access differ. Named vectors cost memory for each vector you keep, so ask whether you will query each one.

Self-hosted or Weaviate Cloud?

Self-hosting gives control over residency, networking and cost at scale, and demands operations skill: upgrades, backups, capacity and on-call. The managed service removes that operational load for a price that depends on dimensions, storage and backups, and offers Shared and Dedicated tiers with different availability targets. In the Gulf or Egypt, residency and the available regions decide it for many employers. A strong answer asks about data sensitivity, team size and traffic pattern, and does not claim terms of any free tier it has not verified.

Why use an alias instead of just naming the collection?

An alias is a second name for a collection. If the application queries the alias, an immutable change (shard count, vectorizer, distance metric, tokenization) becomes: build a new collection, backfill, switch the alias in one call, and keep the old one for rollback. Without an alias, that is a coordinated deploy. Mention that deleting a collection does not delete aliases pointing at it.

Compression: RQ, PQ, BQ or SQ?

The docs recommend RQ. It needs no training and gives about 4x at 8 bits with high recall, or close to 32x at 1 bit. PQ and BQ are older options with different trade-offs, and which one fits depends on the data and recall target. A good answer states the default, the one-way nature of enabling it, and the habit of measuring recall first.

What strong answers cover that weak ones miss

Across all of these, the difference is rarely knowledge of an obscure flag. It is these things.

Quick-fire questions

These are the short ones that fill the gaps between scenarios. Practise saying each answer in two sentences.

What is the default distance metric, and can you change it? Cosine. It is fixed at creation, so changing it means a new collection.

How do you get a deterministic ID so an import is idempotent? Derive a UUID from a stable key with generate_uuid5, so re-running overwrites instead of duplicating.

What does RELATIVE_SCORE do? It normalizes keyword and vector scores onto a common scale before combining them with alpha. It has been the hybrid default since 1.24.

What happens at 90 percent disk? Shards become READONLY. You free space and set the shard status back to READY.

Which tenant states exist? ACTIVE, INACTIVE and OFFLOADED. The old names HOT, COLD and FROZEN are deprecated.

How are cluster metadata and data replicated? Metadata through Raft consensus since 1.25. Data through leaderless replication with a tunable consistency level of ONE, QUORUM or ALL.

Why can a restore fail on a different cluster? Restores need the same node count and node names as the source, and roles and users are not restored by default.

What is the minimum to secure a cluster? Anonymous access off, API keys or OIDC on, RBAC on with at least one root user, TLS at the edge, and keys kept in a secret manager.

Where do model-provider keys go? In request headers or server environment variables, never in the collection config, and the header wins.

What is MMR for? Selecting results that are relevant but different from each other, so the top results are not near-duplicates. It is generally available in 1.39 and works only on HNSW.

Can you resize shards later? No. Create a new collection with the desired shard count and switch an alias.

Questions to ask the interviewer

Good questions signal that you think about operations. Ask how the team evaluates retrieval quality and whether there is a labelled set. Ask how they would restore from a backup and when they last did it. Ask which Weaviate version they run and how they handle minor upgrades, since the project supports only the latest three minors. Ask how embeddings are versioned, because changing the embedding model is a reindex. And ask who owns the on-call for the cluster. The answers tell you whether this is a mature platform or a prototype that has quietly become production.

Practice prompts

Try answering these aloud before reading any notes. A one-minute answer is enough.

1. Your hybrid search finds documents by concept but misses exact part numbers. Explain the likely cause and two fixes.
2. A teammate wants to enable 1-bit compression cluster-wide next week. What do you ask first, and what would you do to de-risk it?
3. You inherit a Compose file with anonymous access on and no volume. List what you would change and in what order.
4. A weekly report needs every object in a collection of five million. How do you read them, and what limits apply?
5. You must move to a new embedding model with no downtime. Walk through the plan.
6. Explain to a product manager why changing the shard count needs a migration rather than a setting.

For the fifth prompt, a good outline is: create a new collection with the new vectorizer, backfill (re-embedding as needed), compare retrieval quality on a labelled set, dual-write during the migration, switch the alias, monitor, and keep the old collection for rollback. For the fourth, use the cursor-based iterator() rather than large limits, because a query is capped by QUERY_MAXIMUM_RESULTS (10,000 by default).

A closing self-check

If you can do these without notes, you are ready for most mid-level Weaviate interviews. Explain what is inside a shard and why that predicts memory. State which index parameters are immutable. Describe the order of retrieval, boost and reranking. Describe a safe reindex with an alias. Say what you would put on a dashboard and which two alerts you would set first. And name three ways an import can appear to succeed while losing data.

For related systems, see Qdrant, Milvus, Pinecone and pgvector, since interviewers often ask you to compare. For evaluating retrieval and RAG quality, see RAGAS.

Part two of three. At this level Weaviate runs and answers queries, so the problems change character. They are no longer "why won't it connect" but "why did recall drop", "why did the import lose rows" and "why did the pod die at 3 a.m." Start with the error table, then the practices in order of leverage, then the practice cards and the operational recipes. Everything targets Weaviate 1.39 and Python client v4.

Common errors at this level

Cumulative: the Beginner errors still apply, and these are the ones that appear once things work.

The practices that pay off most, in order of leverage

These are ranked by how much trouble they prevent per hour spent.

1. Keep a labelled query set and an evaluation script. Twenty to fifty real queries with the document IDs that should come back, and a script that computes hit rate at k. Every other decision in this list, from alpha to compression to a new embedding model, is a guess without it. It is the highest-leverage asset in any retrieval system and takes an afternoon to start. The RAGAS guide covers RAG-level evaluation once retrieval is solid.

2. Make imports idempotent and verified. Deterministic UUIDs with generate_uuid5, batch.stream() for server-side batching, a hard failure on any entry in failed_objects, and a final count check. This removes the whole class of "we do not know what is in the database" incidents.

3. Turn anonymous access off and turn RBAC on. The default is open. Use DB users per service with the narrowest roles, rotate keys with rotate_key, keep keys in a secret manager such as Vault, and never put a model-provider key in a collection config (Weaviate does not store it there anyway). Tighten CORS_ALLOW_ORIGIN and terminate TLS at your edge.

4. Put every collection definition in code and query an alias. The schema belongs in version control, created idempotently. The application should query an alias so an immutable change becomes a backfill and one switch call. Disable AUTOSCHEMA_ENABLED in production so a typo cannot create a property.

5. Pin versions and upgrade deliberately. Pin the image to an exact patch such as 1.39.7, never latest. Upgrade one minor at a time to the latest patch of each, take a backup first, and check /v1/cluster/statistics before upgrading (1.26 and later). Stay within the latest three supported minors.

6. Measure compression before and after. Start with RQ 8-bit for most collections. Remember that you cannot turn quantization off once it is on, so rehearse on a copy.

7. Test restores. A scheduled restore into a scratch cluster, followed by your labelled queries, converts backups from a hope into evidence. Remember that restores need the same node count and names, and that roles and users are not restored by default.

8. Watch disk before memory before latency. The READONLY threshold at 90 percent is a hard stop. Alert at 80 percent. Scrape :2112/metrics with Prometheus after setting PROMETHEUS_MONITORING_ENABLED=true, and dashboard query latency, batch durations, object count, vector index size and tombstones.

9. Close your clients. Use with weaviate.connect_to_local() as client: or a finally: client.close(). In a long-running service, create one client at startup and reuse it rather than connecting per request.

10. Set timeouts on purpose. AdditionalConfig(timeout=Timeout(init=30, query=60, insert=120)) makes failures predictable under load instead of inheriting laptop defaults.

Three errors, read slowly

The table is for scanning. These three examples show the reasoning behind the fix, which is what transfers to the next error you have not seen.

Recall that moves with limit. A team notices that the top ten of a query at limit=10 are not the first ten of the same query at limit=50. Nothing is broken. With ef=-1, Weaviate derives the search width from the limit, bounded by dynamicEfMin, dynamicEfMax and dynamicEfFactor, so a bigger limit triggers a wider, more accurate search. The lesson is that an unset parameter is a policy, not an absence. Fix it by setting ef explicitly in the collection config so evaluation and production run the same effort, then measure recall at that value.

A filter that matches too much. A support tool filters tickets with equal("INC-2041") and gets back tickets numbered 2041, 2042 and others. The property used the default word tokenization, which splits on punctuation, so the filter matched on tokens and not on the whole string. The fix is field tokenization, but that is immutable in the stable API, so the actual remedy is a new collection and an alias switch. The lesson is to decide tokenization per property when you design the schema, using what the property is for: prose gets word, identifiers get field.

An import that said nothing. A nightly job loads documents and exits with status 0, yet a thousand are missing from search. The batch carried on past per-object failures, and nobody read failed_objects. Reading it showed vectorizer 429 responses, so the provider's rate limit was exceeded in bursts. Two changes fix it: fail the job when failed_objects is not empty, and throttle with client.batch.rate_limit(...) or the server-side batching stream. Deterministic UUIDs then make a re-run of the missing part safe. The lesson is that success needs an assertion, and the exit code of a script is not one.

In each case the sequence is the same. State the symptom precisely, find which layer owns it (index parameter, schema, pipeline), change one thing, and measure again.

Common traps with newer features

Several 1.37 to 1.39 features arrive with sharp edges. Mark them in your notes before a colleague finds them.

The Search REST API is experimental and off by default on early 1.39 patches, so a team that reads the blog and calls it on 1.39.0 gets a refusal until the flag is set; check the docs for your exact patch. MMR works only on HNSW, needs Python client 4.23.0 or newer, and is not available on bm25 or multi-vector collections, so a query that works in one collection can fail in another. 4-bit RQ is a preview and rejects flat indexes. The built-in MCP server is off by default and its write access is a separate switch, which should stay off until a security review says otherwise. gRPC-Web is on by default under /v1/grpc-web/ on the REST port, so if you do not use it, disable it in runtime config. And namespaces are a preview, so do not design a product around them yet.

Practice cards

Short exercises, each doable in about fifteen minutes on a development instance.

Operational recipes

Each recipe is a short procedure to keep in a runbook. All commands assume a local instance on port 8080 with anonymous access; add an Authorization: Bearer <key> header when authentication is on.

Recipe: check that a node is healthy

curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/v1/.well-known/live
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/v1/.well-known/ready
curl -s http://localhost:8080/v1/meta | python -m json.tool | head -20
curl -s http://localhost:8080/v1/nodes | python -m json.tool | head -40

Two 200 responses and a node reporting healthy status is the baseline. If live answers and ready does not, the process is up but not yet serving, often while loading shards. Give it time before restarting it.

Recipe: bring a READONLY shard back

1. Find the cause with docker stats or df -h on the data volume and free or add space.
2. List the shards with the verbose nodes call and note the collection and shard name.
3. Set the status back:

curl -X PUT http://localhost:8080/v1/schema/Article/shards/SHARD_NAME \
  -H "Content-Type: application/json" \
  -d '{"status":"READY"}'

4. Confirm writes succeed, and set the 80 percent alert if it did not exist.

Recipe: a safe delete by filter

Never run a delete you have not counted. First run the same filter as a query, see how many it matches, spot-check some, then delete.

from weaviate.classes.query import Filter

col = client.collections.use("Article")
f = Filter.by_property("category").equal("draft")
print(col.aggregate.over_all(filters=f, total_count=True).total_count)
res = col.data.delete_many(where=f)
print(res.matches, res.successful, res.failed)

On recent patches a single filtered delete handles a bounded number of objects (capped by QUERY_MAXIMUM_RESULTS), so repeat until the count reaches zero.

Recipe: rotate a service key

new_key = client.users.db.rotate_key(user_id="search-service")

Store the new key in your secret manager, redeploy the service with it, and confirm the old key now returns 401. Do this on a schedule, not only after a leak.

Recipe: a blue/green reindex, in outline

1. Create Docs_v2 with the corrected immutable settings.
2. Backfill from Docs_v1 with iterator() and batch.stream(). Use the same UUIDs.
3. Handle writes during the copy: dual-write, or repeat the copy for objects updated since it started.
4. Compare counts and run the labelled evaluation against Docs_v2.
5. Switch with client.alias.update(alias_name="Docs", new_target_collection="Docs_v2").
6. Keep Docs_v1 for a few days, then delete it and any stale aliases.

Recipe: a pre-upgrade checklist

1. Confirm the target is exactly one minor above your current one.
2. Run a full backup and verify it completed.
3. Check /v1/cluster/statistics for Raft consistency.
4. Read the release notes of the target minor and strip deprecated or removed environment variables.
5. Rehearse on a staging cluster restored from the backup, and run the labelled queries.
6. Upgrade the server, then the client, and re-run the smoke tests.

Habits that save an hour each

A few small habits are worth more than their size. Print client.get_meta() at the start of a debugging session, so you know the version and the modules. Keep a scratch collection for experiments with a clearly different name. Write collection configuration with Configure.Vectors.* and vector_config= from the start, and treat vectorizer_config= in a search result as a sign that the example is out of date. When an official page and a blog disagree, trust the docs and the release notes for the version you run. And when something behaves oddly, first ask whether it is a setting that is immutable or a default that changed in a recent minor, since the 1.34 filter strategy default and the 1.36 deletion-conflict default both did.

A pre-production checklist

• Anonymous access is off, RBAC is on, and at least one root user is set.
• Keys are in a secret manager, and provider keys are not in any collection config or repository.
• The image tag is pinned to an exact patch, and the deprecated snapshot variables are removed.
• A volume is mounted at /var/lib/weaviate, and ports 8080 and 50051 are both reachable (gRPC enabled on Helm).
• AUTOSCHEMA_ENABLED is false and collection definitions live in code.
• The application queries an alias, not a physical collection.
• Imports use deterministic IDs and batch.stream(), check failed_objects and verify counts.
• Compression choice was measured against an exact baseline.
• Replication factor and consistency level are decided, and replication is not mistaken for a backup.
• Nightly backups run and a restore drill has passed.
• Prometheus metrics, disk alerts at 80 percent and a query-latency alert exist.
• Probes use the live and ready endpoints.
• Labelled queries run in CI against a pinned container, and fail the build on a quality drop.

For comparison with neighbouring tools see Qdrant, Milvus and pgvector. For running the container well see Docker, and for the Kubernetes side see Helm. The senior guide takes the platform view next.

This is part three of three. It takes the Weaviate you already use, a vector database that stores objects with their vectors and serves vector, keyword and hybrid search, and asks what changes when other teams depend on it. Everything here targets Weaviate Database 1.39.x (1.39.7 was current in late September 2026). Version 1.40 is at release candidate, so anything I describe as preview or planned may move.

At this level the questions are different. Which component runs out of room first, which decisions can never be changed in place, what happens to a tenant when a node dies at two in the morning, who is allowed to read which collection, and how you upgrade a cluster that six teams treat as infrastructure.

Where this picks up

I start with architecture, because every other decision in this guide follows from where state lives.

Architecture and what holds state

A Weaviate node is one Go process. Inside it, a collection is split into shards, and each shard is a self-contained unit with three parts: an LSM-tree object store, an inverted index for keyword and filter lookups (also stored in the LSM store), and a vector index. Each shard also has a write-ahead log. By default the LSM files are accessed through memory mapping, which means the operating system's page cache, not the Go heap, decides how much of the object and inverted data is resident.

Nodes discover each other through a gossip protocol (HashiCorp memberlist). That is membership only. Two different mechanisms then move the actual state, and confusing them is the root of most bad capacity and failure reasoning.

Metadata goes through Raft (since 1.25). Schema changes are committed through a leader and need a majority of voters to be reachable. That is why a three-node cluster survives losing one node for schema operations and a two-node cluster does not survive losing either. Object data is leaderless: any node can coordinate a read or write, and consistency is chosen per request as ONE, QUORUM (the default) or ALL. The standard rule applies: reads are strongly consistent when the nodes read from plus the nodes written to exceed the replica count.

The consequence for a platform owner is that two independent availability questions exist. Can the cluster change its schema? That depends on Raft quorum. Can a given tenant or shard serve queries and writes? That depends on replica availability for that shard. A cluster can be healthy on one and degraded on the other, and a dashboard that shows only "pods ready" tells you about neither.

On Kubernetes the deployment is a StatefulSet with a headless Service. Stable pod identity matters because CLUSTER_HOSTNAME is how a node is known to the others, and because a restore requires the same node count and node names as the source. Treat node names as part of your data contract, not as an implementation detail.

Three APIs reach the node. REST on port 8080 handles CRUD, schema and administration. gRPC on port 50051 is the primary search and batch path and is required by the v4 clients. GraphQL at /v1/graphql is the legacy query interface and can be switched off with DISABLE_GRAPHQL. Newer additions, gRPC-Web under /v1/grpc-web/ (enabled by default, disabled through the runtime-config key grpc_web_enabled=false), the built-in MCP server at /v1/mcp, and an experimental search REST API, widen the surface you must decide about. I return to those in the security section.

Failure modes of the core

Most Weaviate incidents fall into a small number of families, and it helps to name them before a page arrives.

Resource exhaustion on the node. Memory is the dominant risk, because the HNSW graph and, unless compressed, the vectors themselves live in RAM. When the process is killed by the kernel, it restarts, replays its write-ahead log, loads or rebuilds indexes, and is unavailable for readiness while it does. Weaviate also protects itself: when disk usage passes DISK_USE_READONLY_PERCENTAGE (default 90) or memory passes MEMORY_READONLY_PERCENTAGE, shards turn READONLY and writes fail. Warnings fire earlier, at DISK_USE_WARNING_PERCENTAGE (default 80). A READONLY shard is a safety feature, but from the application's side it looks like an outage, so alert on the warning thresholds, not the read-only ones.

Slow restart. A node that holds many large shards takes a long time to become ready. Lazy shard loading, which defers loading until first use, kicks in above 1,000 shards or 100 GB (LAZY_LOAD_SHARD_COUNT_THRESHOLD and LAZY_LOAD_SHARD_SIZE_THRESHOLD_GB). Since 1.39 HNSW snapshots are automatic, which shortens the time to rebuild a graph from the commit log. Practical advice: measure restart time during every game day and treat it as a service-level number, because it sets your real recovery time when a pod is rescheduled.

Divergence between replicas. A replica that was down while writes happened is behind. Two mechanisms repair that, and both matter. Repair-on-read fixes inconsistencies a read happens to touch. Async replication compares Merkle hash trees between replicas in the background and repairs differences, and since 1.38 it is cluster-wide, generally available and on by default when the replication factor is above one. If you carried configuration from 1.36 or 1.37, the per-collection asyncConfig settings were removed. The controls are now ASYNC_REPLICATION_SCHEDULER_WORKERS (default 10) and ASYNC_REPLICATION_DISABLED, and two older cluster-level variables were removed in 1.38. A platform team that inherited a values file should grep it for the removed names.

Deletion conflicts. When a delete and a missing replica interact, the system must choose whether the object is deleted or resurrected. The deletion strategy, one of NoAutomatedResolution, DeleteOnConflict or TimeBasedResolution, makes that choice, and since 1.36 the default is TimeBasedResolution. Know which strategy your collections use, because it defines what "delete" means during a partial failure.

Control-plane stalls. If Raft loses quorum, reads and writes to existing shards may continue, but schema operations, tenant creation and RBAC changes cannot commit. A platform that creates tenants on demand turns a control-plane problem into a customer-visible one, which is a reason to keep voter nodes healthy and to think about RAFT_METADATA_ONLY_VOTERS in larger clusters.

Scaling: which component breaks first

Capacity planning for Weaviate is mostly a memory question, then a disk question, then a CPU question during import. The order matters because the fixes differ.

Memory first. An uncompressed float32 vector costs four bytes per dimension, so a 768-dimension vector is about 3 KB before any graph overhead, and the official documentation puts HNSW at roughly 2 to 12 GB of memory per million vectors depending on dimension and settings. The graph adds links per object governed by maxConnections (default 32). Multiply by your object count and by the replication factor, because every replica holds its own copy of the index. A hundred million 768-dimension vectors at replication factor 3 is not a cluster you can reason about on instinct.

Compression changes the arithmetic, and it is the first lever to reach for. Rotational quantization (RQ) is the option the documentation recommends: 8-bit RQ gives about four times compression at roughly 98 to 99 percent recall, and 1-bit RQ gives close to thirty-two times. 4-bit RQ is a preview in 1.39, works on HNSW only, and is rejected by a flat index with the error RQ bits must be either 1 or 8. Product, binary and scalar quantization remain available. Two facts govern how you plan. A new collection has no quantization unless you set one or define a cluster default with DEFAULT_QUANTIZATION, and once a collection is quantized you cannot turn quantization off. So the decision is one-way, and you should test recall on your own data before turning it on for production.

Disk second. The HFresh index, generally available since 1.38, takes a different approach: postings live on disk and an in-memory HNSW index covers only the centroids, with built-in 1-bit RQ. It supports only cosine and l2-squared distance. HFresh is the answer when memory is the binding constraint and you can accept a different latency profile. Its defaults include maxPostingSizeKB of 48, replicas of 4 (immutable) and a searchProbe that was raised to 256 in 1.39. Benchmark it with your own recall target, because I will not quote you a latency number I cannot source.

CPU during import. Vector indexing is CPU-bound. ASYNC_INDEXING=true decouples indexing from the write path and speeds ingestion, at the cost of newly written objects not being immediately searchable by vector. Embedding calls to external providers become the bottleneck long before Weaviate does, so rate-limit imports on the client side with client.batch.rate_limit(...).

Horizontal scale. Shards scale writes and data volume, and replicas scale reads and availability. The shard count is set at creation with Configure.sharding(desired_count=N), defaults to the node count (or DEFAULT_SHARDING_COUNT) and cannot be changed in place. Adding nodes to a collection created with too few shards does not spread its data. The fix is the blue/green pattern covered later: create a new collection with the right count, backfill, and move an alias. Choose shard counts for the size you expect to reach in a year, not the size you have today.

Query limits are another scaling boundary. QUERY_MAXIMUM_RESULTS caps results at 10,000, hybrid is capped separately by QUERY_HYBRID_MAXIMUM_RESULTS (100), and GRPC_MAX_MESSAGE_SIZE is about 100 MB. Full scans use collection.iterator(), which is cursor-based. A platform team that does not publish these limits receives them back as support tickets.

Index and search internals that decide quality

A platform owner is regularly asked why search results changed. Knowing which settings affect recall, and which are frozen, saves long investigations.

HNSW has build-time parameters, efConstruction (default 128) and maxConnections (default 32), which are immutable after creation, as is the distance metric. The query-time parameter ef is mutable. Its default of -1 means Weaviate chooses it dynamically: ef equals the larger of dynamicEfMin (100) and limit times dynamicEfFactor (8), capped by dynamicEfMax (500). The side effect is that recall changes with the limit a client requests. A team that asks for ten results gets a different effective search than a team that asks for fifty, from the same index. If you need predictable recall for a service-level objective, set a fixed ef and publish it.

Filtering combines the vector index with the inverted index. Since 1.34 the default filter strategy is acorn, with sweeping as the alternative, and flatSearchCutoff (default 40,000) decides when a restrictive filter switches to brute force over the matching set. Restrictive filters on a large collection are where latency surprises live, so profile the real filter shapes your tenants use. Query profiling exists (MetadataQuery(query_profile=True), a 1.37 preview) and the slow-query log is controlled by QUERY_SLOW_LOG_ENABLED with a default threshold of two seconds.

Hybrid search fuses vector and BM25 results. The fusion default is relative score fusion, with ranked fusion as the alternative, and alpha moves between pure keyword (0) and pure vector (1). BM25 itself uses k1 of 1.2 and b of 0.75 by default, and uses BlockMax WAND, the default since 1.30. Collections created before 1.30 needed a migration to benefit. One more 1.39 addition is the BM25Operator.and_cross() operator for cross-property AND matching.

Two features that reshape results at query time reached general availability in 1.39. The Boost API rescoring computes (1 - weight) * primary + weight * boost, where the outer weight defaults to 0.5, a condition weight to 1.0 and depth to 100. Conditions can be filters, numeric properties, and time or numeric decay, up to twenty per query, and boost runs before rerankers. MMR diversity selection is written diversity_selection=Diversity.mmr(limit=4, balance=0.3), where balance runs from 0.0 (most diverse, the default) to 1.0 (pure relevance). It needs Python client 4.23.0 or later, works only on HNSW, and is unavailable for bm25 queries or multi-vector collections. Note that an earlier preview showed a different spelling, so older blog posts will mislead engineers.

For a platform, the lesson is to separate index decisions, which are expensive to change, from query decisions, which are cheap. Publish a short list of approved index configurations (distance metric, quantizer, efConstruction, maxConnections) and let teams vary the query side freely. Property data types and tokenization are also fixed at creation (reindexing a property is a preview), so a text field that should have used field tokenization for exact matching and was created with the default word is a reindex project, not a setting.

Security: the trust model, identity and secrets

Out of the box, anonymous access is on. That is a sensible default for a laptop and a hazard on a shared platform, so the first security decision is to turn it off and choose real identity.

Authentication has three working shapes. Static API keys (AUTHENTICATION_APIKEY_ENABLED with AUTHENTICATION_APIKEY_ALLOWED_KEYS and matching AUTHENTICATION_APIKEY_USERS) are the simplest, but they live in environment configuration, and rotating them means redeploying. Database users (since 1.30, AUTHENTICATION_DB_USERS_ENABLED=true) are API keys created and rotated at runtime through the API, with client.users.db.create(user_id=...) returning the key and rotate_key replacing it. For a platform, prefer them. OIDC (AUTHENTICATION_OIDC_ENABLED, with issuer, client ID and claim settings) ties identity to your identity provider, and you can map a groups claim to roles. Set AUTHENTICATION_OIDC_INSECURE_SKIP_TLS_VERIFY never, and certainly not in production; it exists for development against self-signed issuers.

Authorization is RBAC, enabled with AUTHORIZATION_RBAC_ENABLED (the documented spelling; trust the environment-variable reference over a stray rendering that shows a different name). When RBAC is on, at least one root user must be configured through AUTHORIZATION_RBAC_ROOT_USERS or Weaviate will not start. Two roles are built in, root and viewer, and cannot be modified. Everything else is a role you define from permission builders: collections, data, tenants, backups, cluster, nodes, roles, users, replication, aliases and OIDC groups. Wildcards work on collection and tenant names, so a role scoped to Article* covers a family of collections and a role scoped to tenant acme* covers matching tenants. The older AdminList scheme is still documented, but teach and operate RBAC.

from weaviate.classes.rbac import Permissions

client.roles.create(
    role_name="search-team-reader",
    permissions=[
        Permissions.collections(collection="Search*", read_config=True),
        Permissions.data(collection="Search*", read=True),
    ],
)
client.users.db.assign_roles(user_id="svc-search-api", role_names=["search-team-reader"])

Design roles by job, not by person: a read-only service role, an ingestion role with create and update on specific collections, an operator role with backup and cluster permissions, and a very small number of administrators. Two details catch people. RBAC roles and users are not restored by default from a backup, so recovery must include re-creating them from code. And a role that can create or modify roles can grant itself anything, so the roles permission is effectively administrator access and should be rare.

Transport. The REST API has no documented native TLS in the pages I verified, and common practice is to terminate TLS at an ingress or reverse proxy. Treat that as practice rather than a documented guarantee. gRPC supports GRPC_CERT_FILE and GRPC_KEY_FILE. Tighten CORS_ALLOW_ORIGIN from its permissive default.

Outbound trust. Modules call external model providers. Since 1.33 API-based modules are enabled by default, and you can switch them off with API_BASED_MODULES_DISABLED=true. MODULES_VALIDATE_BASE_URL=true restricts the base URLs modules may call. For regulated workloads in the Gulf or Egypt where data residency applies, these two settings are how you prove that document text is not leaving your network for an embedding API in another region. Keep embedding on a self-hosted module such as text2vec-transformers or Ollama when residency demands it, or choose a provider region deliberately.

Secrets. Model-provider keys are never stored in the collection configuration. They arrive either as request headers (X-OpenAI-Api-Key and its siblings), which win over the alternative, or as server environment variables such as OPENAI_APIKEY. On Kubernetes, source those from Secrets, ideally synced from a manager such as Vault (see the Vault guide). Backup-bucket credentials should come from workload identity or an assumed role (S3 role assumption arrived in 1.37), not static keys.

The agent surface. The built-in MCP server, generally available since 1.38, is off by default (MCP_SERVER_ENABLED), and write access is a second switch (MCP_SERVER_WRITE_ACCESS_ENABLED). Its RBAC permissions are read_mcp, create_mcp and update_mcp. If you enable it, enable read-only first, grant only read_mcp to the identities agents use, and remember an agent with upsert rights can change your data on a model's say-so.

Multi-tenancy, namespaces and isolation

There are three isolation tools in Weaviate, and they solve different problems. A platform owner must be able to say which one a request calls for.

Multi-tenancy inside a collection gives each tenant its own shard. That means tenant data is physically separated, a tenant can be deleted cheaply (tenants.remove deletes that tenant's data), and queries are scoped with collection.with_tenant("name"), so a query cannot accidentally span tenants. Tenant names are four to sixty-four characters of alphanumerics, underscore and hyphen. Auto tenant creation (batch inserts only) and auto activation reduce operational code. Tenant states are ACTIVE, INACTIVE (on disk, not loaded) and OFFLOADED (in cloud storage through the offload-s3 module); the old HOT, COLD and FROZEN names are deprecated. The economics follow: only active tenants consume memory, so a platform with many tenants that are idle most of the time is cheap if you deactivate them.

The limits are the other side of the same coin. Each tenant is a shard, so tenant count is shard count, and the lazy-loading thresholds and restart times from earlier apply. Collection schema is shared by all tenants, so a tenant who needs a different data model needs a different collection. And an inactive tenant is slower to answer the first query after reactivation.

Collection-per-team is the other layout. It suits teams with different schemas, different index configurations or different retention. Aliases and RBAC wildcards make a naming convention such as TeamName_Purpose enforceable: a role scoped to Search* governs the search team's collections.

Namespaces (preview in 1.38 and 1.39, with general availability planned for 1.40) add control-plane and data isolation between users of a shared cluster, a level above collections and tenants. Today they are not something to commit a design to; track them, because they target exactly the platform case, and decide whether to wait or to build with collection prefixes and RBAC.

A usable decision rule: use tenancy when every customer has the same schema and you need many of them, collections with RBAC when teams own different data models, and separate clusters when you need blast-radius isolation, different upgrade timing or different compliance zones. Do not stretch one cluster across needs that really want two.

For tenant-level access control, combine tenant wildcards in RBAC with an application layer that never lets the caller choose the tenant name freely. The most common multi-tenant data leak is not a Weaviate defect. It is a service that takes a tenant identifier from a request body.

Replication, backups and disaster recovery

Replication protects against a node loss. Backups protect against everything else. Do not let either stand in for the other.

Replication is configured per collection, for example Configure.replication(factor=3, async_enabled=True, deletion_strategy=...), and the cluster can enforce a floor with REPLICATION_MINIMUM_FACTOR. Consistency is per request through with_consistency_level. A platform default of QUORUM is a good starting point; use ONE only for reads where staleness is acceptable and measured. Replica movement (REPLICA_MOVEMENT_ENABLED, and production-ready since 1.38) lets you rebalance by moving a shard replica between nodes, which is the tool for evacuating a node you plan to retire. It needs the replicate RBAC permission.

Backups need a backup module in ENABLE_MODULES: backup-s3, backup-gcs, backup-azure or backup-filesystem. The filesystem module works on a single node only and is for development and testing. A cluster's backup can be created and restored with client.backup.create and client.backup.restore, with wait_for_completion=True for scripts. Incremental backups, generally available since 1.37, use incremental_base_backup_id, and the base backup must remain available, which means your retention policy must never delete a base that a later incremental depends on. Inactive tenants are included in backups from 1.37, and offloaded tenants are skipped, since their data already lives in object storage.

Restore has hard constraints. The target cluster needs the same node count and node names as the source. RBAC roles and users are not restored by default. A restore cannot be cancelled once it reaches FINALIZING. These facts shape the disaster-recovery design: you cannot restore a 5-node backup into a 3-node cluster, so your recovery environment must be able to match the production topology.

client.backup.create(
    backup_id="2026-10-02-nightly",
    backend="s3",
    include_collections=["Articles"],
    wait_for_completion=True,
)
client.backup.restore(backup_id="2026-10-02-nightly", backend="s3", wait_for_completion=True)

The important discipline is the restore drill. Schedule one monthly: restore the latest backup into a clean cluster with matching topology, run a fixed set of queries, compare object counts against GET /v1/nodes, and record how long it took. The duration is your real recovery time objective, and the first drill almost always uncovers something, usually a missing role, a missing module setting, or a bucket permission.

An alternative recovery path exists for cases where the source is the system of record: rebuild from the source with idempotent imports. Deterministic IDs through generate_uuid5 make re-import safe. If embeddings are expensive, this path is slow and costly, which is an argument for keeping your own copy of vectors or for backups. Record which path each collection relies on in a table the on-call can read.

Upgrades and migrations

Weaviate supports the latest three minor versions. In October 2026 that is 1.39, 1.38 and 1.37, and anything older receives no patches. This is the clock your platform runs on: if you stand still for three minors, you fall off support.

The core rule is to upgrade one minor at a time, to the latest patch of each, never skipping. Before each step, take a full backup, confirm Raft metadata is in sync through /v1/cluster/statistics (the check exists from 1.26), read the release notes for the target minor, and upgrade in a non-production cluster first. On Kubernetes, helm upgrade is idempotent, and you roll replicas through the StatefulSet while watching readiness.

History explains why these rules exist. The 1.25 release moved cluster metadata to Raft and required a manual migration on Kubernetes; chart versions 17.0.0 and above also needed the StatefulSet deleted before upgrading from older versions. Version 1.30 made BlockMax WAND the default, and collections created earlier needed a migration. Version 1.33 enabled API-based modules by default, a behaviour change that affects egress. Version 1.34 changed the default filter strategy to acorn. Each of these is the kind of change a rolling upgrade can apply without any visible error, and which shows up later as a change in behaviour.

For 1.39 specifically, remove the deprecated PERSISTENCE_HNSW_SNAPSHOT_* variables, since they are ignored and logged as warnings (and PERSISTENCE_HNSW_DISABLE_SNAPSHOTS with them). Check that your client library is current: Python client 4.23.x pairs with 1.39, and Python client v3 is no longer supported, so any service still using weaviate.Client(...) or client.schema.* is a migration debt that blocks you. The v4 entry points are weaviate.connect_to_*() and client.collections.*, and several earlier v4 spellings are deprecated: Configure.Vectorizer.* and Configure.NamedVectors.* became Configure.Vectors.*, client.collections.get became client.collections.use, and client.batch.experimental() was replaced by batch.stream() and data.ingest(...). Two behaviour changes also affect clients: a batch delete missing its match fields returns 422 instead of 500, and on 1.39.7 a filtered batch delete resolves at most QUERY_MAXIMUM_RESULTS objects per request, so a very large delete-by-filter needs more than one pass.

Rollback is the part people forget to plan. A minor upgrade can change on-disk formats, so rolling back to the previous minor is not guaranteed to work after the new version has written data. Your rollback plan is the pre-upgrade backup, restored into a cluster of the same topology, not a downgrade of the image tag. State this in the change request so reviewers understand the exposure.

For things the database will not change in place, the blue/green reindex is the standard migration:

from weaviate.util import generate_uuid5

client.collections.create("Articles_v2", vector_config=new_vector_config, properties=props)
v1 = client.collections.use("Articles")
v2 = client.collections.use("Articles_v2")
with v2.batch.stream() as batch:
    for obj in v1.iterator():
        batch.add_object(properties=obj.properties, uuid=obj.uuid)
client.alias.update(alias_name="Articles_live", new_target_collection="Articles_v2")

Clients should address the alias, never a versioned collection name. Aliases arrived in 1.32. Deleting a collection does not delete aliases that point at it, so clean those up deliberately. Keep the old collection until the new one has been validated under real traffic.

Cost and capacity economics

Cost on a self-hosted Weaviate has four drivers, and they are not equally controllable.

RAM for HNSW is the largest and the most controllable. Compression is the main lever, and the cheapest experiment is to enable 8-bit RQ on a copy of production data and measure recall on queries that matter. HFresh and flat indexes are the other levers: flat suits small or per-tenant collections where a brute-force scan is fine, and HFresh trades memory for disk. The dynamic index, which starts flat and moves to HNSW past a threshold (10,000 objects by default), is experimental, needs ASYNC_INDEXING=true and its switch is one-way, so I would not build a platform default on it.

Replication multiplies everything. Replication factor 3 triples memory, disk and indexing work. Apply it where availability matters and use a factor of one, backed by good backups and a fast rebuild, for collections you can regenerate. That decision should be explicit per collection, and REPLICATION_MINIMUM_FACTOR stops teams from silently choosing less than your floor.

Disk and tenant state is the third. Deactivating tenants removes their memory footprint, and offloading to object storage reduces disk cost for cold tenants. If a large share of your tenants are cold at any time, the saving is large.

Embedding calls are the fourth, and they are the one platform owners forget to meter. Every import and every query with a vectorizer module calls a provider. Weaviate avoids re-vectorizing unchanged objects (the check is on by default; REVECTORIZE_CHECK_DISABLED defaults to false), and clients should rate-limit imports. Provider keys travel in headers, which gives you a clean place to attribute spend per team: give each team its own provider key and read the bill per key.

Server-side usage guardrails (added in 1.38) let you cap objects, collections, tenants and shards, and MAXIMUM_ALLOWED_COLLECTIONS_COUNT (default unlimited) bounds collection sprawl. Set these before a team creates a collection per user.

Observability, SLOs and alerting

Enable metrics with PROMETHEUS_MONITORING_ENABLED=true. They are served on port 2112 at /metrics. Set PROMETHEUS_MONITORING_GROUP=true if you have many shards, since it aggregates across shards and keeps cardinality manageable, and note that a metric namespace prefix is available. Official Grafana dashboards live in the weaviate-examples repository. The Prometheus guide and Grafana guide cover the tooling.

Useful metrics include queries_durations_ms, batch_durations_ms, object_count, vector_index_size, vector_index_operations, vector_index_tombstones, and the schema counts weaviate_schema_collections and weaviate_schema_shards, plus LSM compaction and async-replication metrics. Since 1.39 the latency buckets go down to 100 microseconds, which makes percentiles more meaningful for fast queries.

Define service-level objectives on what a consumer feels. A starting set: query latency percentile per collection class, error rate on search and batch, and freshness, meaning the time between an object being written and being searchable. Freshness matters if you use async indexing. Then add resource alerts that fire before the database protects itself: disk above 75 percent, memory above 80 percent, rising tombstones, restart count, and replica lag from the async-replication metrics.

Alert on symptoms with a runbook link, and keep the number of paging alerts small. Pages worth having are sustained query error rate, writes failing (READONLY shards), a node not ready beyond your restart budget, and loss of Raft quorum. Everything else is a ticket.

Governance and change control

A shared database becomes a shared source of incidents unless changes follow a path. Governance for Weaviate is mostly about immutable decisions, because those are the ones that cannot be fixed later.

Turn off automatic schema creation in production. AUTOSCHEMA_ENABLED defaults to true, so a typo in a property name silently creates a new property. With it off, every collection is created deliberately, which makes a collection definition a reviewed artifact. Keep those definitions in code, in a repository, applied by a pipeline (see the Terraform guide and the Argo CD guide for the pattern), instead of letting applications create collections at startup. Have applications check client.collections.exists() rather than create.

Define a collection contract: owner, purpose, shard count, replication factor, vector configuration, quantizer, tokenization choices, retention, and the RBAC roles that touch it. Review it as you would a database schema. The questions that matter in review are the irreversible ones: Is the distance metric right for this embedding model? Is the shard count sized for next year? Is the quantizer decided with measured recall? Is tokenization correct for identifier fields? Is the vectorizer the model you will keep? Changing the embedding model means re-embedding everything, which is a cost the contract should state.

Treat embeddings as versioned. A vector is only meaningful with the model that produced it, so record the model name and version in the collection or in a property, and plan a model change as a blue/green migration, never as a gradual mix in one index. Named vectors let a collection hold two embeddings during a transition, with target_vector choosing at query time, which gives you a safe cutover path.

Guard the control plane. Collection creation and deletion, backup and restore, role changes and replica movement belong to a small group. Deleting a collection removes its objects and is not recoverable except from a backup, so require a reviewed change and consider a second approver. Use quotas and server-side guardrails to bound what any one team can do. Keep an audit trail through your ingress and application logs, since you want to know which identity performed an administrative call.

Incident playbooks

Each playbook below begins with the symptom, the first check and the decision that follows. Keep them short enough to use at night.

Writes fail across a collection. Check shard status in GET /v1/nodes for READONLY. If disk or memory crossed the read-only threshold, free space or add capacity, then set the shard back to READY with PUT /v1/schema/{class}/shards/{shard} and body {"status":"READY"}. Fix the cause before resetting, or it flips again within minutes.

A node is out of memory and restarting. Confirm with pod events that the kill is an out-of-memory kill. Check GOMEMLIMIT and the container limit relationship, since the Go runtime should have a soft limit below the hard container limit. Short term, scale up memory or reduce load. Medium term, enable compression, move to HFresh, or add nodes and rebalance with replica movement.

Search is slow for one tenant or filter. Use query profiling and the slow-query log to find the shape. A restrictive filter against a large collection is the usual suspect: check flatSearchCutoff and filterStrategy, and whether the filtered property is indexed (indexFilterable). Also check whether ef is dynamic and the client's limit is large.

Raft quorum lost. Schema and tenant operations fail. Restore the majority of voters first. Only use one-node recovery (RAFT_ENABLE_ONE_NODE_RECOVERY) as a documented last resort after reading the current operations page, and do it with a second engineer.

Replicas disagree. Confirm async replication is enabled and not disabled by an environment variable. Check worker settings and replica-lag metrics. A forced repair read at ALL consistency touches and repairs the objects it reads.

Data was deleted by mistake. Stop writes if the collection is still live, identify the last good backup, and restore into a separate cluster of matching topology. Then copy the needed objects back with a script keyed by deterministic ID. Do not restore over production unless you have decided to lose everything written since.

Where Weaviate stops

Choosing Weaviate is a decision with edges, and a platform owner should be the one who can name them.

If your workload is a few million vectors and your system already runs PostgreSQL, the pgvector guide describes an option that adds no new system to operate, with the trade-off that you inherit PostgreSQL's scaling model and give up native hybrid search machinery. If you want a fully managed service with no cluster to run, Pinecone is the comparison to make against Weaviate Cloud. For a purpose-built engine with a different design, compare Qdrant and Milvus on your own recall, latency and operational cost, using your data. I am not going to rank them; the honest answer depends on your filters, your scale and your team.

Weaviate is a strong fit when you need hybrid search with filters in one engine, built-in vectorizer and generative integrations, multi-tenancy with tenant-level lifecycle, and tuning options across compression and index types. It is a weaker fit in these situations.

• You need relational integrity and transactions. Weaviate is not a transactional relational database, and cross-references are costly to resolve at query time. The documentation advises denormalising.
• You need your database to be the system of record. Treat it as a derived index over a source of truth, with the ability to rebuild.
• You need to reshard freely. Shard count is fixed at creation, so growth beyond your plan means a migration.
• You need very small footprints on very small data. A single embedded process or a PostgreSQL extension may be cheaper to own.
• You want a pure vector store with no keyword path. You pay in surface area for features you do not use.

Running Weaviate as a platform for other teams

A platform is a product with customers, and the customers are developers who want vector search without learning your cluster. The work divides into an interface, a paved road and a support model.

The interface is a short, versioned contract: endpoints through your gateway, the identity method (database user or OIDC), the approved client versions, the limits (result caps, message sizes, rate limits), and the data classification each cluster accepts. Publish the numbers in section form: QUERY_MAXIMUM_RESULTS, hybrid limits, batch guidance.

The paved road is what makes the safe path the default. Provide a template repository for a collection definition with the approved index configurations, an ingestion library that wraps batch.stream() with deterministic IDs, rate limiting and error handling, a standard Helm values file for a cluster, and a Terraform or GitOps module for roles and users. The Helm guide and the Kubernetes guide are the neighbours here. Make the paved road the shortest path to a working collection, and teams will choose it.

grpcService:
  enabled: true
env:
  AUTOSCHEMA_ENABLED: "false"
  AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: "false"
  AUTHORIZATION_RBAC_ENABLED: "true"
  PROMETHEUS_MONITORING_ENABLED: "true"
  DISK_USE_WARNING_PERCENTAGE: "75"
  LOG_FORMAT: "json"

Check every key against the chart version you deploy, since the chart evolves independently of the database (the official guide references chart 17.8.3 at the time of writing).

The support model covers onboarding (a design review of the collection contract), office hours, a status page, and a clear escalation path. Offer a small number of cluster tiers instead of bespoke clusters: a shared multi-team cluster for ordinary workloads, and dedicated clusters for teams with different compliance, scale or upgrade needs. Say in advance what a dedicated cluster costs the requesting team.

The review checklist

Use this list for design reviews of any new Weaviate use, and for audits of an existing cluster.

Data model. The distance metric matches the embedding model. Tokenization suits each text property. The vectorizer and its version are recorded. Cross-references are avoided or justified. Shard count is sized for a year ahead. Named vectors are used where more than one embedding is needed.

Capacity. Memory is estimated including replication and graph overhead. A quantizer decision exists with measured recall. Disk headroom is stated. Import throughput and embedding spend are estimated.

Resilience. Replication factor and consistency level are stated per collection. A deletion strategy is known. Async replication is on and observed. Restart time has been measured. Raft quorum is understood.

Security. Anonymous access is off. Identity is OIDC or database users. RBAC roles are by job with wildcards scoped narrowly. Root users are few. TLS terminates somewhere known. Provider keys are in headers or Secrets, not config. Outbound modules are restricted where required. The MCP server and gRPC-Web are deliberately on or off. CORS is tightened.

Operations. Backups run on a schedule to a durable module, with a drill record. Alerts exist for disk, memory, read-only shards and quorum, each with a runbook. Metrics are enabled. Automatic schema creation is off. Definitions live in code.

Lifecycle. The upgrade path is one minor at a time. Deprecated variables and client spellings are removed. The client library is on a supported major version. A blue/green reindex has been rehearsed. A retention policy and TTL are decided.

Governance. Owner, cost center and contact are recorded. Preview features are flagged. A decommission date or review date exists.

The complete picture

Reading the three levels together, the Weaviate story is a chain of decisions, with the earliest being the least reversible.

You chose what a vector means: the embedding model and distance metric, fixed with the index. You chose how it is held: HNSW in memory, compressed or not, or HFresh on disk, and quantization you cannot undo. You chose how it is divided: shard count at creation, tenancy or collection-per-team, replication factor. You chose who may touch it: authentication, RBAC roles and the outbound reach of modules. You chose how it survives: replication, backups, restore drills and a rebuild path. You chose how it changes: aliases and blue/green reindexing, one-minor upgrades, code-defined schemas. And you chose how you know it is working: metrics, SLOs, and retrieval evaluation.

Each choice has a cheap version early and an expensive version late. The platform owner's job is to move the thinking forward, so the expensive versions never happen by accident.

Where the series leaves you

After the three levels you can start Weaviate locally, model data into collections, import it idempotently, run vector, keyword and hybrid queries with filters, tune indexes and compression, operate multi-tenant collections, secure a cluster with RBAC, back it up and prove you can restore it, upgrade it on a safe cadence, watch it with the right signals, and decide with evidence when a different engine is the better answer.

Natural next steps are on neighbouring topics. Compare a different engine on your own data with the Qdrant guide, measure the retrieval quality of whatever you build with the RAGAS guide, trace the application that calls Weaviate with the Langfuse guide, and package the cluster's configuration with the Helm guide. The durable skill is the habit this level teaches: before a change, ask which parts are irreversible, who is affected, and how you would know it went wrong.

Sources

• Weaviate documentation
• Weaviate release notes
• Weaviate releases on GitHub
• Release v1.39.7
• Weaviate 1.39 release blog
• Weaviate 1.38 release blog
• Weaviate 1.37 release blog
• Environment variables reference
• Authentication
• Configuring RBAC
• Managing RBAC roles
• Managing users
• Backups
• Monitoring
• Migration guides
• Kubernetes installation
• Cluster architecture
• Consistency
• Vector index concepts
• Vector index configuration reference
• Rotational quantization
• Multi-tenancy
• Collection aliases
• Time to live
• Python client
• Weaviate Cloud

Senior interviews about Weaviate rarely ask what a collection is. They hand you a requirement, a constraint and a failure, and watch how you reason. This file is built around that: system-design prompts, incident questions, and how to argue a trade-off to people who do not care about vectors. Answers are written as you would say them, with the reasoning visible, because interviewers at this level score the reasoning more than the conclusion. Everything assumes Weaviate 1.39.x.

What interviewers are really testing

Three things separate a senior answer from a mid-level one. First, irreversibility: you know which choices cannot be changed in place (shard count, distance metric, vectorizer, tokenization, and the fact that quantization cannot be turned off once on) and you raise them before being asked. Second, blast radius: for every design you can say what fails, for whom, and how you would notice. Third, evidence: you say "I would measure recall on our queries" instead of asserting that a setting is fine.

A useful habit in any answer is to name the failure first. If someone asks how to scale search, start with "memory is the first constraint, because HNSW lives in RAM". It shows you understand the system, not the marketing.

Part one: system-design prompts

Design multi-tenant semantic search for a SaaS product with thousands of customers

Start by asking what varies between tenants. If every customer has the same schema and wants isolated data, use multi-tenancy inside one collection. Each tenant gets its own shard, queries are scoped with with_tenant, deleting a customer is cheap because removing the tenant removes its data, and idle tenants can be set to INACTIVE so they stop consuming memory. Cold tenants can be OFFLOADED to object storage through the offload-s3 module.

Then name the costs. A tenant is a shard, so ten thousand tenants is ten thousand shards, which affects restart time and triggers lazy shard loading above roughly a thousand shards or a hundred gigabytes. I would measure restart time at the expected tenant count before committing. I would enable automatic tenant creation and activation so onboarding needs no control-plane call beyond the first batch insert, but I would remember that Raft quorum affects tenant creation, so a metadata outage becomes a signup outage.

For access control, the application authenticates users and derives the tenant name from the verified identity, never from a request field. At the database, a service role gets tenant permissions with wildcards only where needed. I would turn anonymous access off, use database users so keys rotate without redeploying, and keep provider keys in headers sourced from a secrets manager.

For recovery I would say: replication factor 3 with QUORUM, nightly backups to object storage with incremental backups, monthly restore drills, and a rebuild path from the source of truth with deterministic IDs. I would close with the signals I would watch: per-collection object count, index size, tombstones, query latency and read-only shards.

A strong answer also says when not to do this. If large customers need different schemas or different embedding models, tenancy is the wrong tool, and collections with a naming convention, or a dedicated cluster for the biggest customers, is better.

A team wants to swap their embedding model. How do you handle it?

This is a trap for people who think of the database as the unit of change. A vector is only meaningful alongside the model that made it, and distance metric and vectorizer are fixed when the collection is created. So a model change means re-embedding everything, and the safe pattern is blue/green.

Create a new collection (or add a second named vector to the existing one) configured for the new model, backfill it with batch.stream() while the old one keeps serving, then compare retrieval quality offline on a fixed query set, and only then move an alias to the new collection. Aliases, available since 1.32, are the switch: clients address Articles_live, and client.alias.update(...) repoints it. Keep the old collection for a rollback window, and delete it deliberately, because deleting a collection does not delete aliases pointing at it.

I would add three points. Cost: re-embedding sends every object to the provider, so budget the calls and rate-limit the import. Quality: a better benchmark score is not evidence for your data, so evaluate with your queries (the RAGAS guide at /student-guides/ragas covers it). Residency: if the new model is an API, confirm where the text goes.

Memory is the constraint. Walk me through reducing the footprint

First measure instead of guessing: object count, dimensions, replication factor and the vector_index_size metric. Then go through the levers in order of how reversible they are.

Compression is the first lever. Rotational quantization at 8 bits gives about four times compression at roughly 98 to 99 percent recall according to the documentation, and 1-bit gives close to thirty-two times with more recall loss. Four-bit is a preview on HNSW only. The catch: once quantization is on, it cannot be turned off, so I test on a copy with real queries and compare recall before I touch production.

The second lever is the index type. HFresh, generally available since 1.38, keeps postings on disk with only centroids in memory, so it trades memory for disk. It supports only cosine and l2-squared. Flat is acceptable for small collections, often per-tenant.

Third, reduce what you hold: deactivate cold tenants, expire data with Object TTL (and remember nothing is deleted until the schedule variable is set), and lower the replication factor for collections that can be rebuilt. Last, add nodes, but note that existing collections do not spread to new nodes beyond their fixed shard count, which is why shard count should be planned up front.

A good closing sentence: "I would not enable quantization first and measure after. I would measure recall on a copy, then roll it out per collection."

Design the platform so six teams can share one cluster safely

I would separate the interface, the guardrails and the support model.

The interface is a gateway endpoint, an identity method and a short published contract: approved client versions, result limits, message size limits and data classification. The guardrails are technical. Anonymous access is off. Each team gets database users with RBAC roles scoped by collection prefix, for instance Search*, with a read-only service role, an ingestion role and a small operator role. Automatic schema creation is off, so collections come from reviewed definitions in a repository. Server-side guardrails and MAXIMUM_ALLOWED_COLLECTIONS_COUNT stop collection sprawl. A minimum replication factor is enforced cluster-wide.

The support model includes a design review of the collection contract (shard count, vector configuration, quantizer, tokenization, retention), per-team embedding keys for cost attribution, and a published upgrade calendar.

Then I would state the limits of sharing. If one team needs different upgrade timing, a different compliance zone, or has a noisy workload, they get a dedicated cluster, because a shared cluster shares its upgrades and its failures. I would mention that namespaces target this exact problem but are in preview in 1.39, so I would track them and not depend on them.

How would you build a zero-downtime schema change?

Distinguish what is mutable. Query-time ef, flatSearchCutoff, filter strategy and turning quantization on can be updated. The distance metric, efConstruction, maxConnections, the vectorizer, property types and tokenization cannot. Adding a property works with add_property.

For anything immutable, I run the blue/green reindex: create X_v2, backfill from v1 using collection.iterator() and batch.stream() with the same deterministic IDs, handle writes that arrive during the backfill (dual-write from the pipeline, or a final catch-up pass using update timestamps), validate counts and a query sample, then repoint the alias. Concurrent writes are the part candidates forget: without them, you cut over to a stale copy.

Part two: incident-response questions

Writes suddenly fail with errors, reads still work

My first suspicion is READONLY shards. Weaviate sets shards read-only when disk usage reaches DISK_USE_READONLY_PERCENTAGE (90 by default) or memory crosses its read-only threshold. I would check node status with GET /v1/nodes, confirm the shard status, find out why disk or memory grew (a large import, tombstone buildup, backups written locally, a runaway collection), free or add capacity, and only then set the shard back to READY. Resetting the status without fixing the cause makes it flip again.

The follow-up they want: what would have caught it earlier? An alert at the warning threshold (80 percent disk by default, and I would set mine lower), not at the read-only point.

A node keeps restarting and the cluster is degraded

Check whether it is an out-of-memory kill, a failing readiness check or a crash loop in startup. If memory, look at GOMEMLIMIT against the container limit, since the Go runtime benefits from a soft limit under the hard one, and at what changed: a new collection, an import, an index without compression. If the node is slow to become ready because it holds many shards, the readiness probe may be killing it before it finishes loading, so I would look at probe timing and at lazy loading thresholds.

While it is down, replicas serve reads at QUORUM if the replication factor allows. When it returns, async replication and repair-on-read reconcile the writes it missed. I would verify with metrics that replica lag shrinks, not assume it.

Search quality dropped after an upgrade

I would not assume the database is at fault. First list what changed: the Weaviate version, the client version, the embedding model or provider behaviour, and any new data. Then reproduce with a fixed query set against a pre-upgrade backup restored in a test cluster. Version-specific behaviour changes are plausible candidates: the default filter strategy changed to acorn in 1.34, BlockMax WAND became the default in 1.30 (collections needed a migration), and relative score fusion is the hybrid default. If ef is dynamic, the client's limit affects recall. The principle: bisect with data, not opinions.

Someone deleted a production collection

Stop writes to anything depending on it, identify the last good backup and its timestamp, and restore into a separate cluster with the same node count and node names as the source. Then copy the needed objects back, keyed by deterministic ID. Restoring over production would discard everything written since the backup, so that is a decision the data owner makes, not me alone. Afterwards: remember RBAC roles and users are not restored by default, so recreate them from code, and fix the root cause, which is usually that too many identities could delete collections.

Part three: arguing a trade-off to leadership

"Why not use the database we already have?"

If that database is PostgreSQL and the dataset is modest, this is a legitimate question, and the honest answer starts with the case for the incumbent: no new system to run, transactional integrity with your relational data, one backup story. The pgvector guide (/student-guides/pgvector) covers that route. I would then say what would change my mind: a need for hybrid search with filters at scale in one engine, tenant-level lifecycle with cheap deletion and cold storage, index choices such as compression and disk-based indexing, or latency targets that the relational engine cannot meet under our actual filters. Then I would propose a time-boxed benchmark on our data, with recall, latency under filters, and an operational cost estimate, and let the numbers decide.

"Why not just use the managed service?"

Frame it as buying time versus buying control. A managed service removes upgrades, backups and capacity work, and adds a recurring bill that depends on vector dimensions, storage and backups. Self-hosting gives you control over region and residency, network placement, and cost at scale, and takes on the on-call burden. For companies in the Gulf and Egypt, data residency and available regions are often the deciding facts, so I would start with those. I would present the decision as a table with a total-cost estimate for three years including engineer time, and say plainly which assumption would flip it.

"Why is this so expensive?"

Break the bill into four drivers: memory for the index, replication multiplying it, disk and idle tenants, and embedding calls. Then show the levers with their cost in risk: compression (save memory, test recall), tenant deactivation and offloading (save memory and disk, accept slower first queries), lower replication on rebuildable data (save a multiple, accept longer recovery), and rate-limited or cached embeddings. Quote savings in measured numbers from a test, and never promise a percentage before the test. Leadership trusts the person who says "I measured it on a copy".

"Is it production ready?"

Be precise. The core is generally available, and so are many features: RBAC, replication with async repair, backups including incremental, HFresh, the MCP server and the Boost API. Others are preview or experimental in 1.39: namespaces, four-bit quantization, the search REST API, nested object filtering and property reindexing. My rule: preview features go behind a flag with a rollback plan and never carry a critical path. That answer shows you read release notes, not slides.

Common interview questions

What is the difference between how Weaviate replicates metadata and data? Metadata (schema, tenants, RBAC data) goes through Raft with a leader and a majority. Object data is replicated leaderless, with a consistency level of ONE, QUORUM or ALL chosen per request.

Why can recall differ between two clients on the same index? With ef left at its dynamic default, the effective value depends on the limit requested, bounded by dynamicEfMin, dynamicEfFactor and dynamicEfMax. Set a fixed ef for predictable recall.

What cannot be changed after a collection is created? The distance metric, efConstruction, maxConnections, the vectorizer, property types and tokenization, and the shard count. The replication factor is changed through replica movement and not through the configuration alone. Quantization can be turned on but not off.

How do you make an import idempotent? Use deterministic UUIDs with generate_uuid5 so a retry overwrites instead of duplicating, and batch with batch.stream() or data.ingest(...).

What does READONLY mean and who sets it? Weaviate sets a shard read-only when disk or memory crosses the read-only threshold, to protect data. You restore capacity and then set the shard status back to READY.

How do you restore a backup into a different environment? The target needs the same node count and node names. Roles and users are not restored by default.

Where do model-provider keys live? In request headers such as X-OpenAI-Api-Key, or in server environment variables. Never in the collection configuration, and headers take precedence.

Which index would you choose for a collection of ten thousand objects per tenant? Flat is a sensible candidate for small tenants because it keeps memory low and brute force is fast at that size, and HNSW for larger ones. The dynamic index automates the switch but is experimental in this release, so I would be cautious about making it a default.

What is new in 1.39 that you would care about? Boost and MMR diversity selection are generally available, HNSW snapshots are automatic and the old snapshot variables are ignored, and 4-bit RQ and the search REST API are preview. I would also mention that a batch delete with missing match fields now returns 422.

When would you not choose Weaviate? For transactional relational workloads, as a system of record, for tiny datasets where an extension is cheaper to own, or when you need to reshard freely. See the pgvector (/student-guides/pgvector), Qdrant (/student-guides/qdrant) and Milvus (/student-guides/milvus) guides for alternatives.

What a strong answer covers that a weak one misses

A weak answer lists features. A strong answer picks the one that matters, names its limits, and states how it would be verified. Weak answers say "turn on replication" and stop. Strong answers give the factor, the consistency level, the deletion strategy, the repair mechanism and the drill that proves it works. Weak answers treat a backup as done when it is written. Strong answers talk about restore topology, roles, and timing. Weak answers say "upgrade the image tag". Strong answers say one minor at a time, backup first, and rollback means restore.

Final self-test

Answer aloud, in under two minutes each, without notes.

1. A new product wants semantic search over ten million documents with four filters. List the decisions you must make before creating the collection, in order, and mark which are irreversible.
2. Your cluster has three nodes and a collection with replication factor 3. One node is down. What can still be done at QUORUM, what cannot, and what changes if a second node goes down?
3. Walk through a restore drill for a five-node cluster, including roles and users.
4. A team asks for 4-bit quantization in production this week. What do you say?
5. Explain to a finance partner why the vector database bill tripled after one team enabled replication factor 3 and a larger embedding model.
6. Your search latency doubled for one tenant only. List your first five checks.
7. Describe a safe path from Python client v3 code on an old cluster to 1.39 with a v4 client, including the server upgrade order.
8. Which two preview features in 1.39 are most tempting for a platform, and what would you require before allowing them?

If you can answer all eight with specifics (names of settings, numbers where documented, and an explicit note of what you would measure), you are ready for the senior loop.

This file is the field notes of someone who runs Weaviate for other teams. It covers the traps that only appear at scale, the habits that keep a shared cluster boring, and review checklists you can paste into a pull request template or a design document. Everything assumes Weaviate 1.39.x, with Python client 4.23.x. Where a feature is a preview in this release, I say so.

Production traps that only appear at scale

Each trap below has a symptom, a cause and a fix, because that is the order in which you meet them.

The shard count you cannot change

Symptom. You add nodes to a busy cluster and the hot collection gets no faster. Cause. Shard count is set at creation through Configure.sharding(desired_count=N), defaults to the node count at that moment, and cannot be changed in place. A collection created on a three-node cluster has three shards. Fix. Create a new collection with the right count and move an alias to it after a backfill. Prevention. Size shard count for the data you expect in a year, and put the number in the collection contract that is reviewed before creation.

Dynamic ef makes recall depend on the caller

Symptom. Two services query the same collection and report different quality. Cause. The default ef is -1, so Weaviate computes it from the requested limit: the larger of dynamicEfMin (100) and limit times dynamicEfFactor (8), capped at dynamicEfMax (500). Fix. If you promise a recall level, set a fixed ef, publish it, and test it. ef is mutable, so this is a cheap change.

Quantization is a one-way door

Symptom. After enabling compression, recall is lower than expected and the team wants to go back. Cause. Once a collection is quantized, quantization cannot be turned off. Fix. Reindex into a new collection without it. Prevention. Evaluate on a copy with your real queries before enabling, and prefer 8-bit RQ as a starting point, since the documentation recommends it at roughly four times compression with recall in the high nineties. Treat 4-bit as a preview, and remember a flat index rejects it with RQ bits must be either 1 or 8.

Read-only shards look like an outage

Symptom. Writes fail suddenly, reads work. Cause. Disk or memory crossed the read-only percentage (90 by default for disk). Fix. Add capacity, then set the shard back to READY. Prevention. Alert at 70 to 75 percent disk, well before the 80 percent warning and the 90 percent read-only line.

Restart time is your real recovery time

Symptom. A rescheduled pod takes a long time to return and Kubernetes keeps killing it. Cause. A node with many or large shards needs time to load, and a readiness or liveness probe with short timeouts can kill it mid-load. Fix. Measure start time under production-sized data, set probe timings from that measurement, and use the /v1/.well-known/live and /v1/.well-known/ready endpoints for the right probes. Remember lazy shard loading above 1,000 shards or 100 GB changes what "ready" means for first-query latency.

Tenant counts are shard counts

Symptom. A platform with tens of thousands of tenants has slow restarts and long backups. Cause. One shard per tenant. Fix. Deactivate idle tenants, offload cold ones, and consider splitting tenants across collections or clusters by size. Detail to remember. Tenant names are four to sixty-four characters, and automatic creation only happens on batch inserts.

Collections created by typos

Symptom. A property appears that nobody designed, and queries miss data. Cause. AUTOSCHEMA_ENABLED defaults to true. Fix. Turn it off in production and create collections from reviewed definitions. Applications check client.collections.exists() and fail loudly if the collection is absent.

Tokenization surprises

Symptom. Filters or keyword search on identifiers match too much or too little. Cause. The default word tokenization splits text into tokens. Fix. Use field tokenization for exact matching, and recognise that changing it later means a new collection, because tokenization is fixed at creation (reindexing a property is a preview).

The gRPC path that breaks quietly

Symptom. WeaviateGRPCUnavailableError: gRPC health check could not be completed. Cause. Port 50051 not exposed, a wrong grpc_host or grpc_secure, a disabled gRPC service in Helm, or a changed gRPC URL on a managed cluster. Fix. Publish the port, set grpcService.enabled: true, and use connect_to_custom with correct values. Use skip_init_checks=True only as a short diagnostic.

Oversized batches

Symptom. Sent message larger than max or RESOURCE_EXHAUSTED. Cause. An insert_many payload bigger than GRPC_MAX_MESSAGE_SIZE, about 100 MB by default. Fix. Use collection.data.ingest(...) or batch.stream(). Raise the server limit only if you understand the memory implications.

Removed and deprecated settings that linger in values files

Symptom. Warnings in logs after an upgrade, or settings that silently do nothing. Cause. Configuration carried over from older versions. In 1.39 the five PERSISTENCE_HNSW_SNAPSHOT_* and PERSISTENCE_HNSW_DISABLE_SNAPSHOTS variables are ignored, per-collection asyncConfig was removed in 1.38, and a few cluster-level async variables were removed in the same release. Fix. Grep your values and manifests against the release notes at every minor upgrade.

Client code from another era

Symptom. Imports fail or behaviour differs. Cause. Python client v3 syntax (weaviate.Client, client.schema, with_near_text) on a v4 client, or deprecated v4 spellings. Fix. Use weaviate.connect_to_*(), client.collections.use(...), Configure.Vectors.* and batch.stream(). The client needs Python 3.10 or later and a server of 1.27 or newer.

Unclosed clients

Symptom. ResourceWarning: unclosed ... or WeaviateClosedClientError. Cause. A client created per request and not closed, or used after close. Fix. Create one client for the application lifetime, close it on shutdown, and use with blocks in scripts. The batch objects are not thread-safe, so use one batch per thread.

Platform-ownership habits

Platform owners succeed through routines more than heroics. These are the habits I would adopt first, in order of how much trouble they prevent.

Keep every definition in code. Collections, roles, users, aliases, backup schedules and Helm values live in a repository and are applied by a pipeline. If a cluster is rebuilt tomorrow, the definitions are not lost with it. This also answers the restore question about roles and users, since they are not restored from backups by default.

Run a restore drill every month. Restore the latest backup into a clean cluster that has the same node count and node names, run a fixed query set, compare object counts and record the duration. The first drill finds the gap, usually missing roles or a bucket permission. Keep the base backup of every incremental chain alive.

Rehearse upgrades. Every minor, one at a time, on the latest patch, in a non-production cluster first, with a backup before each step and a check of /v1/cluster/statistics. Put the upgrade calendar next to the three-minor support window, so nobody is surprised by falling off support.

Publish limits and defaults. Result cap (QUERY_MAXIMUM_RESULTS, 10,000), hybrid cap (100), gRPC message size, rate limits, approved client versions and the shard and replication defaults. Teams who know the limits stop opening tickets about them.

Meter by team. Put the owner in the collection name, give each team its own provider key for embeddings, and attribute object counts and index size per collection. Cost conversations become specific.

Control the control plane. A small group may create or delete collections, run restores, change roles and move replicas. Everyone else uses a service role scoped to their prefix. Anyone who can edit roles can grant themselves anything, so treat that permission as administrator access.

Pin and review versions. Pin the image tag to a patch, such as 1.39.7, never latest, and review patch notes, since patches change behaviour: 1.39.7 limits a filtered batch delete to QUERY_MAXIMUM_RESULTS objects per request and changes some error codes from 500 or 403 to 404 or 422.

Write runbooks with first actions. Each alert links to a page that starts with the first command to run. A runbook that begins with a paragraph of background fails at three in the morning.

Treat embeddings as versioned data. Record the model and its version with the collection. A model change is a migration, done with a new collection or a second named vector and an alias switch.

Prefer evidence to opinion. Before enabling compression, changing an index type, or choosing an engine, run a small benchmark on your own data, record the settings and the recall, and keep the result next to the decision.

Settings worth knowing by heart

A short reference of the controls a platform owner touches most, with their defaults from the documentation. Verify each against the version you run.

Two health endpoints back the probes: /v1/.well-known/live and /v1/.well-known/ready. Node and shard status comes from GET /v1/nodes, and version and modules from GET /v1/meta.

Review checklist for a pull request that touches Weaviate

Paste these into the pull request template. A reviewer should be able to answer each from the diff or the description.

Collection definitions

• Is the collection created by code and not at application startup?
• Does the distance metric match the embedding model's recommendation?
• Is the vectorizer and its model recorded, and is the cost of changing it understood?
• Are text properties given the right tokenization, especially identifiers (field)?
• Is the shard count justified for the expected data size in a year?
• Is the replication factor stated and at least the platform minimum?
• Is a quantizer chosen with measured recall, or deliberately left off?
• Are cross-references avoided or justified, given their query-time cost?
• Is Object TTL configured if data should expire, and is the cluster schedule variable set?

Application code

• Does it use client v4 entry points and not deprecated spellings?
• Is there a single long-lived client, closed on shutdown?
• Do imports use batch.stream() or data.ingest(...) with deterministic UUIDs?
• Are provider keys passed as headers from a secrets source, never hard-coded?
• Are limit, ef and filters chosen on purpose, and is limit within the caps?
• Is the tenant name derived from a verified identity?
• Are errors from batches inspected (failed_objects) and not ignored?

Access control

• Does the change add or widen a role? Is the scope narrow, using collection or tenant patterns?
• Does anything receive the roles, users or replicate permissions?
• Are new keys created as database users with rotation in mind?

Operations

• Does the change affect memory, disk or import load enough to need a capacity note?
• Are alerts and runbooks updated?
• Is there a rollback, and for schema changes is it a blue/green move with an alias?

Review checklist for a design document

Use these questions in design review before a new workload is accepted onto the platform. Ask them in this order, because the early ones are the hardest to reverse.

1. What is the source of truth? Weaviate should be a rebuildable index over it. If not, what is the recovery plan?
2. What are the embedding model and distance metric, and who owns changing them?
3. What is the size now and in a year, in objects and dimensions? From this derive memory, disk and shard count.
4. What recall and latency are promised? Measured on whose queries? What ef is fixed?
5. Which filters are used? Are restrictive filters over a large collection involved, and have they been profiled?
6. Is hybrid search needed? What alpha and fusion type, and how is quality evaluated?
7. Tenancy or collections? Same schema across customers means tenancy, different schemas mean collections.
8. What are the availability and recovery objectives? Choose replication, consistency level and backup cadence from them.
9. Where may data go? Which embedding provider and region, and does residency require a self-hosted module?
10. Who may read, write and administer? Name the roles.
11. What is the cost? Memory, replication, embeddings, and who pays.
12. How does it change? Upgrades, model changes, reindexing and retirement.

If a design document cannot answer questions 1 to 3, it is not ready for review.

Operational recipes

Short, copyable procedures. Treat them as starting points and check each against the documentation of your version.

Check cluster health.

curl -s -H "Authorization: Bearer $WEAVIATE_API_KEY" http://localhost:8080/v1/nodes
curl -s -H "Authorization: Bearer $WEAVIATE_API_KEY" http://localhost:8080/v1/cluster/statistics
curl -i http://localhost:8080/v1/.well-known/ready

Reset a read-only shard after fixing capacity.

curl -X PUT -H "Authorization: Bearer $WEAVIATE_API_KEY" -H "Content-Type: application/json" \
  -d '{"status":"READY"}' \
  http://localhost:8080/v1/schema/Articles/shards/<shard-name>

Create a rotatable service identity.

import weaviate
from weaviate.classes.init import Auth

with weaviate.connect_to_local(auth_credentials=Auth.api_key("admin-key")) as client:
    key = client.users.db.create(user_id="svc-search-api")
    client.users.db.assign_roles(user_id="svc-search-api", role_names=["search-team-reader"])
    # later: client.users.db.rotate_key(user_id="svc-search-api")

Take and restore a backup.

client.backup.create(backup_id="nightly-2026-10-02", backend="s3", wait_for_completion=True)
client.backup.restore(backup_id="nightly-2026-10-02", backend="s3", wait_for_completion=True)

Switch an alias to a new collection.

client.alias.create(alias_name="Articles_live", target_collection="Articles_v1")
# after backfilling Articles_v2 and validating:
client.alias.update(alias_name="Articles_live", new_target_collection="Articles_v2")

Upgrade checklist, one minor at a time.

helm repo update
helm show values weaviate/weaviate > values-new.yaml   # diff against your values
helm upgrade --install weaviate weaviate/weaviate --namespace weaviate --values ./values.yaml

Take a backup first, confirm the Raft statistics, read the notes for the target minor, and roll through the StatefulSet while watching readiness. The chart's own notes call out older cases, such as deleting the StatefulSet when moving to chart 17.0.0 or later from a version before 1.25.

Cost levers, ranked by leverage

1. Compression (8-bit RQ first). The largest saving on HNSW memory, with a recall cost you measure.
2. Right-sized replication. Factor 3 for collections that need availability, lower for rebuildable data.
3. Tenant lifecycle. Deactivate idle tenants, offload cold ones.
4. Index choice. HFresh for disk-based scale, flat for small collections.
5. Embedding spend. Rate limits, no needless re-vectorizing (the check is on by default), caching, and per-team keys for visibility.
6. Retention. TTL for data that should expire, set up with the schedule variable.
7. Collection sprawl. Guardrails and MAXIMUM_ALLOWED_COLLECTIONS_COUNT before teams create one collection per user.

Measure each lever on a copy before rolling it out, and keep the measurement with the change request.

Practice cards

Card 1: the quiet cutover. Create Docs_v2 with a different quantizer, backfill from Docs_v1 with deterministic IDs, run twenty real queries against both, compare, repoint the alias, and keep v1 for a week. Write down the rollback.

Card 2: the lost node. On a three-node test cluster with replication factor 3, stop one node during a write test. Read at ONE and at QUORUM. Bring it back and watch replica lag shrink in the metrics. Note how long it took.

Card 3: the leaked key. Create a database user, pretend its key leaked, rotate it, and confirm the old key fails. Write the incident note you would send.

Card 4: the read-only drill. Fill a test node's disk to the read-only threshold, see the failure from the application side, free space, reset the shard and note the alert that should have fired earlier.

Card 5: the restore. Restore a backup into a second cluster, compare object counts, recreate roles from code, and record the total time. That number is your recovery objective.

Card 6: the audit. Run the design-document questions against one existing production collection and list every irreversible decision that was never made on purpose.

Where to read next

The version-specific release notes are the most valuable reading for a platform owner, so read them at every minor, even for features you do not plan to use. Pair this with the neighbouring guides: /student-guides/qdrant and /student-guides/milvus for comparison, /student-guides/pgvector when the incumbent database is PostgreSQL, /student-guides/helm and /student-guides/kubernetes for deployment, /student-guides/prometheus and /student-guides/grafana for observability, /student-guides/vault for secrets, and /student-guides/ragas for evaluating retrieval quality.

The checklist to run before going live

• Anonymous access off, identity chosen, RBAC on, root users few.
• Automatic schema creation off, definitions in code.
• Shard count, replication factor and quantizer decided with evidence and recorded.
• Backups scheduled to a durable module, one restore drill passed, and roles and users recoverable from code.
• Metrics on, alerts for disk, memory, read-only shards, readiness and quorum, each with a runbook.
• Client versions approved, deprecated spellings removed, one long-lived client per service.
• Provider keys in headers or Secrets, outbound module settings reviewed for residency.
• Image tag pinned to a patch, upgrade calendar published.
• Limits and contacts documented for consuming teams.

Ship when every line has an owner and a date, not when every line is perfect.