The Complete FAISS Guide

This is part one of three. It covers everything you need to do real work with FAISS as a first-time user. By the end you can install it, build an index from your own vectors, search it, save it, load it again, make it faster with an IVF or HNSW index, read its most common error messages, and wire it to a small text-search project. Mid-level and Senior take the same topics further; nothing here is thrown away.

Each section ends with a Try it task. Do them as you go. They take a few minutes each, and the ideas in this guide only stick once you have watched your own search return a wrong answer, and then fixed it.

Everything below was checked against FAISS 1.15.1, the current release at the time of writing.

What FAISS is, and the problem it solves

FAISS stands for Facebook AI Similarity Search. It is an open-source library from Meta, written in C++ with Python bindings, that answers one question very quickly: given a vector, which stored vectors are closest to it?

To see why that question matters, start with what a vector is in this context. A vector (also called an embedding) is a list of numbers, for example 384 or 768 of them, produced by a machine-learning model. The useful property of embeddings is that things with similar meaning end up with similar numbers. A sentence-embedding model turns "How do I reset my password?" and "I forgot my login credentials" into two lists of numbers that sit close together, even though the two sentences share almost no words. The same trick works for images, audio clips, products, and code.

Once your data is vectors, "find things similar to this one" becomes "find the stored vectors closest to this query vector". That is the job of FAISS. It powers semantic search, recommendation, duplicate detection, and the retrieval half of retrieval-augmented generation (RAG), where a language model is given relevant documents found by vector search before it answers.

The diagram is the whole workflow. FAISS owns only the third box. It does not create embeddings and it does not store your documents. It stores vectors and hands back the row numbers of the closest ones, and you use those numbers to look up the original text, image, or product in your own storage.

Why does this need a dedicated library? Because the obvious approach, comparing the query against every stored vector, gets expensive fast. With a thousand vectors it is instant. With ten million vectors of 768 numbers each, one query means about seven and a half billion multiplications. FAISS is built to make that fast: it uses optimised linear algebra and CPU vector instructions for the exact approach, and it offers approximate index types that skip most of the data while still finding nearly all the right answers. It can also run on NVIDIA GPUs.

Two things set FAISS apart from other tools you may have heard of, and both explain most of the surprises that beginners hit.

FAISS is a library, not a database. There is no server to start, no port, no login, no query language. You import faiss in your own Python process, and the index lives in that process's memory. If your program exits, the index is gone unless you saved it to a file. Vector databases such as Qdrant, Milvus, Weaviate, Chroma and pgvector add storage, networking, metadata, and updates on top of ideas similar to those in FAISS, and a few of them use FAISS inside. With FAISS you get the engine and you build the car yourself.

FAISS only knows numbers. It stores float32 vectors and returns 64-bit integer ids. It has no idea what text, filenames, or categories are. Keeping the mapping between ids and your real data is your job. This is easy once you expect it and bewildering if you do not.

Where FAISS fits, and when to pick something else

Before FAISS, nearest-neighbour search in Python usually meant one of three things: a NumPy loop or matrix multiplication, scikit-learn's nearest-neighbour classes, or a hand-rolled tree structure. These all work for small data. They slow down badly when the number of vectors reaches millions or the number of dimensions reaches hundreds, because tree structures stop helping in high dimensions and brute force simply costs too much.

FAISS gives you a menu of trade-offs. At one end is exact search, which is always right and gets slower as data grows. At the other end are approximate indexes, which answer in a small fraction of the time and memory but occasionally miss a true neighbour. The skill you will build in this guide is choosing a point on that menu on purpose rather than by accident.

Because FAISS is a library, you should also know when it is the wrong tool. Reach for a vector database instead when you need any of the following:

• Updates and deletes at high frequency, from many users at once.
• Filtering by metadata, such as "only documents from 2025 in the legal category". FAISS has no metadata filtering. It can filter only by integer id.
• A service that several applications call over the network, with authentication.
• Durability and replication that you do not want to build yourself.

Reach for FAISS when you want a fast embedded engine, when your data fits comfortably in memory on one machine, when you are learning how vector search works, or when you are building the retrieval part of a system you are willing to wrap in your own service. For a student project, a research experiment, or a prototype RAG application with a few hundred thousand chunks, FAISS is an excellent fit and has no running costs.

A note for readers working for Gulf and Egyptian employers: because FAISS runs entirely inside your own process, no data leaves your machine or your cloud region when you search. That is genuinely useful when data-residency rules apply. The embedding step is different. If you create embeddings by calling a hosted API, that step sends text to the provider, so check where the provider processes it.

The mental model: four nouns

Almost everything in FAISS comes down to four ideas. If you hold these in your head, the API stops looking like a list of strange class names.

Vector. A row of float32 numbers. FAISS expects a whole collection of them at once as a two-dimensional NumPy array with shape (n, d): n rows, one per item, and d columns, the number of dimensions. The type must be float32, not NumPy's default float64. The value d is called the dimensionality and is fixed when you create an index. Every vector you add or search with must have exactly that many columns.

Index. The central object. An index holds your stored vectors (sometimes in compressed form) and knows how to search them. You create one by choosing a class, such as IndexFlatL2, give it d, and then call methods on it. Different index classes make different trade-offs between speed, memory, and accuracy, but they share almost the same methods: add, search, and a few others. This is why learning one index teaches you most of the others.

Distance (or similarity). How "close" is measured. The two you need now are L2 distance, the straight-line Euclidean distance, and inner product, a similarity where bigger means more alike. FAISS reports L2 as a squared distance, with no square root taken. We will spend a full section on this because it is where beginners meet their first confusing result.

Id. A 64-bit integer that names a stored vector. By default, ids are simply the order in which you added the vectors: the first is 0, the second is 1, and so on. A search returns ids, not vectors and not text.

Put together, the lifecycle of an index looks like this:

The train step needs explaining. Simple indexes know everything they need from birth. Cleverer ones, such as IVF and product quantization, must first learn something from a sample of your data, for example where the clusters are. That learning is called training, and you call index.train(sample) once before adding vectors. Every index has an attribute is_trained that tells you whether it is ready. Training happens only once: FAISS does not support retraining a populated index, so if your data changes character, you build a new index.

Two small attributes will become your most-used debugging tools: index.d, the dimension, and index.ntotal, the number of vectors stored. When something looks wrong, print these two before anything else.

Installing FAISS and checking the setup

For learning, you want the CPU package. It runs on Linux, macOS, and Windows and needs no special hardware. Create a clean virtual environment first so that you do not pollute your system Python or pick up an older FAISS by accident.

python -m venv .venv
source .venv/bin/activate          # on Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install "faiss-cpu==1.15.1" numpy

Pinning the version with == is a good habit for any library you rely on, because FAISS releases every one to three months and some releases change behaviour.

There is a point worth stating plainly because older tutorials get it wrong. Until recently, the faiss-cpu package on PyPI was built by volunteers outside the project, and many guides told readers that the official route was conda only. That has changed. Since version 1.14.2 the FAISS project publishes official pip wheels, and the PyPI packages faiss-cpu, faiss-gpu and faiss-gpu-cuvs are maintained from the main repository. Pip is therefore a perfectly good way to install it. Conda is still described by the project's INSTALL.md as the supported route, and works like this:

conda install -c pytorch -c conda-forge faiss-cpu=1.15.1

On macOS with conda, note that the package supports Apple Silicon (arm64). Pip wheels also exist for Intel Macs on recent macOS versions.

A few platform facts that save hours:

• There is no official GPU package for Windows or macOS. The GPU packages are for Linux with an NVIDIA card. On Windows, WSL2 with a Linux environment is the practical route if you want GPU later, but you do not need a GPU for anything in this guide.
• Do not install faiss-cpu and faiss-gpu in the same environment. Both provide a Python module called faiss, and they will fight.
• Install into the interpreter you actually run. A very common first error is ModuleNotFoundError: No module named 'faiss', which almost always means you installed with one Python and ran with another.

Now verify the install. Save this as check_faiss.py:

import faiss
import numpy as np

print("faiss version:", faiss.__version__)
print("compile options:", faiss.get_compile_options())
print("gpus visible:", faiss.get_num_gpus())

d = 64
xb = np.random.rand(1000, d).astype("float32")

index = faiss.IndexFlatL2(d)
index.add(xb)

D, I = index.search(xb[:5], 4)
assert (I[:, 0] == np.arange(5)).all()
print("ntotal:", index.ntotal)
print("nearest ids for the first query:", I[0])

Run it with python check_faiss.py. You should see the version 1.15.1, a line of compile options that mentions the CPU instruction set in use, gpus visible: 0 on a CPU package, and ntotal: 1000. The assertion checks something intuitive and useful: if you search the database with vectors that are already in it, the nearest neighbour of each one must be itself, at distance zero. That "search for yourself" check is a quick sanity test you will use again.

If the first line fails with ModuleNotFoundError, run python -m pip show faiss-cpu and compare the interpreter path to the one you ran. If it fails with an import error about NumPy, upgrade NumPy; the pip wheels need NumPy 1.25 or newer.

Your first working index, step by step

We will build a small but real index and walk through what each line does. We start with random vectors, because that removes everything except FAISS itself from the picture. Later in the guide we replace them with real embeddings.

Step 1: make the data. FAISS wants a NumPy array of shape (n, d) and dtype float32.

import faiss
import numpy as np

d = 128                      # dimension of every vector
n_db = 100_000               # number of stored vectors
n_q = 10                     # number of queries

rng = np.random.default_rng(42)
xb = rng.random((n_db, d), dtype="float32")   # the database
xq = rng.random((n_q, d), dtype="float32")    # the queries

Using a seeded random generator makes your results repeatable, which matters when you compare two runs. Notice that we asked for float32 directly. If you create data the default way, np.random.random(...) gives float64, and you must convert with .astype("float32"). The high-level Python functions convert some inputs for you, but relying on that hides real bugs, so make float32 a reflex.

Step 2: create the index. IndexFlatL2 is the simplest index. It stores every vector exactly as given and, at search time, compares the query with every one of them.

index = faiss.IndexFlatL2(d)
print(index.is_trained)      # True: a flat index needs no training
print(index.ntotal)          # 0: nothing stored yet

The name decodes as follows. Flat means "stored as is, no compression, no shortcuts". L2 means the distance is Euclidean. There is a sibling, IndexFlatIP, that uses inner product, which we cover in a later section.

Step 3: add the vectors. One call, and the ids are assigned for you.

index.add(xb)
print(index.ntotal)          # 100000

The first vector in xb gets id 0, the second id 1, and so on. Adding in several calls continues the numbering, so if you add 1,000 vectors and then 500 more, the second batch gets ids 1,000 to 1,499. FAISS does not copy these ids anywhere you can edit; they are simply positions.

Step 4: search. Ask for the 5 nearest neighbours of each of the 10 queries.

k = 5
D, I = index.search(xq, k)
print(D.shape, I.shape)      # (10, 5) (10, 5)
print(I[0])                  # ids of the 5 nearest vectors to query 0
print(D[0])                  # their squared L2 distances, smallest first

search returns two arrays, and learning to read them is the core skill of this whole guide.

• I holds ids, with shape (n_q, k). Row i lists the ids of the k nearest stored vectors to query i, best first. Its dtype is int64.
• D holds distances, with the same shape. D[i][j] is the distance between query i and the vector whose id is I[i][j].

To go from an id back to the vector itself in a flat index, you can index your own array: xb[I[0][0]]. FAISS also offers index.reconstruct(i) for some index types, but because you usually keep the original data anyway, indexing your own array is simpler.

Step 5: sanity-check against yourself. Search with vectors you know are in the database.

D_self, I_self = index.search(xb[:3], 1)
print(I_self.ravel())        # [0 1 2]
print(D_self.ravel())        # [0. 0. 0.] (or extremely close to 0)

If the self-search does not return the right ids with distance near zero, something is wrong with your data pipeline, not with FAISS. Hold on to that diagnostic.

Step 6: time it. Try a larger query batch and measure.

import time

xq_big = rng.random((1_000, d), dtype="float32")
t0 = time.perf_counter()
D, I = index.search(xq_big, 10)
print(f"{time.perf_counter() - t0:.3f} s for 1000 queries")

You will see that a flat search over 100,000 vectors is fast on a normal laptop, because it uses optimised matrix operations over a whole batch. That is an important lesson about FAISS. It is built for batches. Searching 1,000 queries in one call is much faster per query than calling search a thousand times with one query each. When you build an application, collect queries into batches where you can.

Reading distances: L2, inner product, and cosine similarity

You now have a search that returns numbers, so the next question is what the numbers mean. This is the most conceptually tricky part for beginners, and a few minutes of care here prevents a lot of silent wrong answers.

L2 distance. IndexFlatL2 uses METRIC_L2. The value in D is the squared Euclidean distance: the sum of the squared differences between the two vectors' components, with no square root applied. Smaller means closer, and an exact match gives 0. FAISS skips the square root because it does not change the order of results and costs time. If you need the true distance, take np.sqrt(D) yourself.

Inner product. IndexFlatIP uses METRIC_INNER_PRODUCT. The value is the dot product of the query and the stored vector. Here bigger means more similar, and the results come back sorted from highest to lowest. This reversal trips people up: with L2 you read the smallest number first, with inner product the largest.

Cosine similarity. Many embedding models are designed to be compared by cosine similarity, which measures the angle between two vectors and ignores their length. FAISS has no cosine index. The standard recipe is simple and worth memorising: normalise every vector to length 1, then use inner product. For unit-length vectors, the inner product is the cosine similarity.

import faiss
import numpy as np

d = 128
rng = np.random.default_rng(0)
xb = rng.random((10_000, d), dtype="float32")
xq = rng.random((5, d), dtype="float32")

faiss.normalize_L2(xb)       # in place! xb is modified
faiss.normalize_L2(xq)       # normalise queries the same way

index = faiss.IndexFlatIP(d)
index.add(xb)
D, I = index.search(xq, 5)
print(D[0])                  # cosine similarities, in [-1, 1], largest first

Three details in that snippet matter.

First, faiss.normalize_L2 works in place. It changes the array you pass in and returns nothing, so there is no xb = faiss.normalize_L2(xb). It also requires a float32 array, which is another reason to get the dtype right from the start.

Second, you must normalise both the stored vectors and the queries, with the same function. If you normalise only one side, your "cosine" scores are meaningless, and the results will look plausible but be wrong. This is the single most common silent bug in beginner FAISS code.

Third, some embedding models already return unit-length vectors. Normalising a vector that is already unit length does no harm, so when in doubt, normalise.

For unit-length vectors, L2 and cosine rank results identically, because the squared L2 distance equals 2 - 2 x cosine. So an L2 index on normalised vectors gives the same ordering as an inner-product index. Use whichever you find easier to explain.

How do you decide which metric to use? Read the model card of your embedding model. If it says to use cosine similarity or dot product with normalised embeddings, normalise and use IndexFlatIP. If it says Euclidean distance, use IndexFlatL2. Using the metric a model was trained for gives the best results.

Ids, custom ids, and keeping your own data

By default, FAISS ids are just positions: 0, 1, 2 and so on. That is fine if your documents live in a Python list or a table in the same order. You search, get ids, and index into your list.

documents = ["reset your password", "update billing details", "export your data"]
# ... suppose these were embedded, in this order, and added to the index ...
# D, I = index.search(query_vector, 2)
# for doc_id in I[0]:
#     print(documents[doc_id])

That pattern is the entire "storage" story for beginners: a list or a database table whose row number equals the FAISS id. If you ever delete or reorder the list without rebuilding the index, the ids point at the wrong things. That is a silent, nasty bug, so keep the list and the index together, and save them together.

Sometimes you want your own ids, for example database primary keys such as 10042 and 10043. A flat index does not accept them directly. You wrap the index in an IndexIDMap:

import faiss
import numpy as np

d = 64
rng = np.random.default_rng(1)
xb = rng.random((5, d), dtype="float32")
ids = np.array([101, 102, 103, 104, 105], dtype="int64")

index = faiss.IndexIDMap(faiss.IndexFlatL2(d))
index.add_with_ids(xb, ids)

D, I = index.search(xb[:1], 3)
print(I)        # ids drawn from 101..105, never 0..4

Pay attention to the details. Ids must be a NumPy array of dtype int64. FAISS has no string ids and never will, so a product code such as "SKU-9912" must be mapped to an integer through a dictionary you maintain. You wrap the index before adding any vectors; wrapping a non-empty index fails with an error that says the index must be empty on input. And if you call plain add on an IndexIDMap, it raises an error, because it needs ids; use add_with_ids.

If you try add_with_ids directly on IndexFlatL2 you will get add_with_ids not implemented for this type of index. Flat and HNSW indexes only know sequential ids. Wrapping them with IndexIDMap fixes that, and the index factory string for the same thing is "IDMap,Flat".

Finally, remember that FAISS stores no metadata. If a search result is "id 4017" you need your own table to learn that id 4017 is a document called "Refund policy, v3", written in Arabic, tagged "billing". A simple Python dictionary, a CSV file or a database table all work for a learning project.

Saving and loading an index

An index in memory disappears when the process ends. Saving takes two functions.

import faiss

faiss.write_index(index, "vectors.index")
index2 = faiss.read_index("vectors.index")
print(index2.ntotal, index2.d)

write_index writes the whole index to a single file, including anything it learned during training. read_index gives you back an equivalent index that you can search immediately. For a flat index of 100,000 vectors of 128 numbers, the file is about 51 MB, because every vector is stored as four bytes per number, plus a small header. You can check that with ls -lh vectors.index.

A handful of practical points about saving:

• The directory must already exist, or you get could not open ... for writing. A wrong path on loading gives could not open ... for reading: No such file or directory.
• Save your id-to-data table beside the index, in the same folder and version. A restored index is useless if the list that explains its ids is missing or has changed.
• A newer FAISS can read files written by older versions, but not necessarily the other way round. If you write an index with a recent FAISS and try to read it with an old one, you may see Index type ... not recognized. Pin the same version in every environment that touches the file.
• GPU indexes cannot be written directly. You convert to a CPU index first. This does not matter on the CPU package, but remember it for later.

There is also a security point that is easy to miss and important to learn on day one. The FAISS project's own documentation warns that no attempt is made to verify that a loaded index file is valid, and that a faulty or malicious file could cause out-of-memory errors or, if crafted expertly, code execution. Recent releases hardened the reading code considerably, but the rule remains: load only index files you created yourself or received from a source you trust, and never load one a stranger sent you. The same applies to the files that some wrapper libraries save next to the index, which can contain Python pickles.

Making search faster: the IVF index

Flat search is exact, and for up to a few hundred thousand vectors it is often all you need. Past that, it becomes the bottleneck, because every query compares against every vector. The first and most widely used shortcut is the inverted file index, written IVF.

The idea is easy to picture. Imagine a library where books are sorted into rooms by topic. To find a book about gardening, you do not walk through every room; you go to the gardening room and perhaps one or two next to it. IVF does this with vectors. At training time it runs k-means clustering on a sample of your data to find nlist cluster centres (called centroids), and each region of space closest to one centroid is a cell. When you add vectors, each is filed in the cell of its nearest centroid. At search time, FAISS finds the nprobe cells closest to the query and only compares against vectors in those cells.

This produces the central trade-off of approximate search. A smaller nprobe is faster but can miss a true neighbour that sat in a cell you did not visit. A larger nprobe is slower and more accurate. When nprobe equals nlist, you visit every cell and the search is exhaustive again, which gives the same answers as the flat index.

import faiss
import numpy as np

d = 128
rng = np.random.default_rng(7)
xb = rng.random((100_000, d), dtype="float32")
xq = rng.random((10, d), dtype="float32")

nlist = 1024
quantizer = faiss.IndexFlatL2(d)               # decides which cell a vector belongs to
ivf = faiss.IndexIVFFlat(quantizer, d, nlist)  # metric defaults to L2

print(ivf.is_trained)     # False: IVF must learn its cells first
ivf.train(xb)             # k-means over the data
print(ivf.is_trained)     # True

ivf.add(xb)
ivf.nprobe = 16           # the default is 1, which is too low for good recall
D, I = ivf.search(xq, 5)

Read this carefully, because three points confuse beginners every time.

The quantizer is a small separate index used only to find the nearest centroid. It is usually a flat index. It is an unusual name for it, but the role is simple: it assigns vectors to cells. Keep a reference to it alive as long as the IVF index is in use.

Training is mandatory. If you skip ivf.train(...) and call add you get Error: 'is_trained' failed. The training sample should resemble the data you will index. Here we trained on all of xb, but with larger data you train on a random subset. As a rule of thumb from the FAISS documentation, use somewhere between 30 and 256 training vectors per cell, and expect FAISS to print a warning if you give fewer than 39 per cell. For nlist = 1024 that means at least about 40,000 training vectors; the warning looks like WARNING clustering 1000 points to 1024 centroids: please provide at least 39936 training points.

nprobe defaults to 1, and that is the cause of most "my IVF index returns bad results" complaints. With one cell visited, you only see vectors in the single closest cell. Raise it to 8, 16, or 32 and compare. A good starting rule for the number of cells is between 4 and 16 times the square root of the number of vectors, so for 100,000 vectors, roughly 1,300 to 5,000. The official guidelines give this for collections up to a million vectors.

To measure how good an approximate index is, you need a ground truth: the answers from an exact flat search. The measure is recall: of the true top-k neighbours, what fraction did the approximate index also return?

flat = faiss.IndexFlatL2(d)
flat.add(xb)
_, I_true = flat.search(xq, 5)

for nprobe in (1, 4, 16, 64):
    ivf.nprobe = nprobe
    _, I_ivf = ivf.search(xq, 5)
    hits = [len(set(a) & set(b)) for a, b in zip(I_true, I_ivf)]
    print(nprobe, sum(hits) / (5 * len(xq)))

You will see recall climb as nprobe grows. One honest caveat: random vectors have no cluster structure, so IVF performs worse on them than on real embeddings, and recall will be lower than you would see on real text or image data. Do not draw conclusions about real-world accuracy from random numbers; use them only to learn the mechanics.

Making search faster another way: the HNSW graph

IVF is one family. The other popular family is HNSW, which stands for hierarchical navigable small world. Instead of sorting vectors into rooms, it connects each vector to a handful of its neighbours, building a layered graph. To search, FAISS starts at an entry point and repeatedly hops to whichever neighbour is closer to the query, like finding a street by asking people on each corner whether they know the way. The upper layers are sparse and allow big jumps; the bottom layer is dense and allows precise finishing.

import faiss
import numpy as np

d = 128
rng = np.random.default_rng(3)
xb = rng.random((100_000, d), dtype="float32")
xq = rng.random((10, d), dtype="float32")

index = faiss.IndexHNSWFlat(d, 32)       # M = 32 links per vector
index.hnsw.efConstruction = 64           # set BEFORE adding (default 40)
index.add(xb)                            # no training needed
index.hnsw.efSearch = 64                 # default is 16
D, I = index.search(xq, 5)

There are three settings to understand. M is the number of connections per vector: more connections give better accuracy and use more memory. efConstruction controls how carefully the graph is built, and it only matters while adding, so set it before add. efSearch controls how wide the search is at query time, and it plays the same role as nprobe: raise it for better recall and slower queries. The defaults in FAISS are efConstruction = 40 and efSearch = 16.

HNSW has a different personality from IVF, and knowing it helps you choose:

• No training. You can add vectors straight away, which makes it pleasant to use.
• Fast and accurate at the cost of memory. Besides the vectors themselves, each one carries link storage of roughly 256 bytes for M = 32, so HNSW suits cases where RAM is plentiful.
• No deletion. remove_ids is not supported on HNSW indexes. If you must remove items, rebuild the index or use another type.
• Sequential ids only. To use your own ids, wrap it with IndexIDMap, which brings the IDMap rules from earlier.

The index factory: building indexes from a string

Writing the constructor classes by hand gets awkward as indexes become more elaborate. FAISS provides a compact text language called the index factory. You describe the index in a short string and get back the right object.

import faiss

d = 128
a = faiss.index_factory(d, "Flat")                 # same as IndexFlatL2(d)
b = faiss.index_factory(d, "IVF1024,Flat")         # same as IndexIVFFlat with nlist=1024
c = faiss.index_factory(d, "HNSW32")               # same as IndexHNSWFlat(d, 32)
e = faiss.index_factory(d, "IVF1024,Flat", faiss.METRIC_INNER_PRODUCT)
f = faiss.index_factory(d, "IDMap,Flat")           # flat index with custom ids

Read the strings left to right as a pipeline: optional pre-processing, then the structure, then how vectors are stored. "IVF1024,Flat" means "IVF with 1024 cells, storing full vectors". "HNSW32" means "HNSW with 32 links". The third argument sets the metric and defaults to L2, so remember to pass faiss.METRIC_INNER_PRODUCT when working with cosine similarity.

A bad string gives could not parse index string .... Some strings carry hidden requirements: for example, "IVF4096,PQ16" needs d to be divisible by 16. You will meet compression codes like PQ and SQ8 in the next level of this series; for now you only need the Flat, IVF...,Flat, HNSW... and IDMap,... forms.

A useful trick that avoids a famous trap: with an index created by index_factory, you may not know its concrete class, so setting attributes such as index.nprobe = 16 can silently do nothing. The reliable way to set search settings on any index is ParameterSpace:

ps = faiss.ParameterSpace()
ps.set_index_parameters(b, "nprobe=16")      # works on factory-built or wrapped indexes

When the index is wrapped (for instance in an IndexIDMap), assigning a field directly on the wrapper creates a harmless Python attribute and changes nothing in the C++ object, so searches stay at the default. ParameterSpace reaches through the wrapper and sets the real value.

Choosing an index when you are a beginner

With three families on the table, you need a practical way to choose. The FAISS project publishes "Guidelines to choose an index", and the beginner-relevant parts boil down to a short decision list. Think of it as a ladder: start at the top and only step down when you have a measured reason.

1. Up to roughly a few hundred thousand vectors, or when you need exact results: Flat. It needs no tuning, no training and has no accuracy loss. If a flat search is fast enough for your application, stop here. The official guidance says that for a few thousand searches, or anytime exact answers are required, flat is the answer.
2. Plenty of memory and no deletions: HNSW. HNSW32 is a strong default. It is fast and accurate and needs no training.
3. Larger data, or memory is somewhat tight: IVFK,Flat. Choose K as 4 to 16 times the square root of your number of vectors, train it on 30 to 256 vectors per cell, and raise nprobe until recall is acceptable.

Beyond roughly a million vectors, the guidelines recommend IVF with an HNSW coarse quantizer, such as IVF65536_HNSW32 for one to ten million vectors, and compression to cut memory. Those are mid-level and senior topics, and the later parts of this series pick them up.

It helps to do the memory arithmetic before you start, because RAM is usually the limit that decides things. A flat index stores 4 x d bytes per vector. For 1 million vectors of 768 numbers, that is about 3 GB. At 100 million vectors it is about 307 GB, which does not fit on a laptop and will be the reason to learn compression later. HNSW adds around 256 bytes per vector for links, and IVF adds 8 bytes per vector for the id.

Useful helpers: brute-force search and k-means

Two helpers deserve a mention, because they show that FAISS is more than indexes. The first is a brute-force search with no index object at all, useful for quick comparisons and for checking an index.

import faiss
import numpy as np

d = 32
rng = np.random.default_rng(5)
xb = rng.random((20_000, d), dtype="float32")
xq = rng.random((4, d), dtype="float32")

D, I = faiss.knn(xq, xb, 5, metric=faiss.METRIC_L2)   # exact top 5, no index to build
dist = faiss.pairwise_distances(xq, xb)               # a full (4, 20000) distance matrix

faiss.knn is handy in notebooks and tests: it returns exactly what a flat index would. pairwise_distances gives every distance between two sets and is only suitable for modest sizes, because the result has one entry per pair.

The second helper is k-means clustering, the same algorithm IVF uses internally, offered directly:

km = faiss.Kmeans(d, 10, niter=20, verbose=True)   # 10 clusters, 20 iterations
km.train(xb)
print(km.centroids.shape)                          # (10, 32)
D, labels = km.index.search(xb, 1)                 # nearest centroid per vector

After training, km.centroids holds the cluster centres, and searching km.index with your data gives each vector's cluster. This is a fast way to group embeddings, for example to see themes in a pile of customer messages. The default number of iterations is 25 if you do not pass niter. As with IVF, if you ask for many clusters from too few points, FAISS warns you that you should provide more.

Configuration you will actually touch: threads and data types

FAISS has few configuration files, because it is a library, but a few settings matter even for beginners.

Threads. FAISS uses OpenMP to run a batch of queries in parallel across CPU cores. You normally leave it alone. When you need to limit it, for instance on a shared machine or inside a container, there are two ways:

export OMP_NUM_THREADS=4
python first_index.py

or in Python, faiss.omp_set_num_threads(4). A subtlety makes the second form surprising: the setting applies to the current thread only. If you call it in one thread and run searches in another, it appears to have no effect. The environment variable has no such problem, so prefer it in scripts and services. To see the current value, call faiss.omp_get_max_threads().

If searches are mysteriously slow on a machine that uses OpenBLAS, the cause can be two layers of threading fighting each other. Setting OMP_WAIT_POLICY=PASSIVE is the documented remedy.

Common errors and how to read them

FAISS is written in C++, and its exceptions arrive in Python as RuntimeError, with the message in the form Error in <function> at <file>:<line>: <message>. The useful part is the end. Some failures are harder: a hard assertion prints Faiss assertion '<condition>' failed and kills the whole process, with no chance to catch it. That is a crash, not an exception. Use the table to turn messages into fixes.

Putting it all together: a small semantic search project

Time to combine everything into one useful program: a tiny semantic search engine over a handful of support questions. It uses sentence-transformers to produce real embeddings, which is a separate library you install alongside FAISS. Real embeddings are what make the results meaningful, and they also let you see why normalisation matters.

First install the extra library and set up a folder:

python -m pip install sentence-transformers
mkdir faiss_search && cd faiss_search

Now the program. It embeds documents, builds a cosine-similarity index, saves it with its document list, and answers queries.

import json
import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

docs = [
    "How do I reset my password?",
    "I forgot my login credentials",
    "How can I update my billing address?",
    "Where do I download my invoices?",
    "How do I cancel my subscription?",
    "The app crashes when I open it",
    "How do I export all my data?",
    "Can I change my account email?",
]

model = SentenceTransformer("all-MiniLM-L6-v2")      # produces 384-number vectors
xb = model.encode(docs).astype("float32")             # shape (8, 384)
faiss.normalize_L2(xb)                                # unit length, in place

d = xb.shape[1]
index = faiss.IndexFlatIP(d)                          # inner product on unit vectors = cosine
index.add(xb)

faiss.write_index(index, "support.index")
with open("support_docs.json", "w") as f:
    json.dump(docs, f)

def ask(question, k=3):
    q = model.encode([question]).astype("float32")    # note the list: shape (1, 384)
    faiss.normalize_L2(q)
    D, I = index.search(q, k)
    for score, i in zip(D[0], I[0]):
        print(f"{score:.3f}  {docs[i]}")

ask("I can't sign in to my account")
ask("where can I find my receipts")

Walk through what happens. The model turns each sentence into a 384-number vector (that dimension is a property of this particular model, so d is read from the data instead of being typed in). We convert to float32 and normalise in place. The index uses inner product, which on unit vectors equals cosine similarity. We save two files together: the FAISS index and a JSON list of the documents whose positions are the ids.

The ask function is where the lessons pay off. A single question is encoded as a list of one string, which gives shape (1, 384) and avoids the dimension error. We normalise it the same way as the stored vectors. Then search returns scores in D and ids in I, and we use the ids to look up the original text from docs.

When you run it, "I can't sign in to my account" should rank the password and login questions first, even though it shares hardly any words with them. That is the point of embeddings. "Where can I find my receipts" should surface the invoice question. If the first run downloads a model, it needs internet once and then caches it. Remember that embedding a text with this local model happens entirely on your machine, which keeps your documents in place; a hosted embedding API would send them to the provider.

To reuse the saved index later, in a fresh process, load both files and you never need to embed the documents again:

import json
import faiss

index = faiss.read_index("support.index")
with open("support_docs.json") as f:
    docs = json.load(f)
print(index.ntotal, len(docs))      # these two numbers must be equal

That final comparison is a good habit: the number of vectors and the number of documents must match, and a mismatch means the two files are out of sync.

To grow this into something realistic, change three things. Embed in batches rather than one at a time. Switch the index to HNSW32 or IVF... once you have enough data to need speed, measuring recall against flat. And add a ParameterSpace setting to tune nprobe or efSearch. If you later want to feed these retrieved documents to a language model for question answering, that is a RAG pipeline, and frameworks such as LangChain and LlamaIndex provide FAISS wrappers. Those wrappers belong to those projects, so check their documentation for current signatures and be careful with how they save files, since some use pickling.

What you can now do, and what comes next

You can now do real work with FAISS. You know that it is an embedded library for nearest-neighbour search over float32 vectors, and not a database. You can install an official, pinned version, verify it, build a flat index, add vectors, search them, and read the two arrays that come back. You understand the difference between squared L2, inner product, and cosine, and you know the normalise-both-sides recipe. You can use custom ids through IndexIDMap, keep a side table for your real data, and save and load an index while treating the file as something to trust carefully. You can speed up search with IVF and HNSW, measure their recall against a flat baseline, use the index factory, and set search parameters safely. You can read the common error messages in the table above.

What comes next, in the later parts of this series:

• Mid-level covers compression (product quantization, scalar quantization, FastScan), re-ranking with refinement, per-query search parameters that are safe across threads, filtering by id with selectors, deletes and updates, and tuning with ParameterSpace.
• Senior covers running FAISS as a service, GPUs and cuVS, indexes that do not fit in RAM, sharding, upgrades, security, multi-tenancy, and when to move to a vector database.

Beyond this series, look at Qdrant, Milvus, Weaviate, Chroma, Pinecone and pgvector once you need persistence, metadata filters, or a network API. For evaluation of the retrieval step in a RAG system, look at RAGAS. Having built a FAISS index by hand, you will understand precisely what each of those systems is doing for you.

Sources

• FAISS documentation
• FAISS repository and README
• FAISS releases
• FAISS CHANGELOG
• FAISS INSTALL.md
• FAISS wiki: Getting started
• FAISS wiki: Faiss indexes
• FAISS wiki: The index factory
• FAISS wiki: Guidelines to choose an index
• FAISS wiki: MetricType and distances
• FAISS wiki: Index IO, cloning and hyper-parameter tuning
• FAISS wiki: Threads and asynchronous calls
• FAISS wiki: FAQ
• FAISS wiki: Troubleshooting
• faiss-cpu on PyPI
• The Faiss library (Douze et al.)

Part one of three. A twenty-five-minute review of the Beginner Detailed track, then the questions interviewers ask about FAISS. Mid-level reviews this plus its own material; Senior reviews all three. Everything here assumes FAISS 1.15.1.

The thirty-second answer

FAISS is an open-source library from Meta, written in C++ with Python bindings, for similarity search over dense vectors. You build an index object from float32 embeddings, add vectors to it, and call search with a query to get back the ids and distances of the nearest stored vectors. It can search exactly, with a flat index, or approximately, with IVF or HNSW indexes that trade a little accuracy for a lot of speed and memory. It is a library that runs inside your process, not a database server.

Then the line that shows you have used it: "it only stores vectors and integer ids, so I keep the mapping from ids to my real documents and metadata myself."

The model

The commands to have ready

You will often be asked to write a small FAISS program on a whiteboard or in a shared editor. These are the lines to know cold.

import faiss, numpy as np

d = 128
xb = np.random.random((100_000, d)).astype("float32")
xq = np.random.random((10, d)).astype("float32")

index = faiss.IndexFlatL2(d)       # exact, squared L2
index.add(xb)                      # ids 0..n-1
D, I = index.search(xq, 5)         # D: distances, I: ids, shape (10, 5)

faiss.normalize_L2(xb)             # cosine: normalise both sides, in place
faiss.normalize_L2(xq)
ip = faiss.IndexFlatIP(d)
ip.add(xb)

ivf = faiss.IndexIVFFlat(faiss.IndexFlatL2(d), d, 1024)
ivf.train(xb)                      # required
ivf.add(xb)
ivf.nprobe = 16                    # default is 1

hnsw = faiss.IndexHNSWFlat(d, 32)
hnsw.hnsw.efSearch = 64            # default is 16

faiss.write_index(index, "vectors.index")
index = faiss.read_index("vectors.index")

Be ready to explain every line, especially the float32 conversion, the in-place normalisation, and why train is needed for IVF but not for flat or HNSW. Also know the installation line: python -m pip install "faiss-cpu==1.15.1".

The classic traps and why they happen

These are the mistakes interviewers love to probe, because each one tests whether you understand the model and not only the syntax.

The dtype trap. NumPy creates float64 arrays by default, and FAISS wants float32. The cure is .astype("float32"). The trap is subtle because some high-level calls convert silently while low-level ones and normalize_L2 do not, so a bug can appear in one place but not another.

The shape trap. search needs a 2-D array, (n_queries, d). A single query of shape (d,) fails with an assertion about dimensions. The fix is q.reshape(1, -1).

The cosine trap. FAISS has no cosine index. You get cosine by normalising vectors to unit length and using inner product. The classic mistake is normalising only the database or only the queries. Another is forgetting that normalize_L2 works in place and returns nothing.

The distance-direction trap. L2 returns squared distance, smallest first. Inner product returns a similarity, largest first. People compare the wrong direction or take a square root they do not need.

The nprobe trap. IVF defaults to nprobe = 1, so out of the box it looks at a single cell and recall is poor. People blame the index type instead of raising the parameter.

The untrained trap. add on an IVF index before train fails with Error: 'is_trained' failed. Flat and HNSW are trained from birth, which is why only some indexes need the step.

The ids trap. Ids are positions unless you wrap in IndexIDMap. Delete or reorder your own document list and the ids silently point at the wrong text. There are no string ids.

The wrapper trap. Setting index.nprobe = 16 on a wrapped or factory-built index may do nothing, because you set a Python attribute on a proxy. The robust way is ParameterSpace().set_index_parameters(index, "nprobe=16") or faiss.extract_index_ivf(index).nprobe = 16.

The trust trap. Index files can come from untrusted places. The FAISS documentation warns that a malicious file could cause out-of-memory errors or code execution, and recent releases added hardening, but the rule remains: only load files you built or trust.

Common interview questions

1. What is FAISS, and is it a vector database?

It is a library, not a database. It gives you index data structures and search algorithms, and it runs in-process with the index in memory. There is no server, no authentication, no persistence layer beyond write_index, no metadata filtering and no query language. A vector database such as Qdrant, Milvus, Weaviate or pgvector adds those things around similar algorithms, and some of them embed FAISS as an engine. A strong answer names what you would have to build yourself if you used FAISS in production: persistence, replication, filtering, and a network API.

2. What does search return, and how do you read it?

It returns two arrays of shape (n_queries, k). D holds distances or similarities and I holds int64 ids, both sorted best-first. With IndexFlatL2 the distances are squared Euclidean and smaller is better. With IndexFlatIP the values are inner products and larger is better. If fewer than k results exist, or an approximate index could not find enough candidates, the missing slots hold -1 for the id and a very large distance. A weak answer forgets the -1 case or says the distances are plain Euclidean.

3. How do you do cosine similarity in FAISS?

Normalise every stored vector and every query to unit length with faiss.normalize_L2, which works in place on a float32 array, and then use IndexFlatIP. For unit vectors, the inner product equals the cosine. You can also use an L2 index on normalised vectors, because squared L2 equals 2 - 2 x cosine, so the ranking is the same. The point to stress is that both sides must be normalised.

4. Why do you need to train some indexes?

IVF needs to learn where the cell centres are, using k-means on a sample. Product quantization needs to learn codebooks and scalar quantization needs value ranges. Training should use a sample that looks like your real data. A flat index and HNSW need nothing, so is_trained is true immediately. Training is a one-off: FAISS does not support retraining an index that already holds data, so if the data distribution changes substantially, or if you change the embedding model, you build a new index.

5. Explain IVF and the role of nlist and nprobe.

IVF clusters the vectors into nlist cells during training. Each added vector goes in its nearest cell. At query time, the search finds the nprobe closest cells and compares only against the vectors in them. Raising nprobe gives better recall and slower searches, and at nprobe = nlist the search is exhaustive. nlist is fixed at build time and nprobe is a search-time setting with a default of 1. For up to a million vectors the guidelines suggest nlist between roughly 4 and 16 times the square root of the number of vectors, with 30 to 256 training vectors per cell. A strong answer mentions measuring recall against a flat baseline.

6. When would you choose HNSW over IVF?

HNSW is attractive when memory is plentiful, when you do not want a training step, and when you want high recall at low latency. Its costs are memory, since each vector carries about 256 bytes of links at M = 32, no support for remove_ids, and sequential ids only unless wrapped in IndexIDMap. IVF uses less memory, supports deletion, and scales more naturally with compression and larger collections, but needs training and a tuned nprobe. Neither is universally better; the answer depends on constraints, and a good candidate says they would measure.

7. How do you measure how good an approximate index is?

Compute ground truth with a flat index on the same data and queries, then compute recall: the fraction of the exact top-k ids that the approximate index also returns. Measure speed at the same time, in queries per second, using realistic batch sizes. Plot or tabulate the trade-off as you vary nprobe or efSearch, and choose the cheapest setting that meets the recall target. Random data is a poor test because it has no cluster structure; use real embeddings.

8. How do you use custom ids?

Wrap the index in faiss.IndexIDMap before adding anything, then call add_with_ids(x, ids) with ids as an int64 NumPy array. Plain add is not allowed on the wrapper. The factory string is "IDMap,Flat". Calling add_with_ids directly on a flat or HNSW index gives add_with_ids not implemented for this type of index. Ids are always 64-bit integers, so string keys need an external dictionary.

9. How do you save and load an index, and what should worry you?

faiss.write_index(index, path) and faiss.read_index(path). Save the id-to-document table in the same place, and check that ntotal equals the number of documents after loading. Worry about three things: the version, because a newer FAISS reads older files but not necessarily the reverse; trust, because index files are not verified on load; and GPU indexes, which must be converted to CPU before writing. Mention that you would store a checksum alongside the file.

10. Why is search faster in batches?

FAISS is built around batched queries. For a flat index, a batch becomes a matrix multiplication handled by optimised BLAS code, and across queries it uses OpenMP threads. One thousand queries in a single call cost far less per query than one thousand separate calls. So in an application you gather requests into batches when latency allows.

11. How do you control threads?

Set the OMP_NUM_THREADS environment variable, or call faiss.omp_set_num_threads(n). The function's effect is per thread, so calling it in one thread does not affect another thread that runs the search. That is why the environment variable is more reliable in services. If you see strange slowness with OpenBLAS, OMP_WAIT_POLICY=PASSIVE is the documented remedy for thread over-subscription.

12. What does the index factory do?

faiss.index_factory(d, "IVF1024,Flat") builds an index from a compact string. It lets you store the index description in configuration, change index types by editing one line, and follow the official guidelines literally. The metric defaults to L2, so for cosine you pass faiss.METRIC_INNER_PRODUCT. Bad strings produce could not parse index string, and some require that d be divisible by a number in the string.

13. How would you install it, and what changed recently?

pip install "faiss-cpu==1.15.1" into a virtual environment, or the conda route conda install -c pytorch -c conda-forge faiss-cpu=1.15.1. A strong answer notes that official pip wheels now exist, since 1.14.2 for CPU and 1.14.3 for GPU, so older advice that GPU is conda-only is out of date. There is no official GPU package for macOS or Windows. And you do not install faiss-cpu and faiss-gpu into the same environment.

14. A teammate says search results got worse after they switched embedding models. What is going on?

An index built from the old model's vectors lives in a different vector space. Mixing old stored vectors with new query vectors gives meaningless distances, and a changed dimension would fail outright. The index must be rebuilt from the new model's embeddings. Add that you would version the index together with the model name and compare recall on a fixed test set before switching.

15. When would you not use FAISS?

When you need metadata filtering, frequent updates and deletes from many clients, a networked service with access control, or built-in replication, and you do not want to build those yourself. A vector database is the better choice there. FAISS fits embedded use, research, prototypes, and cases where you are happy to build a thin service around a fast library.

What a strong answer covers that a weak one misses

A weak answer lists class names. A strong answer says what each one trades. It mentions the float32 requirement, the two-dimensional input, the metric's direction, the default nprobe, and the fact that FAISS stores ids and not documents. It describes how recall would be measured instead of asserting that an index is accurate. It names the trust issue with index files without being asked, and it knows that tutorials from before 2025 may describe installation and APIs that have since changed. When it does not know something, it says it would check the documentation, which interviewers read as maturity and not weakness.

Sixty-second self-test

Cover the answers and say them aloud.

1. What are the two return values of search, and which way is "better" for L2 and for inner product?
2. Why does IndexFlatL2 return squared distances?
3. What exactly must you do to get cosine similarity?
4. Which indexes need train, and which do not?
5. What is the default nprobe, and what happens at nprobe = nlist?
6. What does -1 in the id array mean?
7. How do you give your own ids to a flat index?
8. Which common index type cannot delete vectors?
9. Why is an index file a trust issue?
10. What do you rebuild when you change the embedding model?

Answers: D and I, with smaller better for L2 and larger better for inner product. The square root does not change the order and costs time. Normalise both vectors and queries and use inner product. IVF, PQ, SQ and similar need it; flat and HNSW do not. The default is 1, and at nprobe = nlist the search is exhaustive. It means no result was found for that slot. Wrap in IndexIDMap and use add_with_ids. HNSW. Loaded files are not verified and could be malicious. The whole index.

Whiteboard drills

Interviewers sometimes ask you to reason on paper, so practise these calculations.

Memory of a flat index. A flat index costs 4 x d bytes per vector. For 5 million vectors of 768 dimensions that is 5,000,000 x 768 x 4, which is about 15.4 GB. If the interviewer asks whether it fits on a 16 GB machine, the right answer is that it technically fits but leaves no room for anything else, so you would consider compression or a bigger machine. Mention the HNSW extra of about 256 bytes per vector (about 1.3 GB here) and the 8 bytes per vector that IVF adds for ids.

Brute-force cost. One query against n vectors of d dimensions needs about n x d multiply-adds. For one million vectors of 768 dimensions that is about 768 million operations per query. The point of an approximate index is to avoid most of them.

Choosing nlist. For 250,000 vectors, 4 to 16 times the square root is about 2,000 to 8,000 cells. Pick a value in range, say 4,096. Then check the training requirement: at least 39 points per cell, so about 160,000 training vectors, and the guidelines suggest up to 256 per cell. Here you could train on the full 250,000.

Recall. If the true top-5 ids for a query are 3, 9, 12, 40, 41 and the index returns 3, 9, 12, 77, 90, then recall at 5 is 3 of 5, or 0.6. Average over many queries to get the figure that matters.

Being able to produce these numbers in your head while explaining your reasoning is more impressive than naming advanced index types.

Questions to ask the interviewer

If the interview allows questions, these show practical awareness. How many vectors do you index, and how fast is it growing? Which embedding model produces them, and how do you handle changing it? How do you measure retrieval quality today? Do you need metadata filters or deletes? Where does the index live, and who is allowed to write to it? The answers tell you whether the team uses FAISS as a quick engine or as the heart of a system, and they let you apply what you learned in this series.

Last-minute checklist

• Vectors are float32, two-dimensional, with d columns equal to the index d.
• Cosine means normalise both sides, then inner product.
• L2 returns squared distance, smaller first; inner product returns larger first.
• Only some indexes need train; check is_trained.
• IVF: raise nprobe from its default of 1 and measure recall against a flat index.
• HNSW: no training, no deletes, memory heavy.
• Ids are int64; strings need your own mapping; wrap in IndexIDMap for custom ids.
• Save the document table beside the index and compare counts after loading.
• Load only index files you trust.
• Pin the version: faiss-cpu==1.15.1.

Part one of three. Most beginner problems with FAISS come from four causes: the data is the wrong type or shape, the metric is misunderstood, an approximate index is left at its default settings, or the index and the documents it points at drift apart. Start with the table, then read the symptom-by-symptom explanations, the habits, and the checklist.

Common errors at this level

The mistakes beginners actually make

Each one below has the same shape: what you see, why it happens, and what to do.

Mistake 1: using float64 arrays

Symptom. An odd error, a TypeError, or results that look wrong, especially after normalize_L2.

Cause. NumPy's default floating type is float64. FAISS stores float32. Some wrapper functions convert for you, which hides the issue until you call one that does not. normalize_L2 modifies its argument in place, so if a conversion happened silently on a copy, your original array is never normalised.

Fix. Create or convert your data once, at the start of the pipeline: x = np.asarray(x, dtype="float32"). For arrays from slicing or transposing, use np.ascontiguousarray(x, dtype="float32"). Then check x.dtype and x.flags['C_CONTIGUOUS'] while debugging.

Mistake 2: passing a single query as a one-dimensional array

Symptom. An assertion error mentioning dimensions, even though the vector has exactly d numbers.

Cause. FAISS always works on a matrix of queries. A vector of shape (128,) is not a matrix with one row.

Fix. q = q.reshape(1, -1), or, when you encode text, pass a list: model.encode([question]) instead of model.encode(question). The second form returns a one-dimensional vector in many libraries.

Mistake 3: thinking the distances are plain Euclidean distances

Symptom. You compute a distance with NumPy and get a different number than FAISS shows. Or you write a threshold such as "distance below 0.5" and nothing matches.

Cause. IndexFlatL2 reports the squared L2 distance. FAISS leaves out the square root because it does not change the ordering.

Fix. Take np.sqrt(D) if you need the real distance, and choose thresholds on the same scale as what you measure. Better, pick a threshold by looking at real examples: print the scores of good and bad matches, and place the cut between them.

Mistake 4: reading inner product the wrong way round

Symptom. The best match appears last, or you take argmin over scores and get the worst result.

Cause. With IndexFlatIP, larger is better. With IndexFlatL2, smaller is better. The search returns each row sorted best-first for its own metric, so the first column is always the best, but your own post-processing may assume the opposite.

Fix. Keep the first column as the best one and avoid re-sorting. When you filter by score, write the direction into a comment next to the comparison.

Mistake 5: cosine search without normalising

Symptom. An inner-product index returns long vectors first, whatever the topic, and scores above 1.

Cause. Inner product depends on vector length. Cosine similarity does not. FAISS has no cosine index, so you must normalise to unit length yourself.

Fix. faiss.normalize_L2(xb) and faiss.normalize_L2(xq) before add and before search. Check that the norms are close to 1. Read your embedding model's documentation to see whether it already outputs normalised vectors.

Mistake 6: leaving nprobe at 1

Symptom. An IVF index is quick but misses obvious neighbours. A flat index finds them.

Cause. The default nprobe is 1, which searches only the closest cell. A true neighbour sitting just across a cell boundary is never examined.

Fix. Raise nprobe: try 4, 8, 16, 32, and measure recall each time against a flat index. Keep the smallest value that reaches your target.

Mistake 7: training on too little or unrepresentative data

Symptom. A warning: clustering ... points to ... centroids: please provide at least ... training points. Or recall is poor and uneven.

Cause. FAISS wants at least 39 training vectors per centroid, and the official guidance is 30 to 256 per cell. If the sample is much smaller, or comes from a different part of the data than what you later add, the cells are badly placed.

Fix. Train on a random sample of your real vectors, not the first thousand rows, which may all be from one source. Use a smaller nlist if your sample is small.

Mistake 8: forgetting that FAISS stores no documents

Symptom. You search and get numbers like 4017 and wonder where the text is.

Cause. The index holds vectors and integer ids. That is all.

Fix. Keep a list, a dictionary, or a database table that maps id to document, and save it next to the index. After loading both, check that index.ntotal == len(docs).

Mistake 9: letting the index and the documents drift apart

Symptom. Results are plausible but the wrong text comes back, especially after you "cleaned" or re-sorted the data.

Cause. Ids are positions. If you drop a row from your document list, every later id now points one row off, and nothing raises an error.

Fix. Never edit the document list without rebuilding the index. Better, use stable ids from your own database through IndexIDMap and add_with_ids, so each result names its row directly. Save index and documents as one versioned unit.

Mistake 10: expecting string ids

Symptom. An error when passing strings as ids, or surprising conversions.

Cause. Ids are always 64-bit integers. FAISS will not support string ids.

Fix. Keep a dictionary from your string keys to integers, and its reverse, and persist both.

Mistake 11: wrapping a non-empty index in IndexIDMap

Symptom. Error: 'index_->ntotal == 0' failed: index must be empty on input.

Cause. The wrapper must see every vector as it is added so that it can record the ids.

Fix. Create the inner index, wrap it immediately, and only then call add_with_ids.

Mistake 12: trying to delete from HNSW

Symptom. remove_ids not implemented for this type of index.

Cause. HNSW's graph cannot remove nodes. This is a property of the index type, not a bug.

Fix. Choose a Flat or IVF index if you need deletes, or rebuild the HNSW index from the current data on a schedule. As a stopgap you can drop deleted ids from results in your own code, remembering that you still pay for them in memory.

Mistake 13: changing the embedding model without rebuilding

Symptom. After an upgrade, search quality collapses, or a dimension error appears.

Cause. Vectors from different models live in different spaces. Their distances are not comparable, and the dimensions may differ.

Fix. Rebuild the whole index from the new model's output. Record the model name and version beside the index so the mismatch is detectable.

Mistake 14: calling search one query at a time in a loop

Symptom. A script that should be fast takes minutes.

Cause. FAISS is optimised for batches. Each call has overhead, and a single-row matrix cannot use the matrix-multiplication speed-up.

Fix. Stack your queries into one array and search once. If you are building a service, collect requests for a few milliseconds and process them together.

Mistake 15: installing both CPU and GPU packages

Symptom. Import errors, missing GPU classes, or the wrong version loading.

Cause. faiss-cpu and faiss-gpu both provide a module called faiss.

Fix. Uninstall both, then install exactly one. Use a fresh virtual environment per project.

Mistake 16: following an old tutorial

Symptom. A tutorial says there is no pip package, or that GPU needs conda, or uses names that do not exist.

Cause. FAISS changed. Official pip wheels now exist, and several internals and names were renamed in recent releases.

Fix. Check the date of any tutorial and compare its commands with the official documentation and the version you installed. Pin the version in your notes.

Mistake 17: loading an index file you did not make

Symptom. None, until something goes wrong. That is the problem.

Cause. read_index does not verify the file. The project documentation warns that a crafted file could cause memory errors or code execution.

Fix. Load only your own or trusted files. Keep a SHA-256 checksum, and store index files in a place with restricted write access.

Habits worth adopting on day one

Keep a flat baseline

Whatever index you build, keep the ability to produce exact results for a sample of queries. A flat index over a few thousand vectors, or faiss.knn over a sample, costs almost nothing and turns "it seems fine" into a number. Recall is the only honest measure of an approximate index.

Print shapes and counts

Before every add and search, print x.shape, x.dtype, index.d and index.ntotal while you are developing. Most beginner problems show up in those four values. Once the code works, convert the prints into assertions so that the program fails loudly if the data changes.

assert x.dtype == np.float32 and x.ndim == 2 and x.shape[1] == index.d

Seed your randomness

When you use random vectors for experiments, create them from np.random.default_rng(seed). Two runs with the same seed give the same data, so any change in results is due to your change and not luck.

Save the pieces together

Write the index, the id-to-document table, the name of the embedding model, the FAISS version, and the build date to the same folder. Treat the folder as a single artefact. Teams that use experiment trackers such as MLflow often log the folder as an artefact so that a result can be traced back to its index.

Learn one index at a time

Master flat, then IVF, then HNSW. Do not try to understand every factory string before you have measured recall on the simple ones. The simplest index that meets your latency and accuracy needs is the right index.

Keep tests small and real

For development, use a small real sample of your own embeddings: a thousand items are enough to check that cosine search returns sensible neighbours. Random vectors are fine for checking shapes and speed, but they hide the behaviour of approximate indexes, because random data has no cluster structure.

Write down the metric

At the top of your script, state in a comment which metric you chose and why: "cosine; embeddings normalised; IndexFlatIP". Six months later, the person debugging the pipeline, who might be you, will be grateful.

Limit threads on shared machines

In containers or on shared servers, set OMP_NUM_THREADS to a value that matches the CPU you have been given. Using too many threads makes everything slower. When you benchmark, use the same setting for every index type you compare.

Read the first line and the last words of an error

C++ errors arrive in Python as RuntimeError: Error in <function> at <file>:<line>: <message>. The message after the last colon tells you what is wrong; the function and file tell you where to search the documentation or source. A line starting Faiss assertion means the process is about to die, so there will be no traceback to catch.

Practice cards

Each card is a ten-minute exercise. Do them in order.

Card 1: the self-search. Build a flat index of 1,000 random vectors and search with the first five of them. Confirm the nearest ids are 0 to 4 with distance close to zero. Then change one vector's dtype to float64 before adding and see what happens.

Card 2: both directions. Build IndexFlatL2 and IndexFlatIP on the same unnormalised data. Compare the top results. Then normalise and compare again. Explain in two sentences why they now agree.

Card 3: recall. Build an IVF index with nlist = 256 on 50,000 random vectors. Measure recall against flat for nprobe of 1, 2, 4, 8, 16, 32. Draw the numbers on paper.

Card 4: the warning. Train an IVF index with nlist = 1024 on only 2,000 vectors. Read the warning. Calculate the minimum training size it asks for.

Card 5: custom ids. Wrap a flat index in IndexIDMap, add ten vectors with ids from 900 to 909, and search. Try add without ids and read the error. Try wrapping an index that already has vectors and read that error too.

Card 6: save and reload. Write an index and a JSON list of documents. In a new Python process, load both, and compare ntotal to the list length. Delete one document from the list and see how to detect the problem.

Card 7: the factory. Build "Flat", "IVF256,Flat" and "HNSW32" with index_factory, and compare the build time and search time for 100,000 vectors. Set nprobe with ParameterSpace.

Card 8: a wrapper no-op. Wrap an IVF index in IndexIDMap, set .nprobe on the wrapper, and measure whether recall changes. Then set it through ParameterSpace and compare.

Card 9: batching. Time 1,000 queries sent as one batch and as 1,000 single-row calls. Write down the ratio.

Card 10: real text. Embed twenty sentences with a small sentence-embedding model, build a cosine index, and ask five questions. Check that the right sentence is first. Deliberately skip the query normalisation and observe the change.

Operational recipes

Check your environment in one command.

python -c "import faiss, numpy; print(faiss.__version__, numpy.__version__, faiss.get_num_gpus())"

This prints the FAISS version, the NumPy version, and the number of visible GPUs (0 on a CPU package).

Pin and freeze.

python -m pip install "faiss-cpu==1.15.1"
python -m pip freeze > requirements.txt

Limit threads for a script.

OMP_NUM_THREADS=4 python search_app.py

Find which Python you are running.

which python
python -m pip --version

If the path of pip and python differ, you will install into one and run the other.

Checksum an index.

shasum -a 256 vectors.index

Store the output with the file. Compare it before loading in another environment.

Size check for planning. For a flat index the file is about ntotal x d x 4 bytes plus a small header. If the file is far larger or smaller than that, you may have the wrong index type or an unexpected d.

The short checklist

• Data is float32, two-dimensional, shape (n, d), C-contiguous.
• A single query is reshaped to (1, d).
• I know whether my metric is L2 (smaller is better) or inner product (larger is better).
• For cosine, I normalised both stored vectors and queries, in place.
• I checked is_trained and trained IVF indexes on a representative sample.
• I set nprobe or efSearch and measured recall against a flat baseline.
• Ids map to my documents, and the mapping is saved with the index.
• index.ntotal equals the number of documents after every load.
• I will rebuild the index if I change the embedding model.
• I only load index files I created or trust.
• FAISS is pinned to a version, and faiss-cpu and faiss-gpu are not mixed.

Once these habits are automatic, the next level of the series will feel like an extension rather than a new subject: compression to save memory, filtering by id, per-query search parameters, and tuning with measured trade-offs.

This is part two of three. It picks up where Beginner ended and takes every topic from there further: the flat index you already trust, the first IVF index you built, the nprobe knob you turned by hand. Here we explain why they behave as they do, so you can predict recall, memory and latency before you run anything. The guide is written against FAISS 1.15.1.

Where this picks up

Each section opens with the problem it solves and closes, in most cases, with a Try it you can run in a few minutes.

One framing point before we start. FAISS is a library, not a database. It has no server, no authentication, no persistence layer and no query language. Everything a vector database gives you for free, you build around it, and the sections on serving and artifacts are about doing that well. If you find yourself building too much of that, the final section on choosing tools is for you.

How a search runs inside an IVF index

At Beginner you learned the recipe: train, add, set nprobe, search. To tune an index you need the mechanism, because every tuning decision is a statement about one of its three stages.

Stage one, training. train(x) runs k-means on your sample and stores nlist centroids in the coarse quantizer, which is itself an index (usually IndexFlatL2). Nothing about your database vectors has been stored yet. Training is a one-way step: FAISS does not support retraining, so the states are untrained, then trained and empty, then trained and populated, and reset() returns you to trained and empty. If the data distribution changes, you build a new index.

Stage two, adding. For each vector, FAISS asks the coarse quantizer for the nearest centroid and appends the vector (and its id) to that cell's inverted list. In IndexIVFFlat the list stores the raw vectors; in IndexIVFPQ it stores compressed codes. Each entry also carries an 8-byte id, which is why IVF always costs eight bytes per vector on top of the code.

Stage three, searching. For each query, the coarse quantizer returns the nprobe closest centroids, and FAISS scans only those lists, keeping the best k. The cost is roughly nprobe / nlist of a full scan plus the small cost of finding the centroids. When nprobe equals nlist you have an exhaustive search, and results match flat for IVFFlat.

This explains the failure you have probably already seen. If the true nearest neighbour lives in a cell whose centroid is not among the nprobe closest to the query, no amount of scanning will find it. Queries that fall near a boundary between cells are the vulnerable ones. Raising nprobe fixes this by widening the net, and it is the single most effective recall lever you have. The default is nprobe = 1, taken from IndexIVF.h, which is a deliberately cheap starting point and almost never what you want in production.

It also explains why training data matters so much. The centroids define where the boundaries sit. If you train on a sample that does not resemble your queries and database, the cells will be badly shaped: some huge, some nearly empty, and the huge ones dominate your scan time. You can inspect this directly.

import faiss, numpy as np

index = faiss.index_factory(d, "IVF4096,Flat")
index.train(xt)
index.add(xb)

ivf = faiss.extract_index_ivf(index)
sizes = np.array([ivf.get_list_size(i) for i in range(ivf.nlist)])
print("mean", sizes.mean(), "max", sizes.max(), "empty", (sizes == 0).sum())

A healthy index has a max within a few multiples of the mean and very few empty cells. A max that is fifty times the mean tells you your training sample is unrepresentative, or your vectors are heavily clustered, and queries landing in the big cells are slow however you set nprobe.

The same three-stage pattern applies to the compressed variants. Only what the lists hold changes, and with it the speed and the accuracy of the scan.

Choosing nlist and training well

Beginner told you to pick nlist and train. Now the choice has to be justified, because it fixes both your build cost and your memory floor.

The official guidance, from the "Guidelines to choose an index" wiki page, is a table keyed on the number of vectors. Below one million vectors, use IVF with K between 4*sqrt(N) and 16*sqrt(N), and train on between 30*K and 256*K vectors. From one million to ten million, move to IVF65536_HNSW32. From ten million to a hundred million, IVF262144_HNSW32; beyond that, IVF1048576_HNSW32. The _HNSW32 suffix swaps the coarse quantizer for an HNSW graph, because once you have tens of thousands of centroids, finding the closest nprobe of them by brute force costs as much as the scan you were trying to avoid.

There are two sides to choosing nlist. A larger value makes cells smaller, so each scan touches fewer vectors, but the quantizer step is more expensive and you need more training data to place the centroids well. A smaller value is cheap to train and probe but scans larger cells. The sweet spot depends on N, which is why the guideline is expressed as a multiple of its square root.

Training volume has a floor you can see in the logs. The k-means implementation defaults to min_points_per_centroid = 39 and max_points_per_centroid = 256. If you provide fewer than 39 points per centroid it prints a warning such as WARNING clustering 1000 points to 1024 centroids: please provide at least 39936 training points. More than 256 per centroid and FAISS silently subsamples. The FAQ adds that quality gains flatten out beyond about 1000 times the centroid count in points and about 20 iterations. The default niter is 25, so you are already there.

What you train on matters as much as how much. The sample must be drawn from the same distribution as the data you will index and, ideally, the queries you will serve. Drawing the first N rows of a file sorted by date or category is a classic way to build a quietly bad index. Sample uniformly instead:

rng = np.random.default_rng(42)
n_train = 64 * 4096                         # about 64 points per centroid
sample = rng.choice(len(xb), size=n_train, replace=False)
xt = np.ascontiguousarray(xb[sample], dtype="float32")

index = faiss.index_factory(d, "IVF4096,Flat")
index.train(xt)
assert index.is_trained

Remember the dtype and contiguity rule from the fact sheet: core APIs want float32 row-major arrays. The high-level Python wrappers convert for you, but a float64 array silently doubles your memory during the copy, and the low-level swig_ptr APIs do not convert at all.

One more point on metrics. For cosine similarity, normalize vectors with faiss.normalize_L2(x) before training, before adding and before every query batch, then use an inner-product index. Normalizing only at add time is a frequent bug: the centroids were trained on unnormalized data, and the queries are unnormalized, so results look plausible and are quietly wrong. Put normalization in one function that every path calls. Remember also that normalize_L2 works in place and requires float32, so it will modify the array you hand it.

Finally, do not expect L2 distances to be what you printed from NumPy. METRIC_L2 returns squared Euclidean distance, with no square root. If a downstream threshold was tuned on true distances, take the root yourself.

Product quantization, SQ and what compression costs

An IVFFlat index stores every vector in full, so the memory is 4*d bytes per vector plus the id. At 768 dimensions that is about 3 KB per vector, so 100 million vectors need roughly 307 GB. Compression is how you fit that into a machine, and it is also a speed lever, since scanning smaller codes moves less memory.

Scalar quantization (SQ) is the gentle option. SQ8 stores each component in one byte, so the code is d bytes, a four-fold saving over float32, with usually modest recall loss. SQfp16 stores two bytes per component. In the factory you write IVF4096,SQ8. SQ learns per-dimension ranges, which is part of why it needs training. Note that in 1.15.0 the quantizer previously named SQI8 was renamed SQ8, so older code that uses the old name needs updating.

Product quantization (PQ) is the aggressive option. It splits each vector into M sub-vectors and, for each sub-space, learns a codebook of 2^nbits centroids. A vector becomes M small integers, a code of M*nbits/8 bytes. A 768-dimensional vector with PQ64 (64 sub-vectors, 8 bits each) takes 64 bytes instead of 3,072: a 48-fold reduction. Two constraints follow from the definition. d must be divisible by M, which is the usual cause of could not parse index string errors on strings like IVF4096,PQ16 when d is, say, 100. And the distances computed from codes are approximations, so the ranking is perturbed.

In code, the explicit constructor takes nlist, M and nbits in that order:

d = 768
pq = faiss.IndexIVFPQ(faiss.IndexFlatL2(d), d, 4096, 64, 8)   # nlist, M, nbits
pq.train(xt)
pq.add(xb)
pq.nprobe = 32

Memory per vector is now M bytes of code plus 8 bytes of id. For 100 million vectors that is about 6.4 GB of codes plus 0.8 GB of ids, around 7.2 GB, against 307 GB for flat. This is the calculation you should be able to do on a whiteboard.

OPQ adds a learned rotation before PQ so the sub-vectors carry more balanced information. It appears in factory strings as OPQ16_64, meaning a rotation to 64 dimensions feeding a 16-sub-vector PQ. It is a pre-transform, so the final object is an IndexPreTransform wrapping an IVF index, which matters later when you need to reach the inner IVF object.

FastScan is PQ redesigned around SIMD registers. With 4-bit codes (PQ32x4fs), the lookup tables for distance computation fit in CPU registers, and scanning becomes much faster than classic PQ. The r suffix in PQ32x4fsr means the per-list data is rearranged for the scan. Because 4-bit codes are coarser, FastScan is usually paired with a refinement stage, which we cover next.

The factory string is a compact way to express all this, and reading one fluently is a mid-level skill. Take OPQ16_64,IVF4096,PQ16x4fsr,RFlat. Reading left to right: rotate with OPQ into 64 dimensions using 16 sub-blocks; partition into 4,096 cells; encode with 16 four-bit FastScan sub-quantizers; then keep a flat copy for re-ranking.

RaBitQ is the most aggressive entry. It is worth knowing it exists, and that its multi-bit forms trade size for accuracy. The detail of tuning it belongs at the next level.

HNSW in practice

If IVF narrows the search by partitioning space, HNSW narrows it by walking a graph. Each vector is a node linked to M neighbours, arranged in layers: sparse upper layers for long jumps, a dense bottom layer for the final precise walk. A query enters at the top, greedily moves toward the query, descends, and widens its beam at the bottom layer. No training is needed, which makes HNSW pleasant to build.

The parameters are three. M is the number of links per node, and the factory default is 32. efConstruction controls how wide the candidate beam is when inserting, with a default of 40. efSearch controls the beam at query time, defaulting to 16. The first two are fixed once the graph is built, and the last is yours to tune per query.

hnsw = faiss.IndexHNSWFlat(d, 32)          # M = 32
hnsw.hnsw.efConstruction = 64              # set BEFORE adding vectors
hnsw.add(xb)

D, I = hnsw.search(xq, 10, params=faiss.SearchParametersHNSW(efSearch=128))

The ordering of that code is the first trap. efConstruction is read during add, so setting it afterwards changes nothing about the graph you already built. The second trap is efSearch below k. A beam narrower than the number of results you ask for cannot return good results, so keep efSearch at or above k, and raise it until recall stops improving.

HNSW has trade-offs you should be able to state precisely.

• Memory. About d*4 + M*2*4 bytes per vector, so roughly 256 bytes of links on top of the raw vector for M = 32. It is a RAM-heavy index.
• Deletion. remove_ids is not supported. Calling it raises remove_ids not implemented for this type of index.
• Ids. HNSW supports only sequential ids. For custom ids, wrap it in IndexIDMap (the factory string IDMap,HNSW32), and wrap it before adding anything.
• No training, so you can start adding immediately and grow incrementally.
• Build time. Construction is more expensive than a flat add, and it scales with efConstruction.

Two details from recent releases are worth knowing. In 1.15.0 the HNSW visited-table hash-set threshold went from 500,000 to 10 million, and the visited table is reused across searches, which helps repeated query throughput on large indexes. And in 1.15.1 the lock-based graph-build flag was removed after an opt-in deterministic lock-free build (hnsw_deterministic_build) arrived. If you find a tutorial that flips a lock-related HNSW flag, it predates this change, so leave it out.

HNSW is also used inside IVF. IVF65536_HNSW32 means "IVF with 65,536 cells, whose coarse quantizer is an HNSW graph", the large-scale recipe from the previous section. When you need to tune that quantizer at query time, the nested parameter form reaches it:

params = faiss.SearchParametersIVF(
    nprobe=32,
    quantizer_params=faiss.SearchParametersHNSW(efSearch=128),
)
D, I = index.search(xq, 10, params=params)

When should you choose HNSW over IVF? Pick HNSW when RAM is plentiful, latency matters, the dataset is static or append-only, and you do not need deletion. Pick IVF with compression when memory is the constraint or you need removals.

Refinement: buying accuracy back

Compression loses accuracy. Refinement recovers most of it cheaply, and it is the technique that makes aggressive codes usable in production.

The idea is two-stage retrieval inside one index. The compressed index fetches more candidates than you need, k * k_factor of them, and a second stage recomputes their distances with a more accurate representation (the original vectors, or a finer code) and re-sorts. Because only a few hundred candidates are re-scored rather than the whole database, the extra cost is small.

In the factory, RFlat as a final element does exactly this with flat, exact vectors:

index = faiss.index_factory(d, "IVF4096,PQ32x4fs,RFlat")
index.train(xt)
index.add(xb)

ps = faiss.ParameterSpace()
ps.set_index_parameters(index, "nprobe=32")
D, I = index.search(xq, 10)

Here nprobe reaches the IVF layer through the opaque index, and the number of extra candidates the refine stage re-ranks (the k_factor) is a runtime parameter of the refine layer that ParameterSpace also knows about. Rather than guessing its string name, let ParameterSpace.explore, from the next section, enumerate and report the combinations for your build.

The price is memory: RFlat keeps the full-precision vectors, so you pay 4*d per vector again, though they may live in a separate flat index that is only touched for the shortlisted candidates. That is the trade you are making consciously. If the compressed codes serve the scan and the flat vectors serve the re-rank, you get PQ-like scan speed with near-exact final ordering. If you cannot afford to store full vectors at all, refine with a finer code (Refine(...) with a different quantizer), or accept the recall of the compressed index.

Think of the knobs in this stack as a ladder, each of which you can measure: nprobe (how many cells), the compression level (how coarse the scan), k_factor (how many candidates to re-rank), and the refine quality. A useful habit is to change one rung at a time against a fixed query set, so you know which one bought each gain.

Measuring recall properly

Every claim above is a claim about recall, so you need a way to measure it that you trust. The only valid yardstick is exact search on the same data, which is why flat never goes away: it becomes your ground-truth generator.

Define recall@k as the fraction of the true top-k neighbours (from flat) that the approximate index also returned in its top-k. Compute it on a held-out set of queries that look like production traffic, not on vectors you added to the index, since a database vector's nearest neighbour is itself and flatters every index.

def recall_at_k(I_approx, I_true, k):
    hits = 0
    for approx, true in zip(I_approx[:, :k], I_true[:, :k]):
        hits += len(set(approx.tolist()) & set(true.tolist()))
    return hits / (len(I_true) * k)

flat = faiss.IndexFlatL2(d)
flat.add(xb)
_, I_true = flat.search(xq, 100)

_, I_ann = index.search(xq, 10)
print("recall@10", recall_at_k(I_ann, I_true, 10))

Remember to record latency alongside it. A recall number without a queries-per-second number is half a measurement, and you should always report both for a configuration, because the whole point of tuning is the trade between them.

FAISS ships a tool for sweeping that trade: ParameterSpace. It enumerates the runtime parameters of an index (nprobe, quantizer_efSearch, k_factor, and so on) and explores combinations, keeping only those on the speed/accuracy Pareto frontier.

gt_I = I_true[:, :1]                                  # 1 nearest neighbour per query
crit = faiss.OneRecallAtRCriterion(len(xq), 1)        # 1-recall@1
crit.set_groundtruth(None, gt_I)

ps = faiss.ParameterSpace()
ps.initialize(index)
op = ps.explore(index, xq, crit)
op.display()

The output is a table of operating points, each a parameter string with its accuracy and time. The first column to read is the cheapest point that meets your recall target, and that point, not the highest-recall point, is what you deploy. A recall SLO phrased as "at least 0.95 recall@10 at p95 under 20 ms" turns a vague tuning exercise into a search over a short table.

Two cautions. First, ParameterSpace measures with the batch you give it, so use realistic batch sizes. Second, an accuracy criterion such as OneRecallAtR has a specific definition (whether the true nearest neighbour appears within the top R), which differs from recall@k. Decide which your product needs and use the matching criterion.

A last sanity step: build a small canary set of 200 to 500 representative queries with stored flat ground truth, kept next to the index artifact. Re-running it after any rebuild, library upgrade or parameter change takes seconds and catches regressions before users do.

Ids, deletion, updates and filtering

Real data changes. Documents are deleted, edited and tenant-scoped, and FAISS offers a limited but sufficient toolkit for this, provided you know exactly where the limits are.

Ids. FAISS ids are signed 64-bit integers, always. There are no string ids and there never will be, so you keep an external mapping from your document identifiers to int64 ids. A common pattern is to store that mapping in the same artifact bundle as the index so they cannot drift apart.

Custom ids need an index that supports add_with_ids. IndexIVF* indexes do natively. Flat and HNSW store sequential ids only, so wrap them in IndexIDMap (or IndexIDMap2 if you also need reconstruct) before adding anything. Wrapping a non-empty index fails with index must be empty on input.

Deletion. remove_ids works on flat, IVF and IDMap indexes, and fails on HNSW. On IVF, removal needs to find each vector's location. Without help, FAISS scans the lists, which is slow at scale. Enabling a direct map makes it O(1) per id and also enables reconstruct:

ivf = faiss.extract_index_ivf(index)
ivf.set_direct_map_type(faiss.DirectMap.Hashtable)

n_removed = index.remove_ids(np.array([10_001, 10_002], dtype="int64"))

Set the direct map before adding data. DirectMap.Array is cheaper but cannot be combined with custom ids; trying it raises cannot have array direct map and add with ids, so use Hashtable when you assign your own ids. Removal compacts the lists, which shifts internal positions, which is a further reason FAISS works with ids rather than positions.

Updates. There is no in-place update. An update is a removal followed by add_with_ids with the new vector. In a service that must stay available, perform that pair under a write lock, as discussed in the threads section.

Filtering. FAISS can restrict a search to a subset of ids by passing an IDSelector through the search parameters. The selectors cover ranges, arrays, hashed batches with a Bloom filter, bitmaps, and boolean combinations:

sel = faiss.IDSelectorBatch(np.array([1, 5, 9], dtype="int64"))
D, I = index.search(xq, 10, params=faiss.SearchParametersIVF(sel=sel, nprobe=32))

tenant = faiss.IDSelectorRange(2_000_000, 3_000_000)    # [imin, imax)
D, I = index.search(xq, 10, params=faiss.SearchParametersIVF(sel=tenant, nprobe=32))

Understand what this is and is not. It filters on ids only. There is no metadata filtering, so any attribute logic ("category is shoes and price under 50") must be translated by you into a set of ids first. And the filter is applied while scanning, so a very selective filter still scans the visited cells and can return fewer than k results, padded with -1. When you see -1 ids with 3.4028235e+38 distances, either k exceeded the number of eligible vectors or your filter excluded nearly everything in the probed cells.

For multi-tenancy, three designs exist. One index per tenant gives the best isolation and speed, and is what the project recommends for attributes with few discrete values. A shared index with tenant-encoded id ranges plus IDSelectorRange is simpler to operate but slower. Oversampling k and post-filtering works for mild filters. Choose by tenant count and size: thousands of tiny tenants favour one shared index, a few large ones favour separate indexes.

Threads, batches and concurrency

A common production surprise is that a FAISS service is fast in a benchmark and slow behind a web framework. The cause is almost always how work reaches the library.

FAISS parallelizes with OpenMP, and its search is designed for batches. A call with 1,000 queries spreads them across cores and uses matrix multiplication to amortize work. A call with one query gets little parallelism and pays full overhead. So throughput comes from batching, and a service that handles each request with a one-vector search will use a fraction of the hardware.

The practical design is micro-batching: collect requests for a few milliseconds, search them together, and fan the results back out. This trades a small, bounded latency for a large throughput gain, and you tune the window against your latency SLO.

Thread count matters. Set OMP_NUM_THREADS to the number of physical cores, not logical ones, and do not run many batched searches from several Python threads at once, because each would spawn its own OpenMP team and oversubscribe the cores. If you call faiss.omp_set_num_threads(8) in code, remember that the setting is thread-local: it applies to the thread that called it, so a call from your main thread has no effect on a worker thread that runs the search. Set the environment variable, or call the function in the thread that runs FAISS.

With the OpenBLAS build, a second oversubscription arises when OpenMP threads each start pthread-based BLAS threads. The documented fix is export OMP_WAIT_POLICY=PASSIVE, and on Ubuntu to prefer the libopenblas0-openmp package. Your symptom is brute-force searches that are slower than expected and CPU usage far above core count.

Now concurrency safety. Concurrent searches on a CPU index are safe. Mutation (add, remove_ids, train) concurrent with searches is not, and can crash or corrupt results. So a service that ingests and serves needs a reader-writer discipline: many simultaneous readers, one exclusive writer. A cleaner pattern is to avoid mutation of the serving index entirely, building a new index in the background and swapping a reference atomically. This pairs naturally with the artifact workflow in a later section.

For per-request parameters, use SearchParameters, passed through the keyword-only params argument, rather than assigning attributes on the shared index:

def search(xq, k, nprobe):
    index = current_index                 # a plain reference read
    params = faiss.SearchParametersIVF(nprobe=nprobe)
    return index.search(xq, k, params=params)

Because the parameters live in a per-call object, two requests with different nprobe values cannot interfere. There is a related trap with wrappers: setting index_shards.nprobe = 123, or cpu_index.index.nprobe = 123, silently does nothing, because Python just attaches a new attribute to a proxy object. Use ParameterSpace().set_index_parameter(idx, "nprobe", 123) or reach the IVF object with faiss.extract_index_ivf.

Persistence, IO flags and memory mapping

An index in RAM disappears with the process, so persistence is a design problem you own. faiss.write_index(index, path) and faiss.read_index(path) are the basic pair, and serialize_index / deserialize_index give you the same as a NumPy uint8 array, convenient for object storage clients that take bytes.

A few behaviours matter in practice. Writing needs an existing directory, and the error could not open <path> for writing: No such file or directory means exactly that. A GPU index cannot be written directly. Convert it first with faiss.index_gpu_to_cpu, then write. Binary indexes use their own pair, write_index_binary and read_index_binary; passing one to read_index yields Index type ... not recognized, as does a file written by a newer FAISS than the one reading it. The format is backward compatible (new reads old) but not forward compatible (old reading new), so upgrade readers before writers.

read_index accepts an io_flags argument that changes how loading happens. The two you will use most:

idx = faiss.read_index("big.index", faiss.IO_FLAG_MMAP)        # map inverted lists
idx = faiss.read_index("big.index", faiss.IO_FLAG_READ_ONLY)   # read-only on-disk lists

IO_FLAG_MMAP memory-maps the inverted lists instead of copying them into process memory. The operating system pages data in on demand, so start-up is nearly instant and resident memory stays low until queries touch the data. That is attractive for large IVF indexes, and for running many replicas on one host, since mapped pages can be shared through the page cache. The cost moves from RAM to SSD reads, so first queries after a cold start are slower, and performance depends on storage latency. It works best on local SSD, and the documentation notes that on-disk inverted lists are known to fail under some environments such as ASAN and certain VMs, so test on your actual target.

Two other points. Adding vectors to on-disk inverted lists is described as very inefficient, so treat mapped indexes as read-only serving artifacts built elsewhere. And mapping helps only when the bulk of the bytes sit in the inverted lists. An HNSW graph is mostly pointer-chasing and benefits less.

A working layout for an index artifact is a directory, versioned and immutable once written:

faiss-index/2026-10-02-a1b2c3d/
  index.faiss          # the FAISS file
  id_map.parquet       # int64 id -> document id
  manifest.json        # faiss version, embedding model, d, nlist, nprobe, build date
  canary_queries.npy   # 500 held-out queries
  canary_truth.npy     # flat ground-truth ids
  SHA256SUMS           # checksums of the above

The manifest records what is needed to know whether an index can be used with a given query encoder. A changed embedding model requires a new index, because vectors from different models are not comparable even when the dimension matches. Storing the model name and version in the manifest, and having the service refuse to start on a mismatch, turns a silent quality bug into a loud startup failure.

Wrapping FAISS in a service

Because FAISS has no server, "deploying FAISS" means writing a small service. The shape is the same in most teams, and getting it right matters more than the choice of web framework. If you use FastAPI, the pattern below drops in directly.

The service has four jobs: load an index at start-up, validate and batch incoming queries, run search, and map internal ids back to documents. It should also expose health and readiness, report its index version, and shut down cleanly.

import os
from contextlib import asynccontextmanager
import numpy as np
import faiss
from fastapi import FastAPI, HTTPException

INDEX_PATH = os.environ["INDEX_PATH"]
EXPECTED_D = int(os.environ["EXPECTED_D"])
NPROBE = int(os.environ.get("NPROBE", "32"))

state = {}

@asynccontextmanager
async def lifespan(app):
    index = faiss.read_index(INDEX_PATH, faiss.IO_FLAG_MMAP)
    if index.d != EXPECTED_D:
        raise RuntimeError(f"index d={index.d}, expected {EXPECTED_D}")
    state["index"] = index
    yield

app = FastAPI(lifespan=lifespan)

@app.post("/search")
def search(body: dict):
    xq = np.asarray(body["vectors"], dtype="float32")
    if xq.ndim != 2 or xq.shape[1] != EXPECTED_D:
        raise HTTPException(422, "vectors must have shape (n, d)")
    faiss.normalize_L2(xq)
    params = faiss.SearchParametersIVF(nprobe=NPROBE)
    D, I = state["index"].search(xq, int(body.get("k", 10)), params=params)
    return {"ids": I.tolist(), "scores": D.tolist()}

@app.get("/readyz")
def ready():
    return {"ntotal": state["index"].ntotal}

Several decisions here are deliberate. The dimension check at start-up and per request avoids the AssertionError from assert d == self.d that otherwise escapes as a 500. Parameters come from the environment so that tuning is a configuration change, not a rebuild. The k value passes through int, because NumPy integer scalars cause TypeError: ... argument N of type 'faiss::idx_t'. And normalization sits inside the request path, in one place.

Notice also what the sketch does not do: it does not batch. A production version puts a micro-batching queue in front of search, as described in the thread section, and treats the FastAPI worker count carefully. Several worker processes each load their own copy of the index. With IO_FLAG_MMAP those copies share page-cache pages, but without it you multiply memory by the worker count, so prefer one process with an internal batcher, or mapped indexes.

Observability is yours to build, since FAISS exposes no metrics endpoint. Track queries per second, latency percentiles, batch size, ntotal, index version and process memory. The library's counters help for internals: faiss.cvar.indexIVF_stats exposes ndis (distance computations) and nq for IVF, and faiss.cvar.hnsw_stats does the same for HNSW. Enable collection with faiss.set_search_stats_enabled(True), then reset the counters between sampling windows. A rising ndis per query after a re-index is a clean signal that your cells became unbalanced. For export, publish these through Prometheus and trace requests with OpenTelemetry, so that retrieval latency appears as its own span inside an end-to-end RAG trace.

If your application is retrieval-augmented generation, remember that the framework wrappers are third-party code. LangChain's FAISS vector store, for instance, saves a pickled docstore next to the index and refuses to load it unless you pass allow_dangerous_deserialization=True. That flag name is accurate and worth taking at face value; see the security section. For framework specifics, consult LangChain, LlamaIndex or Haystack directly, since their signatures change on their own schedule.

Packaging, CI and Kubernetes

The install story improved a lot recently, and your Dockerfile should reflect it. Official wheels exist on PyPI as of 1.14.2 and 1.14.3: faiss-cpu, faiss-gpu and faiss-gpu-cuvs, all at 1.15.1. Older blog posts claiming there is no official pip package, or that the faiss-gpu wheel is stuck at 1.7.2, describe the old community builds. Conda remains the route the project's INSTALL.md calls supported, and uses the pytorch and conda-forge channels.

Pin exactly. A CPU service image looks like this, following the practices in the Docker guide:

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS runtime
WORKDIR /app

COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    python -m pip install -r requirements.txt     # contains faiss-cpu==1.15.1 and numpy

COPY app.py .
ENV OMP_NUM_THREADS=4
USER 10001
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

Do not install both faiss-cpu and faiss-gpu in one environment; both provide the faiss module and the winner is accidental. The GPU wheel supports Linux x86-64 only and pulls CUDA 12 runtime libraries from PyPI; the cuVS variant needs CUDA 13 and Python 3.11 or newer. Unofficial wheels such as faiss-gpu-cu12 are not part of the project, so keep them out of a supply chain you are responsible for.

The 1.14.0 release introduced dynamic dispatch: the pip wheels compile every SIMD level into one library and choose at runtime. This is good news for containers, because one image runs correctly on whatever CPU Kubernetes schedules it onto. You can see which level was chosen:

import faiss
print(faiss.__version__)                      # 1.15.1
print(faiss.SIMDConfig.get_level_name())      # the dispatched level
print(faiss.get_compile_options())

Add those three lines to a start-up log. If a performance regression appears after a node-pool change, a different SIMD level is a prime suspect. The loader also honours the FAISS_OPT_LEVEL environment variable, which is useful for A/B benchmarking.

A smoke test belongs in CI so that a broken build never ships. It should install the pinned wheel, build a tiny index, search it and assert the answer:

import numpy as np, faiss

def test_faiss_roundtrip(tmp_path):
    d = 64
    xb = np.random.default_rng(0).random((1000, d), dtype="float32")
    index = faiss.index_factory(d, "Flat")
    index.add(xb)
    path = str(tmp_path / "t.index")
    faiss.write_index(index, path)
    again = faiss.read_index(path)
    _, I = again.search(xb[:5], 1)
    assert (I[:, 0] == np.arange(5)).all()

Add a second, slower job that runs the canary queries against the candidate index and fails the pipeline if recall drops below the SLO. With GitHub Actions that is one more job gated on the build job, and it is the single most valuable check you can add.

On Kubernetes, three settings deserve attention. Set the container memory request equal to its limit and size it from your arithmetic plus headroom, because an index that exceeds its limit is OOM-killed with no warning. Set CPU requests to the physical cores you want FAISS to use, and make OMP_NUM_THREADS match, since OpenMP reads the host's core count by default and ignores your cgroup limit, which leads to oversubscription and throttling. And give the readiness probe a long enough start-up window for index loading, so the pod is not killed while still reading a 40 GB file.

The index file is not a safe input

FAISS has no authentication and no network surface of its own, so the security surface that matters is the file you load. The official wiki says plainly that no attempt is made to check that loaded data is correct, and that a faulty or maliciously crafted file could cause out-of-memory errors or even code execution.

Releases from 1.14.0 through 1.15.1 contain dozens of hardening fixes in deserialization: integer overflow, recursion depth, bounded allocations, invalid metric types, and in 1.15.1 a path-traversal fix in directory extraction. That is a good reason to stay current. It is not a reason to relax. The conclusion for your practice is to treat index files like executables.

• Load only what you built. Indexes produced by your own pipeline, stored in an access-controlled bucket, are fine. An index downloaded from a forum or a shared drive is not.
• Verify before loading. Record a SHA-256 checksum at build time, store it in the manifest, and have the service verify it before calling read_index. If you use MLflow or DVC for artifacts, their content hashes serve the same purpose.
• Restrict who can write. The identity that serves queries should have read-only access to the artifact bucket. The identity that builds should be the only one that can write.
• Mind the wrappers. Framework stores that pickle a docstore carry pickle's risks as well, which is why LangChain makes you opt in explicitly.
• Classify the file like its source. Embeddings can leak information about the text they came from, so an index of confidential documents is confidential. This is relevant where data-residency rules apply, as with many Gulf and Egyptian employers: keep index artifacts in the same region and under the same controls as the source corpus.
• Never expose the demo server. The bundled RPC client and server demo is documented as demonstration only and not for production or untrusted networks.

A short verification step in the loader costs nothing and closes the most likely hole:

import hashlib

def sha256(path):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()

if sha256(INDEX_PATH) != manifest["index_sha256"]:
    raise RuntimeError("index checksum mismatch, refusing to load")

Secrets are the other half. FAISS itself handles no credentials, but the surrounding system does: object-store credentials for artifacts and API keys for embedding calls. Use workload identity or your platform's secret manager, and do not bake credentials into the image next to the index.

Debugging beyond the basics

At Beginner the errors were about syntax. At this level they are about quality and performance, and each deserves a method rather than guesswork.

Recall is poor. Work down a fixed list. First, is it a data problem? faiss.MatrixStats(x).comments reports NaNs, duplicates, constant dimensions and normalization issues, and bad embeddings defeat any index. Second, is nprobe or efSearch still at its default? Third, did training see a representative sample, and enough of it? Fourth, was normalization applied consistently on all three paths (train, add, query)? Fifth, compare against flat on the same queries: if flat is also poor, the embeddings are the problem, not FAISS.

Searches return -1. The cells you probed held fewer than k eligible vectors, k exceeds ntotal, or a filter was too tight. Raise nprobe or efSearch, or lower k.

Latency is erratic. Check list-size skew with the earlier snippet, since a few giant cells cause long tails. Check for memory-mapped cold reads right after start-up. Check that no ingestion job is holding a lock or competing for cores. Check thread oversubscription: more threads than cores shows up as high CPU and low throughput.

A GPU build quietly runs on CPU. If hasattr(faiss, "StandardGpuResources") is false, or you see AttributeError: module 'faiss' has no attribute 'StandardGpuResources', the GPU extension did not load and the loader fell back to CPU. Run from faiss import _swigfaiss_gpu directly to surface the real import error, typically a missing CUDA library such as libcudart.so.

Crashes versus exceptions. Most errors surface as RuntimeError with the form Error in <function> at <file>:<line>: <message>. A hard assertion, such as Faiss assertion 'k <= GPU_MAX_SELECTION_K' failed, aborts the whole process, which in a service means a crash loop. Validate k and nprobe at the API edge so you never reach those assertions.

Slow or hung operations. faiss.TimeoutGuard(2.0) wraps a call and interrupts it via the interrupt callback, which is a useful bound on runaway searches during experiments.

Which SIMD module loaded. Call logging.basicConfig(level=logging.INFO) before import faiss and read the loader's messages, for example "Loading faiss with AVX2 support."

When you need to be sure which concrete object you hold, faiss.downcast_index(idx.index) returns the real subclass of a generic handle, and faiss.extract_index_ivf(idx) digs the IVF layer out of pre-transform wrappers so you can read nlist or set nprobe.

When FAISS is the right tool, and when it is not

You can now see what FAISS gives you: a fast, exhaustively tunable search kernel. You can also see what it leaves to you: persistence, replication, metadata filters, authentication, upserts and multi-tenancy. The honest question for any project is whether you want to own that.

FAISS is the right choice when the corpus is mostly static or rebuilt in batches, when you need maximum control over the speed, memory and recall trade, when search is embedded in a larger Python or C++ process, when you are doing offline work such as deduplication, clustering or evaluation, or when you need GPU search at large scale. Many vector databases embed FAISS as an engine, which tells you something about its quality.

A database is the better choice when you need frequent small updates, rich metadata filtering, many tenants, built-in replication, or a team that does not want to operate a custom service. Read the neighbouring guides for those options: Qdrant, Milvus, Weaviate, Pinecone and pgvector. A simple rule of thumb is that the moment you catch yourself writing a write-ahead log or a metadata query planner around FAISS, a database has become cheaper than your engineering time.

When evaluating either, measure your own data. Use the same ground-truth discipline from earlier, and compare recall, latency at your batch size and cost at your scale. Marketing numbers from a different embedding model and corpus tell you very little about yours.

Putting it all together

Here is one complete workflow, end to end, that uses every idea in this guide. The scenario: a support-article search with 5 million 768-dimensional embeddings, rebuilt nightly, served from Kubernetes.

Plan the memory. Flat would need 5,000,000 * 768 * 4 bytes, about 15.4 GB per replica. That is affordable but heavy to replicate. With IVF and PQ64, codes take 64 bytes plus 8 for ids, about 360 MB, and an RFlat store adds back the 15.4 GB but can sit on SSD behind a memory map. The design is IVF-PQ for the scan and refinement for accuracy, and you have the arithmetic to defend it.

Choose and train. For 5 million vectors the guideline suggests IVF65536_HNSW32. Draw a uniform sample of a few million vectors (at least 39 per centroid, ideally more), normalize it with the same function the service uses, and train.

Build and evaluate. Add in batches, write the index and a manifest, and compute recall@10 and latency on 500 held-out queries against flat ground truth. Use ParameterSpace.explore to find the cheapest nprobe and k_factor that meets the 0.95 recall target.

import faiss, numpy as np, json, hashlib

d = 768
index = faiss.index_factory(d, "IVF65536_HNSW32,PQ64,RFlat", faiss.METRIC_INNER_PRODUCT)

rng = np.random.default_rng(7)
sample = np.ascontiguousarray(load_embeddings(rows=rng.choice(N, 3_000_000, replace=False)))
faiss.normalize_L2(sample)
index.train(sample)

for batch in embedding_batches(size=100_000):
    faiss.normalize_L2(batch)
    index.add(batch)

faiss.write_index(index, "index.faiss")
manifest = {
    "faiss": faiss.__version__, "d": d, "embedding_model": MODEL_NAME,
    "ntotal": index.ntotal, "nprobe": 48, "index_sha256": sha256("index.faiss"),
}
json.dump(manifest, open("manifest.json", "w"))

Treat load_embeddings, embedding_batches and MODEL_NAME as your own pipeline's functions. The factory string above is a plausible composite of elements covered in the earlier sections, and you should confirm that a particular combination such as HNSW-quantized IVF plus PQ plus refinement builds and performs on your data before standardizing on it, since not every combination is equally well exercised.

Gate and publish. CI loads the candidate index, runs the canary queries, and fails if recall or latency regress against the last published index. A passing index is uploaded to an immutable, versioned path, with checksums.

Serve safely. The pods read the manifest, verify the hash, check d and the embedding-model name, map the index, and start answering. OMP_NUM_THREADS matches the CPU request, a micro-batcher feeds search, and per-call SearchParametersIVF carries nprobe. Metrics, traces and the index version flow to your monitoring stack, and a canary query runs periodically against the live service to catch drift.

Roll forward and back. A new index is a new path and a new deployment revision. Rolling back means pointing at the previous path, and because the format is backward compatible, new library versions can read old indexes, while the reverse is not guaranteed, so upgrade readers before writers.

Where you are now

You can now explain why an IVF search misses a neighbour and fix it with nprobe or better training. You can read a factory string, do the memory arithmetic for it, and justify compression and refinement choices. You know HNSW's parameters and its limits, including that it cannot delete. You can measure recall against flat ground truth and use ParameterSpace to pick the cheapest operating point that meets an SLO.

You can also run FAISS as a real service: batching queries, setting per-call parameters safely, mapping large indexes, pinning official wheels, smoke-testing in CI, sizing pods, and treating index files as untrusted input. And you can say, with reasons, when a vector database would serve you better.

The Senior level takes this from one service to a platform: GPU and cuVS indexes, sharding and replication, billion-scale builds, the newer index families such as RaBitQ and Panorama, upgrade and migration strategy, and the failure modes that only appear at scale.

Sources

• FAISS documentation
• FAISS repository
• FAISS releases
• CHANGELOG
• INSTALL.md
• Guidelines to choose an index
• The index factory
• Faster search
• Lower memory footprint
• Setting search parameters for one query
• Special operations on indexes
• Index IO, cloning and hyper-parameter tuning
• Threads and asynchronous calls
• Indexes that do not fit in RAM
• FAQ
• Troubleshooting
• faiss-cpu on PyPI
• The Faiss library paper

Part two of three. A cumulative review of Beginner and Mid-level material, organised by situation rather than by level. At this level interviewers stop asking what an index is and start asking what you would do when recall drops, memory runs out or latency spikes. The answers below are written the way a strong candidate would say them aloud, followed by what separates a strong answer from a weak one.

Where this picks up

The thirty-second answer

FAISS is Meta's open-source library for similarity search and clustering of dense vectors. It runs in-process, keeps everything in memory unless you map it, and offers a family of indexes that trade accuracy, speed and memory against each other. It is not a database: persistence, replication, metadata filtering and authentication are your job.

At mid level, add the sentence that shows you have run it: "I measure every index against an exact flat search on held-out queries, report recall and latency together, and deploy the cheapest configuration that meets the recall target."

Foundations you should say fluently

Three stages of IVF. Training learns nlist centroids by k-means. Adding assigns each vector to its nearest centroid's list. Searching probes the nprobe closest cells and scans only those. Recall is lost when the true neighbour sits in a cell that was not probed.

Memory per vector. Flat is 4*d bytes. SQ8 is d. PQ with M sub-vectors at 8 bits is M. IVF adds 8 bytes for the id. HNSW32 adds roughly 256 bytes of links. You should be able to produce these numbers without notes.

Metrics. METRIC_L2 returns squared distance. Inner product is a similarity, so larger is better. Cosine is normalize-then-inner-product.

Ids. Always signed 64-bit integers. No strings, no metadata. You keep an external mapping.

Training. Not repeatable. You cannot retrain an existing index, you build a new one.

Scenario questions

"Recall dropped after we added a lot of new documents. What do you check?"

A strong answer works through causes in order of likelihood and says how it would verify each.

First, I would confirm the drop is real by running a fixed canary query set against the live index and comparing it with flat ground truth computed on the same data. If I have no canary set, that is the first finding.

Second, drift against the trained centroids. IVF centroids were learned from the original data. If the new documents come from a different distribution, perhaps a new product line or a new language, they pile into a few cells, and those cells become both large and poorly separated. I would print the list sizes with get_list_size, look at the maximum against the mean and count empty cells. A skewed distribution means retraining on a sample from the current corpus and rebuilding.

Third, nprobe left at the default of 1, or tuned for the old data size. As the corpus grows, the same nprobe covers a smaller fraction of the neighbours.

Fourth, the embedding model. If someone switched model versions for new documents, the vectors are not comparable with the old ones, and the index needs a full rebuild. I would check the manifest for the model name.

Fifth, normalization consistency, if the metric is cosine.

What makes it strong: a verification step for each hypothesis, and the recognition that FAISS cannot retrain in place. What makes it weak: "increase nprobe" as the only answer.

"Search is fast in the benchmark but slow behind our API. Why?"

The likeliest cause is query shape. FAISS is optimised for batches. A benchmark searches thousands of queries in one call, while an API handler typically searches one at a time, which gets little parallelism and pays full overhead. The fix is micro-batching: collect requests for a few milliseconds, search together, and fan the results out.

Next, thread configuration. If OMP_NUM_THREADS is unset in a container, OpenMP may see the host's cores rather than the cgroup limit, causing oversubscription and CPU throttling. I would set it to the physical cores allotted. If the API server runs several worker threads each calling batched search, they oversubscribe the cores, so I would funnel searches through one batcher.

Third, faiss.omp_set_num_threads is thread-local, so a call in the main thread does nothing for a worker thread.

Fourth, cold memory-mapped pages after start-up, which would show as slow early requests that later improve.

I would confirm with the loader log and faiss.get_compile_options() that the expected SIMD level loaded, in case a node-pool change landed us on a slower CPU.

"The index no longer fits in memory. Options?"

I would start with arithmetic, because the right answer depends on the gap. For 100 million 768-dimensional vectors, flat needs about 307 GB. In order of increasing aggression:

1. Lower precision. SQfp16 halves it; SQ8 quarters it, usually with small recall loss.
2. Product quantization. PQ64 brings it to about 7.2 GB including ids, at some recall cost.
3. FastScan plus refinement. 4-bit codes scan fast and refinement restores ordering, though refinement needs the full vectors somewhere.
4. Memory mapping. Keep the bulk of the inverted lists on SSD and load with IO_FLAG_MMAP, trading RAM for storage latency.
5. Sharding. Split across machines and merge top-k results.
6. A different tool. If I am building replication and tiering myself, a vector database may be cheaper.

I would measure recall at each step against flat on a sample before committing, because the compression level that fits memory may not meet the recall target.

"A teammate wants to load an index someone shared on a forum. Thoughts?"

I would say no, and explain why. FAISS performs limited validation on loaded data. The project's own documentation says a faulty or malicious file could cause out-of-memory errors or, if expertly crafted, code execution. Recent releases hardened deserialization considerably, which is a good reason to upgrade, but it does not make untrusted files safe. We load only indexes our own pipeline produced, verify a recorded SHA-256 before loading, and keep artifact buckets write-restricted. Rebuild from the source embeddings instead if the data is genuinely needed.

"We need to delete documents. We built HNSW. Now what?"

HNSW does not support remove_ids; it raises remove_ids not implemented for this type of index. Realistic choices are to rebuild on a schedule while hiding deleted ids in the meantime with an IDSelectorNot or by over-fetching and post-filtering, or to switch to an IVF index, which supports removal. For IVF I would enable a hashtable direct map so removal is not a scan of the lists. The decision depends on delete volume. A trickle of deletions suits periodic rebuilds, while heavy churn argues for IVF or a database.

"Queries sometimes return fewer than k results, with -1 ids. Bug?"

Not a bug, an expected consequence of approximate search and filtering. Either k exceeds the number of eligible vectors in the cells probed, or a selective filter excluded most of what the scan saw. The fix is to raise nprobe or efSearch, lower k, or over-fetch. If every query returns -1 and a distance of 3.4028235e+38, check that the index has data and is trained.

"How do you roll out a FAISS upgrade safely?"

Pin the version in the image. Read the production index with the new version in staging, since new readers handle old files. Run the canary queries and compare recall and latency to the previous version. Read the changelog for behaviour changes, such as the 1.15.1 change making cuVS opt-in for GPU builds. Roll out serving replicas first. Only then let the build pipeline write indexes with the new version, because older readers cannot necessarily read newer files.

Trade-off questions

IVF versus HNSW

HNSW gives excellent speed and recall with no training, at the cost of high RAM, no deletion and sequential ids unless wrapped. IVF needs training and a well-chosen nprobe, but compresses well, supports removal and scales to billions with a good coarse quantizer. I choose HNSW when memory is plentiful and data is static or append-only, and IVF with compression when memory binds or deletes matter.

More cells or more probes

More cells make each cell small, so scans are cheap, but training needs more data and the coarse search costs more. More probes raise recall at linear scan cost. In practice I fix nlist from the corpus size using the official guideline of 4*sqrt(N) to 16*sqrt(N) below a million vectors, then treat nprobe as the runtime knob.

Compression versus refinement

Compression saves memory and speeds scanning but blurs ranking. Refinement re-ranks a shortlist with better distances, recovering accuracy, but needs the finer representation stored, usually full vectors, so it gives back memory. Refinement on SSD behind a memory map is a common compromise.

One index per tenant versus a shared index with filters

Per-tenant indexes give isolation and speed, and match the project's own recommendation for attributes with few discrete values. They cost memory overhead per index and operational complexity at many tenants. A shared index with id ranges and IDSelectorRange is simpler but slower, because filtering inspects every scanned id. I decide by tenant count and size distribution.

FAISS versus a vector database

FAISS wins on control, cost and embedding in your own process. A database wins on updates, metadata filtering, replication and operations. The rule I use: when I start building a write-ahead log or a filter planner around FAISS, the database is cheaper than my time. See the Qdrant, Milvus, Weaviate, Pinecone and pgvector guides in this catalogue for the alternatives.

Exact versus approximate

Exact flat search is right below a few hundred thousand vectors if latency allows, when the answer must be exact, or when I need ground truth. Approximate indexes are right when scale or latency makes brute force impractical and a measured recall below 1.0 is acceptable.

What a strong answer covers that a weak one misses

Rapid-fire questions

What does index.search return for a missing result? An id of -1, with a sentinel distance.

Why can IndexIDMap wrapping fail? The index must be empty. Wrap before adding.

What is nprobe by default? One, which is almost always too low.

Does normalize_L2 copy? No, it works in place on a float32 array.

What happens if d does not match? An AssertionError from the Python wrapper. Reshape single queries to (1, d).

Can you change efConstruction after adding? You can set it, but it only affects vectors added afterwards.

Which indexes train on data? IVF, PQ, SQ (other than fp16) and OPQ or PCA. Flat and HNSW do not.

What is the cost of an id in IVF? Eight bytes per vector.

How do you write a GPU index? Convert to CPU first with index_gpu_to_cpu, then write_index.

What does a hard assertion do? Aborts the process. It is a crash, not a catchable exception.

Common interview questions with model answers

Q: How would you choose an index for ten million vectors? A: Start from the memory budget and recall target. Ten million falls in the guideline's range for IVF65536_HNSW32. If RAM is plentiful and deletes are not needed, HNSW32 is a strong alternative. I would build both on a sample, measure recall and latency against flat, and pick the cheapest that meets the SLO.

Q: Why not just always use HNSW? A: Memory, deletion and training flexibility. It is RAM-heavy, cannot remove ids, and offers less compression than IVF-PQ. It is excellent when those do not bind.

Q: How do you do cosine similarity? A: Normalize with faiss.normalize_L2 and use IndexFlatIP or an IVF index built with METRIC_INNER_PRODUCT. The normalization must happen on the training sample, the database and every query batch.

Q: What do you monitor in production? A: FAISS has no metrics endpoint, so I instrument the service: QPS, latency percentiles, batch size, ntotal, index version, process memory, and a periodic canary recall check against stored ground truth. I also watch the library's internal counters such as distance computations per query to catch skewed cells.

Q: How do you make a FAISS service highly available? A: Treat indexes as immutable versioned artifacts built offline, replicate the serving pods behind a load balancer, and swap versions by rolling deployments. FAISS does no replication itself.

Q: What changed recently that you would mention? A: Official pip wheels now exist for faiss-cpu and faiss-gpu, so older advice that GPU support is conda-only is out of date. The extended-API names moved from an Ex suffix to _ex, cuVS replaced RAFT, and cuVS dispatch became opt-in in 1.15.1.

Sixty-second self-test

Answer aloud without looking back.

1. Name the three stages of an IVF search and say where recall is lost.
2. Compute the memory for 50 million 384-dimensional vectors as flat, SQ8 and PQ48, with ids.
3. Why must efConstruction be set before add?
4. Which index families cannot do add_with_ids natively, and what wraps them?
5. Give two reasons a service might be slower than its benchmark.
6. What does -1 in a result mean?
7. Why is loading a shared index file dangerous?
8. What would make you recommend a vector database instead?

Be honest with yourself while you do this: hesitation on the memory arithmetic or on the three stages of IVF is the most common gap interviewers find. If any answer took longer than ten seconds, reread the matching section of the detailed guide. The Senior level adds GPU indexes, sharding, billion-scale builds and platform ownership.

Part two of three. At this level your indexes build and your searches return results, so the problems change character. They are no longer "why won't this import" but "why did recall fall", "why is this index 40 GB" and "why is the service slower than the benchmark". Start with the error table, then the practices ranked by leverage, the practice cards, and the operational recipes.

Where this picks up

Common errors at this level

Cumulative: Beginner's errors still apply, and these are the ones that appear once things work.

Two entries deserve more than a line.

Training on the wrong sample. This is the most expensive mistake at this level because the index builds, searches and returns plausible results. The only symptom is poor recall on real traffic, discovered weeks later. Defend against it by drawing training data uniformly at random from the full corpus, and by evaluating on held-out queries that resemble production, never on vectors that were added to the index. A database vector's nearest neighbour is itself, so evaluating on indexed vectors flatters every configuration.

Inconsistent normalization. For cosine similarity you normalize and use inner product. The failure is normalizing in some paths and forgetting others, often because the query encoder lives in a different service. Make normalization a single shared function, and consider a startup check that embeds a known string and asserts the vector's norm is within tolerance.

The practices that pay off most

These are ordered by leverage: the first few change the outcome of a project, the last few polish it.

Why a canary set earns its place

A canary set is 200 to 500 representative queries with flat ground truth saved beside the index. It costs an afternoon to build and pays back every time anything changes: a rebuild, a library upgrade, a new embedding model, a changed nprobe. Run it in CI before an index is published and periodically against the live service. It converts "users say search got worse" into a number with a date on it.

Why per-call parameters matter

Setting index.nprobe = 64 is fine in a notebook. In a service it creates a race: two requests wanting different settings overwrite each other, and the result depends on timing. Passing SearchParametersIVF(nprobe=64) through params keeps the setting local to the call. It also makes your tuning auditable, because the setting appears in the code path rather than hiding in shared state.

Why immutable artifacts

Mutating a live index is where concurrency bugs and corrupted files come from. Search can run concurrently with other searches, but add and remove_ids concurrent with search can crash. Building a fresh index offline and swapping a reference removes that entire class of problem, and rollback becomes pointing at the previous directory.

Practice cards

Short drills. Each takes ten to twenty minutes on a laptop with random or sample embeddings.

Operational recipes

Recipe: a recall report you can paste into a pull request

import time
import numpy as np
import faiss

def evaluate(index, xq, I_true, k=10, **search_kwargs):
    t0 = time.perf_counter()
    _, I = index.search(xq, k, **search_kwargs)
    dt = time.perf_counter() - t0
    hits = sum(len(set(a.tolist()) & set(t[:k].tolist())) for a, t in zip(I, I_true))
    return {"recall@%d" % k: hits / (len(xq) * k), "qps": len(xq) / dt}

for nprobe in (1, 4, 16, 64):
    params = faiss.SearchParametersIVF(nprobe=nprobe)
    print(nprobe, evaluate(index, xq, I_true, params=params))

Run it for each candidate configuration and paste the table into the change. Reviewers can then argue with numbers.

Recipe: inspect a skewed IVF index

ivf = faiss.extract_index_ivf(index)
sizes = np.array([ivf.get_list_size(i) for i in range(ivf.nlist)])
print("nlist", ivf.nlist, "mean", sizes.mean(), "max", sizes.max(),
      "empty", int((sizes == 0).sum()))

A maximum many times the mean, or a large number of empty cells, points to an unrepresentative training sample or heavily clustered data.

Recipe: log what you are really running

import logging
logging.basicConfig(level=logging.INFO)    # before importing faiss, to see the loader
import faiss
print(faiss.__version__)
print(faiss.SIMDConfig.get_level_name())
print(faiss.get_compile_options())
print(faiss.get_num_gpus(), hasattr(faiss, "StandardGpuResources"))

Put this in your service's start-up log. After any node-pool or base-image change, a different SIMD level or a missing GPU shows up immediately.

Recipe: a manifest that prevents silent mismatches

{
  "faiss_version": "1.15.1",
  "index_file": "index.faiss",
  "index_sha256": "<recorded at build time>",
  "d": 768,
  "metric": "inner_product",
  "normalized": true,
  "embedding_model": "<name and version of the query encoder>",
  "ntotal": 5000000,
  "default_nprobe": 48,
  "built_at": "2026-10-02T03:00:00Z"
}

Have the service compare d, the metric and the embedding model with its own configuration at start-up, and refuse to start on a mismatch. A changed embedding model needs a new index, so this turns a silent quality bug into a loud one.

Recipe: a CI recall gate

name: index-gate
on:
  workflow_dispatch:
  pull_request:
    paths: ["pipelines/index/**"]
jobs:
  recall:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python -m pip install "faiss-cpu==1.15.1" numpy
      - run: python pipelines/index/check_canary.py --min-recall 0.95 --max-p95-ms 25

The script, which you write, loads the candidate index, runs the canary queries, and exits non-zero when recall or latency misses the threshold. Connect this to the practices in the GitHub Actions guide so the gate blocks publication.

Recipe: sizing a Kubernetes pod

resources:
  requests:
    cpu: "4"
    memory: 24Gi
  limits:
    memory: 24Gi        # equal to the request: an index over the limit is OOM-killed
env:
  - name: OMP_NUM_THREADS
    value: "4"          # match the CPU request, not the node's core count
readinessProbe:
  httpGet: { path: /readyz, port: 8080 }
  initialDelaySeconds: 60   # allow time to read the index
  periodSeconds: 10

Compute the memory line from your arithmetic, add headroom for the service and queries, and then confirm it under load. See the Kubernetes guide for requests, limits and probes in depth.

Recipe: a safe upgrade, step by step

1. Pin the new version in a staging image.
2. Read yesterday's production index with it.
3. Run the canary queries and compare recall and latency with the old version.
4. Read the changelog for behaviour changes. In 1.15.1, for example, cuVS dispatch became opt-in, so set use_cuvs explicitly on a GPU build.
5. Roll out the serving replicas.
6. Only then change the build pipeline to write indexes with the new version.

New readers handle old files, but old readers are not guaranteed to handle new files, so the order matters.

Recipe: when you do need to rebuild

Treat rebuilds as routine, not as an emergency. Schedule them when the corpus has grown by a set fraction, when the embedding model changes, or when the list-size statistics show drift. Build into a new directory, run the canary gate, publish, and roll out. Keep the previous artifact for rollback for as long as your retention policy allows.

Three diagnoses worth rehearsing

The error table gives one-line fixes. These three problems deserve a method, because in each the symptom points away from the cause.

"Search got slower after we re-indexed." Resist the urge to blame the hardware. Compare the old and new index on the same queries, and look first at the work done per query rather than the time taken. Enable faiss.set_search_stats_enabled(True), reset faiss.cvar.indexIVF_stats, run a fixed batch, and read ndis, the number of distance computations. If ndis per query rose, the index is doing more work: cells became larger or unbalanced, or a higher nprobe was baked into the new configuration. If ndis is unchanged and time rose, the cause lies outside the algorithm: a different SIMD level after a node change, memory-mapped pages that are not yet warm, thread oversubscription, or a noisy neighbour on the host. That single split, algorithmic work versus environment, removes half of the possible causes in a minute.

"Recall is great offline and poor online." The offline and online paths differ somewhere, and your job is to find where. Check that the online query encoder is the same model and version as the one that produced the index, since a silent encoder upgrade breaks comparability even when the dimension matches. Check normalization on the online path. Check that the online service applies the same nprobe that the evaluation used, since a default of one would explain everything. Then replay a sample of real production queries through the offline evaluation, because synthetic or held-out queries often differ in length, language and topic from real traffic. Where users write in Arabic and English mixed in one query, an English-only evaluation set will systematically overstate quality.

"The pod keeps restarting." Separate the two likely stories. If the exit reason is out-of-memory, the index plus the service exceeded the limit: recompute the arithmetic, remembering that loading can briefly need more memory than steady state, and that several worker processes each loading a copy multiply the footprint unless the index is memory-mapped. If the process dies without a Python traceback, suspect a hard assertion, for example a GPU k above 2048 or an unsupported PQ configuration, and validate inputs at the edge. If readiness fails before the index finishes loading, lengthen the start-up window rather than the liveness timeout, so the pod is not killed while it is still reading a large file.

In all three, notice the pattern. You form a hypothesis about where the two situations differ, you collect one measurement that splits the possibilities, and only then do you change something. That discipline is what separates tuning from guessing, and it is the same habit the Senior level relies on when the system is shared by many teams.

Habits worth adopting

• Write down your SLO before you tune. "0.95 recall@10 at p95 under 20 ms" turns tuning into a search over a table.
• Change one knob at a time against a fixed query set so you know what bought each gain.
• Keep ids and documents together. Store the id map in the same artifact as the index so they cannot drift.
• Log the index version with every response in debugging environments, so a complaint can be tied to a build.
• Keep the region in mind. Index files carry information about the source text. Where Gulf or Egyptian data-residency rules apply to the documents, apply them to the artifacts as well.
• Read the changelog before upgrading. Releases land every one to three months and some change defaults.
• Prefer official sources. Install faiss-cpu, faiss-gpu or faiss-gpu-cuvs from the project, not an unofficial wheel, and ignore tutorials saying there is no official pip package.
• Know when to stop. If you are building persistence, replication and filter planning around FAISS, look at Qdrant, Milvus, pgvector or the other guides in this catalogue.

A short checklist

Before you call an index production-ready, you should be able to answer yes to each of these.

• Do I have flat ground truth and a canary query set saved with the index?
• Is recall reported together with latency, and does it meet a written target?
• Was training done on a uniform sample of at least 39 points per centroid?
• Is normalization applied identically on training, adding and querying?
• Have I done the memory arithmetic and sized the container from it?
• Are search parameters passed per call rather than mutated on a shared index?
• Are queries batched, and does OMP_NUM_THREADS match the CPU request?
• Is the index an immutable, versioned artifact with a manifest and checksum?
• Does the loader verify the checksum and the embedding model before serving?
• Is the FAISS version pinned and exercised by a smoke test in CI?
• Do I know the rollback step and the upgrade order?

If you can tick all eleven, you are using FAISS the way people who run it in production do. The Senior level covers what changes when the index is a shared platform for many teams.

This is part three of three. The earlier levels taught you to build indexes and tune them. This one is about owning FAISS as the retrieval layer that other teams build products on: what the failure modes are, what you promise, what you refuse to promise, and when the honest answer is to use something that is not FAISS at all.

The whole level rests on one fact that is easy to forget once the library is working well: FAISS is a library, not a service. It has no daemon, no network API, no authentication, no persistence layer and no query language. Every property a vector database gives you for free, such as durability, replication, upserts, metadata filters, tenants and access control, is something you build around FAISS or something you do without. A platform owner's job is to decide, deliberately and in writing, which of those you build.

Where this picks up

Everything below uses FAISS 1.15.1, the stable release published on 16 September 2026. Where behaviour changed recently I say so, because a platform inherits every old tutorial your users have read.

Architecture and failure modes

A production FAISS deployment is almost always the same shape. An offline job embeds documents, trains the index on a representative sample, adds the vectors in batches and writes the result to object storage. A serving process downloads the file, loads it into memory (or memory-maps it), and answers search calls from a request handler. Around that sit the things FAISS does not provide: an id-to-document mapping, a metadata store, authentication and a way to roll a new index out.

The failure modes follow from this shape and from what FAISS refuses to do. It is useful to learn them as a list, because each one has a different owner.

Process-killing failures. C++ exceptions reach Python as RuntimeError, and you can catch them. Hard assertions are different: a message such as Faiss assertion 'k <= GPU_MAX_SELECTION_K' failed aborts the process. A caller who asks for k=5000 on a GPU index does not get an error response; your serving pod restarts. The platform has to validate k and nprobe before they reach the library, because the library will not give you a recoverable failure.

Silent degradation. Nothing in FAISS tells you recall has dropped. With IVF the default nprobe is 1, so an index someone forgot to tune returns confident, fast, wrong answers. A changed embedding model, a drifting data distribution, or a corpus that has outgrown the centroids trained on last quarter's sample will all lower recall without a single error in the logs. This is the failure mode that costs the most, because it surfaces as a product complaint rather than an alert.

Load-time failures. A file written by a newer FAISS may use an index type an older one rejects with Index type 0x........ ("xxxx") not recognized. A file that is truncated, or a binary index read with read_index, fails the same way. These show up at deploy time, which is the cheapest time to find them, provided you test a load before you send traffic.

Resource failures. Everything is in RAM unless you chose otherwise, so memory growth is a capacity event, and a load of a large index can fail the allocation after minutes of reading.

What breaks first as you scale

"Scaling FAISS" means different things at different sizes, and a platform owner should know which component fails first at each one, because the fix is different.

Up to a few million vectors, the first thing to break is usually not FAISS. It is the process around it: a request handler that calls search once per request from many threads, so that each call spawns its own OpenMP team and the machine oversubscribes. FAISS is optimized for batch queries, and its search parallelizes over the queries in a batch. A service that runs hundreds of single-query searches concurrently gets poor throughput and unpredictable tail latency. The fix is a batcher that accumulates requests for a few milliseconds and issues one search, with OMP_NUM_THREADS set to the number of physical cores. If you use OpenBLAS, oversubscription between OpenMP and the BLAS pthreads is a known cause of very slow brute force, and OMP_WAIT_POLICY=PASSIVE is the documented mitigation.

Between ten and a hundred million vectors, memory becomes the constraint. A flat index costs 4 times d bytes per vector, so 100 million vectors at 768 dimensions is about 307 GB. The same corpus with PQ64 is about 7.2 GB of codes plus roughly 0.8 GB of ids. HNSW32 adds about 256 bytes of links per vector on top of the vector itself. At this size the other thing that breaks is the coarse quantizer: with a flat quantizer and a large nlist, assigning vectors to cells dominates build time and part of search time. The official guidance moves to an HNSW coarse quantizer from about one million vectors, with factory strings such as IVF65536_HNSW32 for 1M to 10M, IVF262144_HNSW32 for 10M to 100M, and IVF1048576_HNSW32 for 100M to 1B.

Beyond one machine, you are building a distributed system out of a library. The options are IndexShards, which splits the data across sub-indexes and merges their top-k, and IndexReplicas, which copies the data and splits the queries. In practice most platforms do this at the service level rather than inside one process: several independent serving shards, a router that fans a query out, and a merge step that combines per-shard results. FAISS gives you merge_knn_results and ResultHeap for that merge. The trained quantizer pattern matters here: train once, then add each shard's data to a copy of the same trained empty index, so shards share centroids and their results are comparable.

On disk, IVF indexes can keep inverted lists on SSD through OnDiskInvertedLists or by reading with IO_FLAG_MMAP. That moves cost from RAM to SSD IOPS. It works best on SSD, adding to on-disk lists is documented as very inefficient, and the on-disk path is known to fail under some VMs and under ASAN. Treat it as a read-mostly tier, not a general substitute for memory.

Index choice as a platform decision

At the beginner and mid levels, choosing an index is a modelling exercise. At the platform level it is a contract with consequences for memory, rebuild time, upgrade risk and what your users are allowed to ask for. The question to answer is not "which index is best" but "what are we willing to operate for each class of customer".

The official guidelines give a sensible ladder. For a few thousand searches, or when exact results are required, Flat is correct, and you should resist the pull to compress something that fits in memory. With plenty of RAM, HNSW (HNSWM with M between 4 and 64), Intel SVS Vamana, or IVF1024,PQNx4fs,RFlat give high recall at low latency. When RAM is somewhat constrained, IVFK,Flat. When RAM is important, OPQ followed by fast-scan PQ such as OPQM_D,...,PQMx4fsr. When RAM is very important, OPQM_D,...,PQM, and for maximum compression RaBitQ. For the IVF cell count, below a million vectors the guidance is K between 4 and 16 times the square root of N, trained on 30 to 256 times K vectors.

Each rung has an operational cost that the tutorial ladder hides. HNSW needs no training and has excellent recall per millisecond, but it cannot delete, supports only sequential ids unless wrapped in IDMap, and uses the most memory of the graph options. IVF-based indexes are the workhorses for deletion and compression, but they need training on a representative sample, and training is not repeatable on an existing index: states move from untrained to trained to populated, and reset() returns to trained and empty. To retrain, you build a new index. This means that any IVF platform needs a rebuild pipeline from day one, not as a future improvement.

Compressed indexes bring a second axis, which is whether to refine. A compressed index returns candidates by approximate distance, and a refinement stage (RFlat, or Refine(...) with a k_factor) re-ranks those candidates using exact or finer distances. Refinement recovers much of the recall lost to compression at the price of keeping the full vectors somewhere, so it moves memory rather than removing it. A common platform pattern is to keep compressed codes in RAM and the full-precision vectors in a memory-mapped file or an external store.

Newer families change the trade-offs and deserve attention without being adopted blindly. RaBitQ gives about one bit per dimension plus a small overhead, d/8 + 8 bytes per vector, with multi-bit variants for higher accuracy, and its default query quantization bits changed from 0 to 4 in 1.14.0. Panorama indexes add early-pruning search. Intel SVS integration is behind a build flag, FAISS_ENABLE_SVS=ON, which means an official wheel may not include what a blog post demonstrates. Metal support for Apple Silicon is source-build only and young. My rule for a platform: a new family enters the catalogue only after it passes your own recall and latency benchmark on your own data, and after you have checked that your upgrade and rollback story handles its file format.

# A platform-owned catalogue: name -> factory string, metric and operational facts.
CATALOGUE = {
    "exact-small": {
        "factory": "IDMap,Flat",
        "bytes_per_vector": lambda d: 4 * d + 8,
        "deletes": "remove_ids supported",
        "max_vectors": 2_000_000,
    },
    "hnsw-fast": {
        "factory": "IDMap,HNSW32",
        "bytes_per_vector": lambda d: 4 * d + 256 + 8,
        "deletes": "NOT supported: rebuild or filter",
        "max_vectors": 20_000_000,
    },
    "ivfpq-refine": {
        "factory": "IVF65536_HNSW32,PQ64,RFlat",
        "bytes_per_vector": lambda d: 64 + 4 * d + 8,   # codes + refinement vectors + id (illustrative)
        "deletes": "remove_ids scans unless a Hashtable DirectMap is set",
        "max_vectors": 500_000_000,
    },
}

The index lifecycle: updates, deletes and rebuilds

Product teams will ask for "upsert" and "delete" within a month of launch, and FAISS does not have either in the database sense. Your platform has to say clearly what it offers instead.

An update is a remove_ids followed by an add_with_ids. There is no in-place update API. remove_ids works on flat indexes, IVF indexes and IDMap, and does not work on HNSW. For IVF, removal scans the inverted lists unless an id-to-location map is present, which you enable with set_direct_map_type(faiss.DirectMap.Hashtable). That map also enables reconstruct. The array variant of the direct map cannot be combined with custom ids, and fails with cannot have array direct map and add with ids, so the hashtable is the right default for a platform.

For an index that cannot delete, you have three honest options. First, filter at query time with an IDSelectorNot or a tombstone bitmap, and rebuild periodically; the index keeps growing and getting slower as tombstones accumulate, so the rebuild cadence is a function of the delete rate. Second, oversample k and post-filter in application code, which is simple and costs latency. Third, switch to an IVF family for that workload. I recommend writing the choice down per catalogue entry rather than letting each team discover it.

Concurrent writes need an explicit design. The simplest safe pattern for a platform is immutable indexes with atomic swap. Writers never touch the serving index. They build a new index (or a delta index) offline, publish it, and serving replicas load it alongside the current one and switch a pointer when the load completes. This pattern avoids the add-versus-search race entirely, makes rollback a pointer change, and fits object storage and a registry naturally. Its cost is freshness: new documents are not searchable until the next publish.

When freshness matters, a two-tier design works. A large immutable main index serves the bulk of the corpus, and a small mutable delta index, protected by a lock and rebuilt often, holds recent additions. Queries go to both and the results are merged. The delta index is small enough that a flat index with exact search is fast, and it is compacted into the next main build. Be careful that the two tiers use the same metric and the same normalization, or the merged ranking is meaningless.

Merging built shards is also supported: merge_from works for IVF indexes, and for indexes wrapped in a pre-transform there is a helper in ivflib. Training once and adding in parallel on many machines is the standard way to build a very large index quickly.

The security model

The first sentence of any FAISS threat model is that FAISS has no security features. No authentication, no network surface, no encryption, no access control. All of that is the job of the service you wrap around it. The second sentence is the one that matters more, and the project's own wiki states it: no attempt is made to check that loaded data is correct, so a faulty or malicious index file could trigger out-of-memory errors or, when crafted expertly, code execution.

Take that literally. An index file is an input with code-execution-grade risk. The deserializer reads a file and allocates and populates C++ objects from it. Releases from 1.14.0 through 1.15.1 contain a long list of hardening fixes (integer overflows, recursion depth, bounded allocations, null inner indexes, invalid metric types, and in 1.15.1 a path-traversal fix in the directory archiver). That is good news and also a clear signal of how large the surface was. The right conclusion is not "it is fixed now". It is "upgrade, and still never load a file you did not produce".

The trust model for a platform therefore has three parts.

Provenance. Every index the platform loads comes from the platform's own build pipeline. The build job records the FAISS version, the training data snapshot identifier, the factory string and the embedding model version in a manifest, computes a SHA-256 of the index file, and signs or otherwise protects the manifest. Serving replicas verify the digest before calling read_index. A model registry such as MLflow (see MLflow) or plain object storage with immutable versioning can hold these records.

Storage and identity. The bucket that holds index files is write-restricted to the build identity and read-restricted to the serving identity. Use workload identity from the platform rather than long-lived keys, and never bake object-store credentials into an image alongside an index. FAISS itself handles no credentials, so every secret in this system belongs to the surrounding services: the object store, the embedding API, the metadata database. Hold them in your secrets manager (see Vault).

Data classification. Embeddings are derived from source data and can leak information about it. An index file must carry the classification of the most sensitive document it contains, with the same residency rules. For employers in the Gulf and Egypt this is a practical point: if source documents must stay in-country, then the index files and the build jobs that process them must run in an in-country region too, and a "harmless" index copied to a cheaper region is a data transfer.

Two integration warnings follow from the same principle. LangChain's FAISS vector store pickles a docstore next to the index, and it requires allow_dangerous_deserialization=True to load; the flag name is the warning. Treat it with the same distrust as any pickle. And the RPC client and server shipped in the repository as a demo, demo_client_server_ivf.py, is explicitly for demonstration only and is not meant for an untrusted network or production. Do not let it become the reference architecture for a team that needs "a quick service".

Supply chain and installation policy

For years the advice about installing FAISS was folklore: there is no official pip package, use conda, the PyPI GPU wheel is abandoned. Much of it is now wrong, and a platform owner has to replace folklore with a written policy.

The current facts. Official wheels exist: faiss-cpu, faiss-gpu and faiss-gpu-cuvs on PyPI are published by the project and are at 1.15.1, with project URLs pointing at the facebookresearch/faiss repository. The pip build was added in 1.14.2, GPU wheel packaging in 1.14.3, and the cuVS wheel in 1.15.0. Conda remains the route that INSTALL.md calls supported, using the pytorch and conda-forge channels, plus nvidia for GPU and rapidsai for cuVS. Third-party wheels, such as faiss-gpu-cu12 from a community repository, are not official, whatever their version numbers suggest.

The compatibility edges matter for image design. The CPU wheels need Python 3.10 or newer, the cuVS wheel needs 3.11 or newer, and all require numpy>=1.25. The GPU wheel is Linux x86-64 only and depends on the CUDA 12 runtime packages from PyPI. The cuVS wheel needs CUDA 13.2 or newer. There is no official GPU package for Windows or macOS. Do not install faiss-cpu and faiss-gpu into the same environment, because both provide the faiss module, and the failure you get, AttributeError: module 'faiss' has no attribute 'StandardGpuResources', looks like a missing GPU rather than a packaging conflict. When the GPU library fails to load, the default loader silently falls back to CPU; running from faiss import _swigfaiss_gpu surfaces the real error, such as a missing libcudart.so.

A sensible policy has five lines. Install only the official packages. Pin the exact version in a lock file (faiss-cpu==1.15.1). Pick one channel family per image and never mix. Build container images per accelerator class rather than one universal image. And record the loaded SIMD level in startup logs. The last point exists because the pip wheels use dynamic dispatch, introduced in 1.14.0, which compiles several SIMD levels into one library and chooses at runtime. faiss.SIMDConfig.get_level_name() tells you what was picked, and FAISS_DISABLE_CPU_FEATURES can remove features from detection. A fleet with mixed CPU generations will otherwise produce latency differences nobody can explain, because the same image takes different code paths on different nodes.

Building from source is justified in three cases: you need a flag the wheels do not enable (Intel SVS through FAISS_ENABLE_SVS=ON, or a specific FAISS_OPT_LEVEL), you need a platform without a wheel, or you must patch. It requires a C++20 compiler, OpenMP, BLAS (MKL is recommended on x86-64) and CMake, and it makes you responsible for the whole build, so source builds belong in a build pipeline that produces a signed internal wheel, never on a developer laptop that then feeds production.

# Reproducible CPU install in an image build: exact pin, no resolver surprises.
python -m pip install --no-cache-dir "faiss-cpu==1.15.1" "numpy>=1.25"
python - <<'PY'
import faiss
print(faiss.__version__)
print(faiss.SIMDConfig.get_level_name())
PY

Multi-tenancy

FAISS has no tenants. Whatever isolation your customers need, you construct. There are three patterns, in descending order of isolation and ascending order of efficiency, and a platform often offers more than one.

One index per tenant gives the best isolation and the best search speed, because a tenant's query touches only that tenant's data. The official guidance for discrete attributes with few values is exactly this: build one index per value. The costs are fixed overhead per index (loading, memory floors, rollout units) and a long tail of tiny indexes. It works well for tens to low thousands of tenants of reasonable size. For a very long tail, keep small tenants as flat indexes, which need no training, and promote a tenant to a trained IVF only when it grows past a threshold. A shared trained quantizer lets the per-tenant IVF indexes start from common centroids, so you train once and add per tenant.

A shared index with filtering encodes the tenant in the id (for example a range per tenant) and filters with IDSelectorRange, or a bitmap through IDSelectorBitmap, passed per call in the search parameters. This packs many small tenants into one structure, at the price of speed: filtering evaluates every scanned id. Note that filtering works on int64 ids only; there is no metadata filtering, so the tenant must be recoverable from the id.

Oversample and post-filter retrieves more than k and drops what the caller may not see. It is the simplest and the leakiest: the filter runs after retrieval, so any bug in it returns another tenant's data. It also degrades when a tenant is a small share of the corpus, because most of the top results belong to somebody else.

For regulated workloads, prefer per-tenant indexes, or per-tenant shards, because "tenant isolation by an id range filter in a shared structure" is difficult to demonstrate to an auditor. Whatever you choose, the tenant identity must be derived from the authenticated request in your service and never from a parameter the caller supplies.

import faiss
import numpy as np

class TenantIndexes:
    """One in-memory index per tenant, loaded lazily, never shared across tenants."""

    def __init__(self, loader):
        self._loader = loader          # returns a verified faiss index for a tenant id
        self._cache = {}

    def search(self, tenant_id: str, xq: np.ndarray, k: int, nprobe: int):
        if not 1 <= k <= 100 or not 1 <= nprobe <= 256:
            raise ValueError("k or nprobe out of the platform's allowed range")
        index = self._cache.get(tenant_id) or self._loader(tenant_id)
        self._cache[tenant_id] = index
        params = faiss.SearchParametersIVF(nprobe=nprobe)
        return index.search(np.ascontiguousarray(xq, dtype="float32"), k, params=params)

Note the use of SearchParametersIVF per call. Setting index.nprobe = 123 mutates shared state, and on wrapped objects such as a sharded or GPU-cloned index, assigning an attribute from Python can silently do nothing because it lands on a proxy object. Per-call search parameters are thread-safe and explicit, which is why a platform should expose them and forbid mutation.

GPUs and the cuVS change

GPUs are the most seductive and most misunderstood part of FAISS operations. They are five to ten times faster than CPU for the same index and scale nearly linearly across GPUs through replicas, but they impose limits that a platform must encode as rules.

The rules. k and nprobe must be at most 2048 or the process aborts on an assertion. GPU indexes are not thread-safe, even for reads, so you need one StandardGpuResources object per CPU thread. A GPU index cannot be written: convert it back with faiss.index_gpu_to_cpu before write_index. Classic GPU IVF-PQ supports only a fixed set of code sizes, and an unsupported ratio of d to M fails an assertion, which OPQ can fix by changing the dimension. Memory pressure comes from precomputed tables, large add batches and scratch memory, and the documented mitigations are batching adds (about 8192 at a time), useFloat16, keeping indices on the CPU with INDICES_CPU, disabling precomputed tables and setting setTempMemory. A non-writable or non-persistent ~/.nv makes every start slow because of the CUDA JIT cache, which is a surprisingly common cause of "cold starts take minutes" in containers.

The cuVS story changed twice recently and your users' tutorials predate both. RAFT headers were removed in 1.14.0, and GPU acceleration from RAPIDS now goes through cuVS. In 1.15.1, cuVS dispatch became opt-in rather than implied by the build: in a cuVS build you must set use_cuvs=True on the config or cloner options explicitly. The wiki usage page still describes it as automatically enabled, so trust the changelog. Also in 1.15.1, trainMultiGpu was removed and a multi-GPU CAGRA build now goes through train(). CAGRA is a GPU-only graph index that can be converted to a CPU HNSW (IndexHNSWCagra), which makes a "build on GPU, serve on CPU" pattern possible. cuVS indexes do not support multi-GPU in the general case, reserveVecs, precomputed tables or searchPreassigned.

The economic pattern that works for most platforms is to use GPUs where they pay most and CPUs where they do not. Training IVF centroids is a large, bursty, parallel computation, and doing it on a GPU turns hours into minutes. Serving a compressed index from CPUs is usually cheaper at moderate query rates. Serve from GPUs only when latency or throughput requirements demand it and utilization will stay high, since an idle GPU is an expensive way to hold an index in memory.

import faiss

GPU_MAX_K = 2048

def to_gpu(cpu_index, device=0, use_cuvs=False):
    if not hasattr(faiss, "StandardGpuResources"):
        raise RuntimeError("GPU build of faiss not loaded; check `from faiss import _swigfaiss_gpu`")
    res = faiss.StandardGpuResources()           # one per serving thread
    co = faiss.GpuClonerOptions()
    co.useFloat16 = True
    co.use_cuvs = use_cuvs                       # explicit since 1.15.1; requires a cuVS build
    return res, faiss.index_cpu_to_gpu(res, device, cpu_index, co)

def checked_search(gpu_index, xq, k, nprobe):
    if k > GPU_MAX_K or nprobe > GPU_MAX_K:
        raise ValueError("k and nprobe must be <= 2048 on GPU")   # prevents a process abort
    return gpu_index.search(xq, k, params=faiss.SearchParametersIVF(nprobe=nprobe))

Upgrades and migrations

FAISS releases every one to three months, and the changelog is dense. Because the library is embedded in your service, an upgrade is a deploy of your own code, and it has two compatibility questions: can the new version read the old files, and can the old version read the new ones.

The first is guaranteed in practice: the serialization format is backward-compatible, and a backward-compatibility test has run in CI since 1.13.2. The second is not guaranteed. A new index type writes a fourcc code older versions do not recognize, and you get Index type ... not recognized. This asymmetry dictates the order of operations, and it is the single most important upgrade rule for a platform: upgrade the readers first, and write with the new version last. During a rolling deploy, if the build pipeline already writes new-format files while half the serving fleet is on the old version, the old half cannot load the new index and a rollback also fails.

The procedure that follows from this has five steps. Pin the new version in a branch. Load every production index with it in staging and confirm the load. Run a fixed query set with ground truth against both versions and compare recall, latency and memory; the comparison must use the same index file, so you isolate the library change from an index change. Roll the serving replicas. Only then let the build pipeline write with the new version, and keep the previous version's file for a defined rollback window.

A migration is different from an upgrade: it changes the index itself, for example from IVF4096,Flat to a PQ-compressed index, from one embedding model to another, or from flat to HNSW. Changing the embedding model always requires a new index, because vectors from two models are not comparable. Run the new index in shadow: serve both, compare results on live queries, measure overlap and recall against ground truth, and cut over only when the numbers meet the SLO. Keep the old index until the new one has survived a full traffic cycle.

Observability and recall as a service-level indicator

FAISS exposes no metrics endpoint. Everything you know about the service, you instrument. The standard operational metrics apply: request rate, latency percentiles at p50, p95 and p99, batch size, errors, memory (faiss.get_mem_usage_kb() and process RSS), the index version and size, and ntotal. Export them through your normal stack (see Prometheus). Add queue depth for your batcher, because a growing queue is the earliest warning of saturation.

The metric specific to this domain is recall, and it deserves the same treatment as availability. Define it as an SLI, for example recall@10 against exact search, on a fixed canary set, and set an SLO on it. Measure it in two ways. Offline, at build time, against the flat ground truth, as a publish gate: an index that misses the SLO is not published. Online, by running a small set of canary queries on a schedule against the serving index and comparing with stored exact results, and alerting on a drop. The online measure catches the failures the offline one cannot: a deploy that changed a default, a node with a different SIMD path, a mis-set nprobe in a config change.

FAISS also has internal counters. faiss.cvar.indexIVF_stats and faiss.cvar.hnsw_stats expose fields such as ndis, the number of distance computations, and are enabled with faiss.set_search_stats_enabled(True). The number of distance computations per query is the best leading indicator of cost: if it rises, nprobe or efSearch grew or the data changed, and latency will follow. MatrixStats gives data-quality comments (NaNs, duplicates, constant dimensions, normalization problems) and belongs in the build pipeline, since one batch of NaN vectors from a broken embedder can poison k-means training.

TimeoutGuard and the interrupt callback allow bounding long operations. Use them for request deadlines on expensive searches and for build jobs, with the caveat that calling FAISS from native threads that do not hold the Python GIL can deadlock the interrupt callback.

import faiss
import numpy as np

def recall_at_k(index, xq, exact_ids, k=10, **params):
    """Fraction of the exact top-k that the serving index returns."""
    _, found = index.search(xq, k, params=faiss.SearchParametersIVF(**params))
    hits = sum(len(set(f) & set(e)) for f, e in zip(found, exact_ids[:, :k]))
    return hits / exact_ids[:, :k].size

# Run on a schedule against the serving index; export the value as a gauge and alert on a drop.

Cost

Three things cost money in a FAISS platform: RAM, accelerator hours and engineer time. The first two are visible, the third is the one that kills projects.

RAM cost is set by the index family and compression. A reasonable framing for a leadership discussion is cost per million vectors at a stated recall: flat is simplest and most expensive in memory, HNSW is fast and RAM-heavy, IVF with PQ and refinement is RAM-efficient and adds a rebuild pipeline. Compression trades recall for memory, refinement recovers recall by spending some memory back, and memory mapping trades RAM for SSD operations. Each is a lever, and the right setting is the cheapest operating point that still meets the recall SLO. ParameterSpace.explore with a recall criterion maps the speed and recall frontier for an index, so the choice is empirical rather than argued.

Engineer time is the hidden cost of "free". The library costs nothing, but persistence, replication, upsert, filtering, tenancy, access control, a rollout system and on-call all get built and maintained by your team. A fair comparison with a managed vector database or a database extension counts those weeks. For many teams the break-even is in the low tens of millions of vectors with simple access patterns; beyond that, or with rich filtering and frequent updates, the build-versus-buy arithmetic tends to flip. I give this as a heuristic, not a measurement: do the arithmetic with your own numbers.

Governance and ownership

A platform needs rules that survive turnover, and for FAISS the rules are about change and responsibility.

Ownership. Name an owner for the catalogue, the build pipeline, the serving template and the evaluation set. A vector index with no owner becomes a liability within a quarter: nobody knows which embedding model produced it, which source data it contains, or whether it may be deleted.

Change control. Treat three kinds of change as reviewed changes: a new index family in the catalogue, a version bump of FAISS, and any change to search-parameter defaults. Each has an observed history of changing recall or file compatibility, and each should pass the frozen evaluation set before it ships.

Data lifecycle. Because embeddings can reveal source content, deletion requests apply to them too. If a document must be erased, the platform needs a way to remove its vector (a tombstone and periodic rebuild for HNSW, remove_ids elsewhere) with a stated maximum time to effect. Retention of index artifacts should follow the retention of the source data.

Lineage. Every index records the data snapshot, embedding model and version, FAISS version, factory string, training sample and build job. When a search quality complaint arrives, that record is how you tell whether the cause is the data, the model, the index or the service.

Incident playbooks

Most FAISS incidents fall into a small number of patterns. Write them down before you need them.

Recall dropped, no errors. Check what changed first: the index version, the FAISS version, the parameters in the config, the embedding model. Run the canary set against both the serving index and a flat index built from the same vectors. If the flat index also looks wrong, the embeddings changed or are broken, so run MatrixStats on a sample. If only the serving index is wrong, compare nprobe or efSearch, check the SIMD level on affected nodes, and check whether a rebuild used too little training data (the stderr warning about points per centroid appears at build time). Roll back to the previous index pointer first, then diagnose.

Pods crash-looping on startup. Read the exception. Index type ... not recognized means a version mismatch between the file and the library: either a new-format file reached an old reader, or the file is corrupt or of the wrong kind (a binary index needs read_index_binary). could not open ... for reading is a path or permission problem. An out-of-memory kill during load means the index no longer fits; compare file size with the pod limit, and consider IO_FLAG_MMAP. Verify the digest before anything else.

Pods restart under load, no exception. Look for a Faiss assertion line on stderr. On GPU, k or nprobe above 2048 aborts. A concurrent add and search on one index can corrupt or crash. Check whether a deploy added a write path or a second thread to a read-only service.

Latency spike with normal traffic. Check batch size and queue depth (the batcher may have stopped batching), OpenMP thread count against cores (a change in container CPU limits changes what OMP_NUM_THREADS should be), BLAS oversubscription, and a noisy neighbour. On GPU, check that ~/.nv is writable and persistent, and that the loader did not fall back to CPU. Compare distance computations per query with last week.

Memory growing. Distinguish index growth (ntotal, file size) from leaks in the surrounding service. Check whether a delta index was never compacted, or whether tombstones keep an HNSW index growing.

Suspicious index file. If an index of unknown origin was loaded, treat the host as potentially compromised: isolate it, rotate the credentials it held, and trace how the file reached the loader. This is the reason provenance verification exists.

Where FAISS stops

The most senior skill is knowing when not to use the tool. FAISS is excellent at one thing, fast similarity search over dense vectors in memory, and it is deliberately silent about everything else. Choose something else when the requirements move outside that.

Choose a vector database (Qdrant, Milvus, Weaviate, Pinecone) when you need rich metadata filtering combined with similarity, frequent upserts and deletes with durability, built-in replication and failover, or multi-tenancy with access control that you do not want to build. Choose a database extension such as pgvector when your vectors live next to relational data, the scale is moderate, and transactional consistency with the rest of the data matters more than raw speed. Several of these systems embed FAISS or similar libraries as an engine, which tells you something: the algorithms are shared, and what you are buying is everything around them.

Choose FAISS directly when you need the algorithmic control (index families, quantization, GPU, custom pipelines), when the corpus is large but the access pattern is simple and mostly read-only, when you are building a research or batch workload such as deduplication, clustering or offline nearest-neighbour joins, or when you are the team building the engine that others will use. FAISS also remains the right tool for k-means at scale and for brute-force search with faiss.knn.

Running it as a platform for other teams

Everything above converges on how you offer FAISS to others. A platform is not a library with a wiki page. It is a set of defaults, guardrails and paved roads that make the safe path the easy one.

A paved road. Provide a build template (a pipeline that embeds, trains, adds, evaluates, signs and publishes) and a serving template (a container with the verified loader, batcher, input validation, metrics and health checks). A team that follows the road gets security, observability and a rollback path without having read this guide. Containerize both (see Docker) and run them on Kubernetes with memory requests derived from the manifest, readiness gated on a successful verified load and a canary query, and rolling updates that never mix writers and readers of different versions.

Guardrails in code. Put limits in the service, not the documentation: maximum k, maximum nprobe, maximum batch size, allowed dimension and dtype, a request timeout through TimeoutGuard. Expose search through a narrow API of your own, and never expose raw FAISS objects, index files or factory strings to callers.

Contracts. Publish, for each catalogue entry, what you promise: a recall range on reference data, latency percentiles at a stated batch size, freshness (how long until a new document is searchable), the deletion time-to-effect, and the supported scale. Also publish what you do not promise: transactions, string ids, metadata filtering, and cross-index consistency.

The review checklist

Use these for design documents and pull requests that touch the retrieval layer. A "no" is not a rejection, but it needs an answer.

Index design

• Is the index in the supported catalogue, and is the memory formula written down?
• Is the recall target measured against a flat ground truth on this team's data?
• Is the training sample representative, with enough points per centroid (at least 39, ideally far more)?
• What is the delete and update story, and how often is the index rebuilt?
• Is the metric explicit, with vectors normalized if cosine is intended?

Serving

• Are k, nprobe, dimension and dtype validated before the call?
• Are search parameters passed per call rather than mutated on the index?
• Is there a batcher, and is OMP_NUM_THREADS set deliberately?
• Can any code path mutate an index that is being searched?
• On GPU: one resources object per thread, k and nprobe at most 2048, use_cuvs explicit, and a startup assertion that the GPU module loaded?

Security

• Does every read_index and deserialize_index read a file from a verified manifest?
• Who can write to the index bucket, and who can read it?
• Is the data classification and residency of the index the same as its source data?
• Are the packages official, pinned and from one channel family?
• Is any pickle-based loader, such as a framework docstore, loading data we did not produce?

Operations

• Is recall a measured SLI with a canary set and an alert?
• Is there a rollback to the previous index, and has it been practiced?
• Is the upgrade order readers first, writers last?
• Are index versions, FAISS version and SIMD level logged?
• Does each index have an owner, a manifest and a retirement plan?

The complete picture

Pull the three levels together. At the beginner level, FAISS is an object that stores float32 vectors and returns neighbours. At the mid level, it is a family of indexes that trade recall, speed and memory through a few parameters, trained and tuned against a measured ground truth. At the senior level, it is a component inside a system you own: an immutable artifact pipeline on the build side, a verified, batched, guarded serving process on the other, and a set of numbers (recall, latency, memory, freshness) that you promise and monitor.

The shape that recurs is this. Build offline with a representative sample and a ground truth. Publish immutable, signed, versioned indexes. Serve them in processes that validate input, batch queries and never mutate. Measure recall continuously. Upgrade readers before writers. Keep tenants apart by construction. And be honest about the point at which the requirements describe a database and not a library.

Where the series leaves you

You can now design an index for a stated budget, tune it against ground truth, serve it safely, upgrade it without breaking readers, isolate tenants, price it honestly and argue when to use something else. The next steps are practical rather than theoretical.

Sources

• FAISS documentation
• FAISS repository
• FAISS releases
• CHANGELOG.md
• INSTALL.md
• FAISS wiki
• Guidelines to choose an index
• Indexing 1G vectors
• Running on GPUs
• GPU Faiss with cuVS
• Threads and asynchronous calls
• Indexes that do not fit in RAM
• FAISS FAQ
• faiss-cpu on PyPI
• faiss-gpu on PyPI
• The Faiss library (Douze et al.)

Part three of three. Senior interviews about FAISS are rarely about syntax. They test whether you can own a retrieval layer: size it, defend a design, handle an incident at 3 a.m., and tell a director why the answer is or is not "just use a vector database". Each answer below is written the way I would give it aloud, followed by what separates a strong answer from a weak one. All facts are for FAISS 1.15.1.

The thirty-second answer

"FAISS is Meta's open-source library for similarity search and clustering of dense vectors. It is a library, not a database: no server, no authentication, no persistence, no metadata filtering. You embed it in your own service, which means that everything a vector database provides, such as durability, replication, upserts and tenancy, is something I build around it or do without. In exchange I get complete control over the index family, the compression and the hardware, including GPUs."

If you are asked for ten seconds more, add the thing that makes you sound senior: "and the index file is an untrusted-input-grade artifact, so I only load files my own pipeline produced and verified."

The facts to have ready

You will be asked quick factual questions as a filter before the design questions. Have these without hesitating.

• Metrics. METRIC_L2 returns squared Euclidean distance, not the root. Inner product is a similarity, larger is better. Cosine is L2 normalization followed by inner product.
• IVF. nlist cells, nprobe visited, and the default nprobe is 1. Training is required and cannot be repeated on a populated index; you build a new one.
• HNSW. No training, no remove_ids, sequential ids only unless wrapped in IDMap. Memory is about 4*d + 256 bytes per vector at M=32.
• Memory. Flat is 4 times d bytes per vector. IVF-PQ is M bytes plus 8 for the id. RaBitQ is d/8 + 8 bytes.
• GPU. k and nprobe at most 2048, or the process aborts. GPU indexes are not thread-safe. Convert to CPU before write_index.
• Packaging. Official pip wheels exist for faiss-cpu, faiss-gpu and faiss-gpu-cuvs. Do not mix CPU and GPU packages. Pin the version.
• Compatibility. New versions read old files. Old versions may not read new files.
• Security. Never load an index you did not produce.

System design: a semantic search platform for ten teams

The prompt. "Ten product teams want semantic search over their own documents, ranging from fifty thousand to two hundred million vectors. Design the platform around FAISS."

A strong answer. I would begin by asking the questions that determine the architecture, because the numbers decide whether FAISS is the right choice at all. What is the update rate and freshness requirement? Is there metadata filtering, and how selective? What is the recall target and the latency budget? Do the documents have residency or classification constraints? Who will be on call?

Assuming mostly read-heavy, nightly or hourly refresh, and simple access control, I would propose a catalogue with three index shapes. Teams under a couple of million vectors get an exact flat index wrapped in IDMap, because exact search is simple, deletable and needs no training. Mid-size teams get an HNSW or IVF index with a measured recall target. The two-hundred-million team gets IVF with a product-quantized code and exact refinement, an HNSW coarse quantizer, trained once on a representative sample and sharded across machines.

The structure is a build plane and a serving plane. The build plane embeds, trains, adds, evaluates against a flat ground truth, and publishes an immutable index with a manifest holding the FAISS version, factory string, data snapshot, embedding model and a SHA-256 digest. The serving plane is a template that verifies the digest, loads the index, batches incoming requests into one search call, validates k and nprobe before they reach the library, and exposes metrics. Updates work by publishing a new index and swapping a pointer; for freshness I would add a small flat delta index and merge results.

Tenancy is one index per tenant for anyone with a compliance requirement, with a small-tenant tier that is flat. I would not use a shared index with id-range filtering where audit isolation matters. Recall is a measured SLI on a canary set, with an alert.

What makes it strong. It starts with questions. It names the failure modes it is defending against (recall decay, concurrent mutation, untrusted files). It keeps the library out of the request path of untrusted input. It is explicit about what the platform does not promise.

System design: memory budget for a hundred million vectors

The prompt. "One hundred million 768-dimensional embeddings. Budget: three 64 GB machines. Is it possible, and how?"

A strong answer. Do the arithmetic aloud. Flat float32 is 4 times 768 bytes, about 3 KB per vector, so 100 million vectors is about 307 GB. That exceeds even the three machines combined (192 GB). So we compress.

IVF with product quantization at 64 bytes per vector gives about 6.4 GB of codes plus 0.8 GB of 8-byte ids, around 7.2 GB, which fits on one machine with room to spare. The cost is recall, so I would measure it before committing. With the headroom I might choose a larger code (for example 128 bytes) or keep refinement vectors in a memory-mapped file on SSD to re-rank the top candidates exactly. For the coarse quantizer at this scale the official guidance is IVF262144_HNSW32 for 10M to 100M vectors, trained on a sample (the rule of thumb is 30 to 256 training points per cell).

Then the three machines are for throughput and availability, not capacity: three replicas, or one replica pair plus a rebuild spare. If query volume is high, a GPU would speed up scanning but is not necessary for capacity. Build time is the real constraint: train once on a sample (a GPU makes this much faster), add in parallel on several machines to copies of the trained empty index, and merge with merge_from. Rebuilds are a recurring cost, so I would write down the rebuild time as part of the design.

I would say clearly: this is a back-of-the-envelope plan; the recall of PQ at this ratio depends on the data, so the first deliverable is a recall-versus-code-size curve on a real sample, found with ParameterSpace.explore.

What makes it strong. Arithmetic before opinions. Distinguishing capacity from throughput. Noticing that a rebuild exists. Treating recall as something to measure.

System design: filtered search with multi-tenancy

The prompt. "Our product needs 'search within this customer's documents, plus documents they have shared'. We already run FAISS. Can we do it?"

A strong answer. FAISS filters on int64 ids only; it has no metadata filtering. So the question is whether the filter can be expressed as an id set. For "this customer's documents" with a modest number of customers, one index per customer is the cleanest: the filter is the index. For shared documents, I can search the customer's index and a shared index and merge top-k, which keeps the isolation structural.

If one customer-per-index does not scale to the number of customers, I can encode the owner in the id range and pass an IDSelectorRange or IDSelectorBitmap in the per-call search parameters. That works and is thread-safe, but it makes the scan check every id, and I would benchmark it. A third option, oversampling k and post-filtering, is the weakest: a bug in the post-filter leaks data, and a small customer in a large corpus gets mostly other people's results.

At some point I would say the requirement has moved into database territory: arbitrary attribute filters combined with similarity, frequent permission changes, and per-user sharing graphs. That is the point where I would evaluate Qdrant, Milvus or pgvector, and I would bring numbers on the engineering cost of building filtering and access control ourselves.

What makes it strong. It states the limitation up front, offers more than one design with trade-offs, and says where the line is.

System design: a rollout that cannot break readers

The prompt. "We are upgrading FAISS across a fleet that serves production traffic, with a nightly index build. How do you do it safely?"

A strong answer. The key fact is the compatibility asymmetry. New versions read old files, which is tested in the project's CI since 1.13.2. Old versions are not guaranteed to read new files, and a new index type produces a code that older readers reject with Index type ... not recognized. So the order is readers first, writers last.

Step one, pin the new version and build the image. Step two, in staging, load every production index with the new version and run a frozen evaluation set against both versions, using the same file so the library change is isolated, and compare recall, latency and memory. Step three, roll the serving replicas gradually, watching the canary recall metric. Step four, only after the whole fleet is on the new version, switch the build pipeline to write with it. Keep the previous index versions for a rollback window.

I would read the changelog for behaviour changes in the target range. In recent releases these include the cuVS dispatch becoming opt-in in 1.15.1, the RaBitQ default query bits changing in 1.14.0, HNSW construction changes in 1.15.1 that mean rebuilt graphs are not bit-identical, and API renames such as Ex to _ex. A test that exercises our real call paths catches the renames; the evaluation set catches the rest.

What makes it strong. The asymmetry stated first, the order of operations, and a comparison that isolates the variable.

Incident response: recall dropped, no alerts fired

The prompt. "Product says search results got worse last week. Latency and error rates are fine. What do you do?"

A strong answer. First, I accept that nothing in FAISS reports recall, and that this is the failure mode with no built-in signal, so my first step is to measure it. I run the canary set against the serving index and against a flat index built from the same vectors. If the flat index is also bad, the problem is upstream: the embedding model, a changed preprocessing step, or broken vectors, and I run MatrixStats for NaNs, duplicates and normalization. If only the serving index is bad, the index or its parameters changed.

I would compare the manifest of the index served now with the previous one, looking at the FAISS version, the factory string, the training sample size and the data snapshot. I would check nprobe or efSearch in the running config, since the default nprobe is 1 and a refactor can drop the setting silently. I would check the SIMD level on affected nodes, because mixed CPUs take different code paths. If the build used too few training points, the build log would have shown the stderr warning about points per centroid.

Mitigation comes before diagnosis: repoint serving at the previous known-good index. Then I would add the missing guardrail, which is a recall canary with an alert, and a publish gate that fails an index whose offline recall misses the SLO.

What makes it strong. Measure first, bisect by comparing with ground truth, mitigate by rollback, fix the detection gap.

Incident response: pods restart under load

The prompt. "Our GPU search pods restart a few times an hour, with no stack trace in the logs."

A strong answer. No stack trace points to a process abort rather than an exception. FAISS hard assertions print Faiss assertion '<cond>' failed and abort, unlike ordinary errors, which surface as catchable RuntimeError. I would search stderr for that line. On GPU, the usual cause is k or nprobe above 2048, and the second is an unsupported IVF-PQ dimension-to-subquantizer ratio. Either would come from a caller-controlled parameter, so the fix is validation in our service, with a client error, not a catch block.

The other candidates are concurrency, since GPU indexes are not thread-safe even for reads and a concurrent add and search on one index can corrupt memory, and GPU memory exhaustion from large add batches or scratch memory. I would check whether a recent change added a write path or a second thread, and whether each thread has its own StandardGpuResources. Then I would reproduce with the offending parameter in staging.

What makes it strong. Recognizing the abort-versus-exception distinction, and fixing at the API boundary.

Incident response: a suspicious index file

The prompt. "A teammate loaded an index file from a partner's bucket into a staging service to try it. Is that a problem?"

A strong answer. Yes, treat it as a potential security incident. The project's own documentation says no attempt is made to validate loaded data and that a malicious file could trigger out-of-memory errors or, if expertly crafted, code execution. Recent releases hardened the deserializer extensively, which is reassuring and also evidence of how large that surface was, but the guidance is unchanged: load only what your own pipeline produced.

I would isolate the staging host, check what credentials it holds and rotate them, find out how the file was fetched and whether anything from it reached other systems, and review the process's network egress. For the longer term, I would make the loader accept only digests from a signed manifest, restrict who can write to the bucket, and make that path impossible by policy rather than by memory. If the partner genuinely needs to give us vectors, the safe exchange is raw vectors in a plain format, from which we build our own index.

What makes it strong. Treating a file as an input, a concrete containment plan, and a structural fix.

Arguing the trade-off: FAISS or a vector database

The prompt. "Leadership asks whether we should replace our FAISS service with a managed vector database. What do you say?"

A strong answer. I would say the answer depends on three things and ask to decide on them together. First, the requirements: if we need rich filtering, frequent updates with durability, and per-user access, those are database features we are currently building and maintaining ourselves. Second, scale and cost: at hundreds of millions of vectors with a simple, mostly read-only pattern and in-house GPUs, FAISS is very efficient and a managed service may cost much more. Third, risk and ownership: what does our team spend on-call time on, and what would we stop doing.

I would propose a time-boxed comparison on our own data: recall at equal latency, cost per million queries, and the engineering weeks we have spent on the custom layer in the last year. I would flag that several databases embed FAISS-like algorithms, so quality is similar and the difference is operational. And I would offer a staged option: keep FAISS for the large, simple workloads, and move the filter-heavy, update-heavy use cases to a database.

What makes it strong. Numbers, a time-boxed experiment, a staged option, and no ideology.

Questions on the library itself

Why can I not delete from an HNSW index? The graph structure does not support removing a node safely, and remove_ids is not implemented for it. Your options are tombstone filtering with periodic rebuilds, oversampling and post-filtering, or an IVF family.

Why does omp_set_num_threads not work in my service? The setting is thread-local, so it only affects the thread that called it. Set OMP_NUM_THREADS in the environment, or call it in the thread that runs the search.

Why did setting nprobe on my wrapped index do nothing? Assigning an attribute on a proxy or base-class object just adds a Python attribute. Use ParameterSpace().set_index_parameter, extract the inner IVF with faiss.extract_index_ivf, or pass per-call SearchParametersIVF, which is also thread-safe.

How do you pick an index? Start from the official guidelines: exact for small or exactness-critical, HNSW or fast-scan PQ with refinement when RAM is plentiful, IVF-Flat when it is somewhat constrained, OPQ plus PQ when it is scarce, RaBitQ for maximum compression. Then measure recall and latency on the data with ParameterSpace.explore and choose the cheapest point that meets the target.

Sixty-second self-test

Answer aloud, without notes.

1. Why is an assertion different from an exception, and where do you stop it?
2. What is the arithmetic for 100 million 768-dimensional vectors flat, and with 64-byte PQ codes?
3. Which index families cannot delete, and what do you do instead?
4. In what order do you upgrade readers and writers, and why?
5. What does your recall SLI measure, against what, and how often?
6. Name three ways to isolate tenants and the weakness of each.
7. What must be true of an index file before read_index touches it?
8. What does use_cuvs do in 1.15.1 that it did not before?
9. Name two requirements that mean you should be using a database.
10. How do you roll back a bad index in under a minute?

If you can answer all ten with numbers and a reason, you are ready to be asked to own the platform.

Part three of three. This file is the working notes of someone who has run FAISS for other teams: the traps that only show up at scale, the habits of a platform owner, and review checklists you can paste into a pull request template. Versions are FAISS 1.15.1.

Production traps that appear only at scale

Each trap below has the same structure: what you see, why it happens, and what to do. They are ordered roughly by how much damage they do.

Recall decays and nothing alerts

Symptom. Users say results are worse. Latency, errors and CPU are normal.

Cause. FAISS reports nothing about recall. IVF returns quickly whatever it finds in the nprobe cells it visits, and the default nprobe is 1. A corpus that outgrew the centroids, a new embedding model, or a config refactor that dropped a parameter all produce confident wrong answers.

Fix. Measure recall as a service-level indicator: offline at build time against a flat ground truth as a publish gate, and online with a frozen canary set on a schedule. Alert on a drop. Treat the canary set as versioned data.

The batch size of one

Symptom. A powerful machine delivers poor throughput and erratic tail latency, with high CPU use.

Cause. FAISS parallelizes search across the queries in a batch with OpenMP. A handler that calls search once per request from many threads creates competing thread teams. With OpenBLAS there is a further oversubscription between OpenMP and BLAS threads.

Fix. Put a batcher in front of the index that gathers requests for a few milliseconds and issues one call. Set OMP_NUM_THREADS to the physical core count. If brute force is unexpectedly slow with OpenBLAS, try OMP_WAIT_POLICY=PASSIVE.

A parameter that aborts the process

Symptom. Pods restart with no stack trace. The log ends with Faiss assertion '...' failed.

Cause. Hard assertions abort; they are not exceptions. On GPU, k or nprobe above 2048 triggers one. A caller controlled the number.

Fix. Validate before the call and return a client error. Maintain a list of caller-influenced parameters and their bounds in the service, and test each bound.

Mutation during search

Symptom. Rare crashes or corrupted results after a team adds an upsert endpoint.

Cause. add is not safe concurrently with search on the same index, and GPU indexes are not thread-safe even for reads.

Fix. Prefer immutable indexes and an atomic pointer swap. When you must mutate, serialize with a lock, or write to a small delta index and merge results at query time.

The silent CPU fallback

Symptom. A "GPU" service has CPU-level latency, or AttributeError: module 'faiss' has no attribute 'StandardGpuResources'.

Cause. A CPU package is installed, or the CUDA library failed to load and the loader fell back to CPU without complaint.

Fix. At startup assert faiss.get_num_gpus() > 0 and hasattr(faiss, "StandardGpuResources"), and fail the pod otherwise. Debug the real cause with from faiss import _swigfaiss_gpu.

Cold starts that take minutes

Symptom. Every GPU pod start is slow on the first call.

Cause. The CUDA JIT cache under ~/.nv is not writable or not persistent in the container, so kernels are recompiled each time. Building for the wrong CUDA architecture has the same effect.

Fix. Make the cache directory writable and persistent, or build for the right CMAKE_CUDA_ARCHITECTURES.

The index that cannot be rolled back

Symptom. After an upgrade you cannot roll back, because old pods refuse the new files: Index type ... not recognized.

Cause. Serialization is backward-compatible but not forward-compatible. The build pipeline upgraded before the readers.

Fix. Upgrade readers first, writers last, and keep the previous index versions during a rollback window.

The deleted vector that is still returned

Symptom. A document was deleted at the source but still appears in results.

Cause. HNSW has no remove_ids, so the delete never reached the index, or an IVF index without a direct map removed ids by scanning and the job timed out.

Fix. Define a time-to-effect for deletions. For HNSW use tombstone filtering and scheduled rebuilds, and for IVF enable set_direct_map_type(faiss.DirectMap.Hashtable). Test deletions end to end.

Mixed hardware, mixed behaviour

Symptom. The same image performs differently on different nodes.

Cause. The pip wheels use dynamic dispatch and pick a SIMD level at runtime, so different CPUs run different code paths.

Fix. Log faiss.SIMDConfig.get_level_name() at startup and expose it as a label. Benchmark per node class, and pin the node pool when latency is critical.

Traps in the surrounding system

Pickle-based loaders. A framework wrapper that stores a docstore next to the index uses a serialization format that can run code on load. The flag allow_dangerous_deserialization=True in LangChain's FAISS store is a warning and not a nuisance. Load only what your pipeline produced.

The demo server. The RPC client and server in the repository are for demonstration only. Do not promote them to production because they are "already written".

Float64 and integer types. The core APIs need float32. Numpy scalars such as np.int64 passed where a Python int is expected give a TypeError about faiss::idx_t. Convert at your service boundary so callers cannot trigger either.

Changing the embedding model. Vectors from two models are not comparable. A model change always means a new index, a shadow run and a cutover, never a gradual mix.

Unnormalized cosine. If you intended cosine and forgot faiss.normalize_L2, inner product search ranks long vectors first. Normalize in one place, in the platform, and document it.

Data residency. Index files inherit the classification and residency of their source documents. A copy to a cheaper region is a data transfer. For Gulf and Egyptian employers with in-country requirements, make the build and serving regions explicit in the manifest.

Platform-ownership habits

These are the practices that distinguish a platform from a pile of scripts. They are in order of leverage.

1. Keep a frozen evaluation set. A few hundred real queries with exact results from a flat index, versioned with the data. Run it on every index build, every library bump and every parameter change. Nothing else you do will catch as many problems.

2. Make the catalogue small. A short list of supported index configurations, each with a memory formula, a measured recall range, a rebuild procedure and a deletion story. New families, such as RaBitQ, Panorama or SVS, enter only after your own benchmark on your own data.

3. Publish immutable artifacts. One index file, one manifest, one digest. The manifest records the FAISS version, factory string, training sample, data snapshot and embedding model. Rollback is a pointer change.

4. Verify before you load. The serving process loads only a digest named in a signed manifest from a bucket only the build identity can write. A mismatch is a startup failure.

5. Put guardrails in code. Maximum k, maximum nprobe, maximum batch, allowed dimension and dtype, request deadlines through TimeoutGuard. Documentation does not stop an abort.

6. Pass parameters per call. SearchParametersIVF(nprobe=...) and SearchParametersHNSW(efSearch=...) are thread-safe and explicit. Mutating index.nprobe is shared state and, on wrapped indexes, may do nothing at all.

7. Log what explains behaviour. FAISS version, compile options, SIMD level, thread count, index manifest, ntotal, and memory. A strong set of startup facts resolves most "node B is slower" tickets in minutes.

8. Own the rebuild. Retraining is not supported on a populated index, so the rebuild pipeline is a first-class product, with a schedule, an owner and a duration you know.

9. Attribute cost. Memory and compute per index and per owning team, shown on a dashboard, with a list of indexes nobody has queried in thirty days.

10. Practice rollback. Run a game day where you publish a bad index to staging and time detection and recovery.

Tuning traps that waste a quarter

Tuning on the training data. If you measure recall using vectors that were also used to train the index, you measure how well k-means fits its own sample, not how well the index serves new queries. Hold out queries that were not in the training sample, and ideally draw them from real traffic rather than from the corpus. Queries are often distributed differently from documents, and an index trained on documents can be well tuned for the wrong region of the space.

Optimizing one number. Teams pick a recall target and tune until they reach it, then discover latency doubled, or memory grew. Always read the three together, recall, latency at a fixed batch size, and memory, and publish the trade-off curve rather than a single point. ParameterSpace.explore produces the frontier of operating points, which gives you the language to say that a further point of recall costs a stated amount of latency.

Training on too little data. Clustering warns on stderr when there are fewer than 39 points per centroid, and the quality gain flattens beyond about 1000 times the number of centroids. The warning scrolls past in a build log that nobody reads. Make the build fail on that warning, or compute the sample size from nlist in the pipeline so that it cannot happen.

Treating compression as free. Product quantization needs d to be a multiple of M, and the accuracy loss is data dependent. Measure it on your own embeddings, with refinement and without. A rotation such as OPQ may recover accuracy, at the price of a more complicated index and a transform at query time.

Over-trusting a benchmark from a blog. Many published numbers were produced on older versions, with different SIMD levels, on public datasets whose structure differs from yours. Use them to choose candidates, never to choose the winner. Rerun the shortlist on a sample of your own vectors, at your own batch size and on the hardware you will actually deploy to.

Habits for incident reviews

A postmortem on a retrieval incident should answer a small fixed set of questions, and writing them down in advance keeps reviews short and useful. What was the user-visible effect, and how long before we noticed? Which signal would have detected it earlier: a recall canary, a distance-computation counter, a startup assertion? Which guardrail was missing: input validation, a digest check, a publish gate? What changed in the last release, in the index, in the library, in the parameters or in the data? And can the platform now make the mistake impossible rather than merely documented?

The last question is the one that compounds. Every incident that ends in "we told people to be careful" will repeat. Every incident that ends in a check in the template, a bound in the service or a failing build does not. Keep a running list of the incidents and the guardrails they produced, and review it when you onboard a new team: it is the most honest documentation the platform has.

Practice cards

Short drills that take under an hour each. Do them in staging.

Operational recipes

These are the commands and snippets I keep in the runbook.

Check what is installed and which accelerator path is in use:

import faiss
print(faiss.__version__)
print(faiss.get_compile_options())
print(faiss.SIMDConfig.get_level_name())
print(faiss.get_num_gpus(), hasattr(faiss, "StandardGpuResources"))
print(faiss.omp_get_max_threads())

Reveal why a GPU build did not load:

python -c "from faiss import _swigfaiss_gpu"

See which SIMD module the loader picked:

import logging
logging.basicConfig(level=logging.INFO)
import faiss  # logs "Loading faiss with AVX2 support." and similar

Compute the digest and verify before load:

import hashlib
import faiss

def load_verified(path: str, expected_sha256: str):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    if h.hexdigest() != expected_sha256:
        raise RuntimeError("index digest mismatch; refusing to load")
    return faiss.read_index(path)

Explore the recall and speed frontier of an index:

crit = faiss.OneRecallAtRCriterion(nq, 1)
crit.set_groundtruth(None, gt_I)
ps = faiss.ParameterSpace()
ps.initialize(index)
op = ps.explore(index, xq, crit)
op.display()

Enable internal counters for a cost view:

faiss.set_search_stats_enabled(True)
faiss.cvar.indexIVF_stats.reset()
# run queries
print(faiss.cvar.indexIVF_stats.ndis)   # distance computations

Check data quality before training:

print(faiss.MatrixStats(x).comments)

Checklist for a pull request

Paste this into the pull request template for any change that touches retrieval.

• Does the change add or alter an index configuration, and is it in the catalogue?
• Was the frozen evaluation set run, and is recall attached to the description?
• Are k, nprobe, dimension and dtype validated before FAISS is called?
• Are search parameters passed per call, with no mutation of shared index fields?
• Can any new code path call add or remove_ids on a serving index?
• Does any new code call read_index, deserialize_index or a framework loader, and where do the bytes come from?
• If GPU: is there a startup check, one resources object per thread, and use_cuvs set explicitly?
• Is the package official, pinned, and from one channel family?
• Does the change alter memory per vector, and is the capacity forecast updated?
• Are metrics, logs and the manifest updated for the new behaviour?

Checklist for a design document

• What is the recall SLO, how is it measured, and against what ground truth?
• What are the latency percentiles, at what batch size and query rate?
• What is the memory formula, the total, and the headroom at projected growth?
• What is the freshness requirement and the mechanism: rebuild, delta index, swap?
• What is the deletion requirement and time-to-effect, and does the index support it?
• How are tenants isolated, and how would you prove it to an auditor?
• Where do index files come from, who can write them, and how are they verified?
• What is the residency and classification of the index and its build jobs?
• What is the upgrade order, the rollback plan and the rollback window?
• What does this design not promise, and who has accepted that?
• At what scale, access pattern or feature request do we re-evaluate a database?

Closing advice

The consistent theme is that FAISS gives you speed and control and takes no responsibility for the rest. The traps above are almost all one mistake in different forms: assuming the library does something that a database would do for you. Write down what you build around it, test each piece, and measure recall continuously. That is most of the job.

Link the practice back to the rest of your stack. Version your artifacts as you would model artifacts (see MLflow), run the serving template on Kubernetes, watch it with Prometheus, and keep storage credentials in Vault rather than next to the index.