A vector database stores lists of numbers that represent meaning, then finds the stored items closest to whatever you ask about. That is what lets a search box return a document about “cancelling a subscription” when the user typed “how do I stop being charged”. These systems became popular because AI assistants need to look things up in your own material, and keyword matching was not good enough. They are also frequently reached for when a plain database or a small library would have done the job.
Updated September 2026.
Embeddings, the thing being stored
An embedding is what a model produces when you hand it a piece of text, an image or a sound clip. OpenAI’s documentation describes an embedding as a vector, meaning a list of floating point numbers, where the distance between two vectors measures how closely related they are. Its text-embedding-3-small model returns 1,536 numbers by default and text-embedding-3-large returns 3,072.
Those numbers are not readable, and that is fine. What matters is the geometry: text about similar things lands in similar places. OpenAI lists the usual applications as search, clustering, recommendations, anomaly detection, diversity measurement and classification. Qdrant describes the same idea from the storage side, calling embeddings fixed length lists of numbers that represent the conceptual essence of unstructured data.
What a vector database actually does
Weaviate defines the category as systems that store, index and query embeddings, returning the objects whose vectors sit closest to a query vector. The important part of its definition is what comes after: production systems also keep the original objects or references to them, metadata filters, updates, access controls, replication and operational tooling.
Comparing a query against every stored vector is exact but slow, so these systems use approximate nearest neighbour search. Weaviate explains that such indexes organise vectors into structures like graphs, clusters or compressed partitions so a query examines promising candidates instead of everything. The common choice is HNSW, which is what Qdrant builds on. Milvus lists a wider set including HNSW, IVF, brute force FLAT, SCANN and DiskANN, because the right index depends on how much data you have and how much memory you will pay for.
Distance is measured in more than one way. pgvector, the Postgres extension, exposes six: L2 distance, inner product, cosine distance, L1 distance, and Hamming and Jaccard distance for binary vectors. Cosine similarity is the usual default for text.
When a plain database or a library is enough
This is the question most projects skip. If your data already lives in Postgres, pgvector adds vector search to it without adding a new service to operate, and you keep transactions, joins and replication. It stores vectors of up to 16,000 dimensions, though its HNSW and IVFFlat indexes cover up to 2,000 dimensions for the standard vector type, which is enough for most embedding models in common use.
At the other end, a library may be all you need. FAISS describes itself as a library for efficient similarity search and clustering of dense vectors, and says some of its methods scale to billions of vectors in main memory on a single server. Weaviate is candid about the difference: libraries generally do not provide authorisation, replication, backups, multi-tenancy, metadata management or managed scaling, so they suit prototypes, offline pipelines and embedded applications, while a dedicated service earns its place when you need a continuously updated, shared production system.
How a vector database fits retrieval augmented generation
Retrieval augmented generation is the pattern where a model is given relevant source material at the moment of the question rather than relying on what it absorbed during training. The 2020 paper by Lewis and colleagues that named the technique framed it as combining pre-trained parametric memory with non-parametric memory, which in practice means the model plus a searchable store.
The pipeline is short. Split your documents into chunks, embed each chunk, store the vectors with enough metadata to filter and to cite, embed the user’s question with the same model, retrieve the closest chunks, and pass them to the model with an instruction to answer from them. Weaviate puts the role of the store plainly: these systems are not required for every RAG system, but they are “a common way to make the retrieval layer fast, filterable, and continuously updatable”. If you are designing the retrieval layer, context engineering is the discipline that decides what actually reaches the model, and MCP servers are one common way of exposing a store to an assistant.
Two practical warnings. Query and stored vectors must come from the same embedding model, so changing models means re-embedding everything. And retrieval quality usually depends more on how you chunk and filter than on which engine you picked. That evaluation work is now a standard part of the AI engineer role.
6 questions to answer before you add a vector database
- How many vectors will you really have? Tens of thousands is a different problem from hundreds of millions, and only one of them needs dedicated infrastructure.
- Where does the data live now? If it is already in Postgres, try pgvector before adding a service you have to run, back up and monitor.
- How often does it change? A nightly rebuild suits a library. Continuous inserts, updates and deletes are what a database is for.
- Do you need filters and permissions? Combining semantic search with metadata filtering and per-user access control is where libraries run out, and it is rarely optional in a real product.
- Which embedding model, and can you afford to change it? Switching means re-embedding the whole corpus, so plan storage and time for at least one migration.
- How will you measure whether retrieval is good? Without a set of real questions and expected sources, you cannot tell a tuning problem from a tooling problem.
Common questions
What is a vector database in simple terms? It is a store for embeddings, the numeric representations a model produces for text, images or audio, built so you can quickly find the stored items whose numbers are closest to a query. That makes search work by meaning rather than by exact words.
Do I need one for RAG? Not always. Retrieval can run on a library, a search engine or an existing database with a vector extension. A dedicated system helps when retrieval must be fast, filterable and continuously updated for many users.
Can Postgres do this? Yes, with the pgvector extension, which supports vectors up to 16,000 dimensions, HNSW and IVFFlat indexes, and six distance functions alongside normal SQL, transactions and replication.
What is the difference between a vector database and FAISS? FAISS is a library for similarity search that you embed in your own program. A database wraps similar indexing in a service with storage, filtering, access control, backups and scaling.
What is HNSW? Hierarchical Navigable Small World is a graph based index for approximate nearest neighbour search. It examines a small set of promising candidates rather than every vector, which is what makes large collections searchable in milliseconds.
Sources and further reading
Where the figures and rules above come from, so you can check them:
- What an embedding is, and model dimensions: OpenAI API documentation
- Definition, ANN indexes, libraries versus databases and RAG: Weaviate
- Dense embeddings, HNSW, payload filtering and quantisation: Qdrant documentation
- Supported index types including HNSW, IVF, FLAT, SCANN and DiskANN: Milvus documentation
- Dimension limits, index types and distance operators: pgvector
- Similarity search library and scale: FAISS
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020): Lewis et al., arXiv
- Vector search for semantic retrieval and agent memory: Pinecone documentation
Photo credits: BalticServers data center by BalticServers.com, CC BY-SA 3.0, via Wikimedia Commons. 2006 228450201 card catalog by Beatrice Murch from Buenos Aires, Argentina, CC BY 2.0, via Wikimedia Commons.
Join the discussion