What is a Vector Database? The Infrastructure Layer of Modern AI

Quick answer
Vector databases are specialised data systems that store and search high-dimensional vectors using similarity-based queries, enabling AI applications to find information based on meaning rather than exact keyword matches, typically in under 50 milliseconds.
Published: April 16, 2026
Last Updated: November 9, 2026
Vector Databases: Santage Learn cover showing a grid of stored vectors with one highlighted match radiating from a query point, representing similarity search

In the architecture of modern AI, three layers sit in a stack. Large language models provide reasoning. Embeddings translate meaning into math. And between them, holding the entire edifice together, sits a layer of infrastructure that did not meaningfully exist five years ago: the vector database.

Vector databases are the reason AI systems can search the meaning of a billion documents in milliseconds. They are the infrastructure that makes retrieval-augmented generation possible at enterprise scale. And they have moved from niche to standard in a few years: vector search is now built into PostgreSQL, Elasticsearch, MongoDB, and the major cloud data platforms, alongside a field of dedicated vector databases.

Key facts about vector databases

What is a vector database?

A vector database is a specialised data storage and retrieval system designed to index and search high-dimensional vectors, typically embeddings produced by AI models, using similarity-based queries rather than exact matches.

Unlike a traditional relational database that answers "find all customers named Smith," a vector database answers "find the ten documents most similar in meaning to this query." It compares the mathematical distance between stored vectors and a query vector to return the closest matches.

In short: a vector database stores the mathematical meaning of your data, then finds the stored items whose meaning is most similar to a new query, returning results in milliseconds even across billions of entries.

How do vector databases work?

The architecture of a vector database is purpose-built for one task: finding the nearest neighbours in high-dimensional space, as fast as possible.

How Vector Databases Process Queries: 6-stage architecture from query input through embedding, index search, filtering, ranking, to results. Santage.
The 6-stage vector database query pipeline. Approximate nearest neighbor (ANN) search at stage 3 is what enables millisecond retrieval across billions of vectors.
  1. Ingestion: Raw data is passed through an embedding model that converts it into vectors. These vectors, along with metadata, are stored in the database.
  2. Indexing: The database builds specialised index structures for high-dimensional similarity search. The most common is HNSW (Hierarchical Navigable Small World), a graph-based algorithm.
  3. Query embedding: When a user query arrives, it is embedded using the same model, ensuring the query vector exists in the same semantic space as stored vectors.
  4. Approximate nearest neighbour search: The database searches the index to find the closest vectors. ANN algorithms trade a small amount of accuracy for enormous gains in speed.
  5. Metadata filtering and hybrid search: Most production queries combine vector similarity with traditional filters and keyword matching (BM25) in a pattern called hybrid search.
  6. Ranked results: The database returns the top matches, ranked by similarity score, typically within milliseconds.

The core idea behind vector databases

Vector databases search by meaning, not by match.

A traditional database asks "does this record exactly equal the query?" A vector database asks "which stored records are most similar in meaning to the query?" The first is deterministic and binary. The second is probabilistic and continuous.

How do vector databases differ from related systems?

ConceptDifference
Vector DB vs Relational DBRelational databases use exact matches on structured data. Vector databases use similarity search on high-dimensional vectors
Vector DB vs Search EngineSearch engines excel at keyword matching. Vector databases excel at semantic similarity. Modern hybrid systems combine both
Vector DB vs Vector LibraryLibraries like FAISS provide algorithms but lack database features such as persistence, replication, and concurrent access
Vector DB vs Vector IndexA vector index is a data structure. A vector database is a full system managing indexes, data, metadata, and queries
Vector DB vs Graph DBGraph databases excel at relationship traversal. Vector databases excel at similarity across unstructured data

How do vector databases power RAG systems?

Retrieval-augmented generation is the single most important application pattern for vector databases. In a RAG system, the vector database holds embeddings of an organisation's knowledge base. When a user asks a question, the system retrieves the most relevant documents and passes them to a large language model as context.

The pattern traces to the 2020 RAG paper from Facebook AI Research, University College London, and New York University, which paired a neural retriever over a dense vector index of Wikipedia with a text generator and found it produced more specific, diverse, and factual language than a generation-only model (Lewis et al., 2020). It is now the default way to connect a language model to private knowledge.

In short: vector databases are the component that makes AI useful in production. Without them, enterprise AI cannot retrieve relevant context at scale, and modern architectures like RAG cannot function.

What are the leading vector databases in 2026?

  1. Pinecone: An early fully managed vector database. Prioritises ease of use and serverless operations.
  2. Weaviate: Open-source with enterprise support. Distinctive strength in native hybrid search that combines vector and keyword retrieval.
  3. Milvus: The scalability leader. Engineered for billions-of-vectors workloads with GPU acceleration and distributed deployment. Maintained by Zilliz.
  4. Qdrant: Performance-focused open-source option. Written in Rust. Strong support for complex metadata filtering.
  5. Chroma: Developer-experience-first option for rapid prototyping with minimal configuration.
  6. pgvector: PostgreSQL extension adding vector search to the most widely deployed relational database in the world.

What are the challenges of using vector databases?

Where are vector databases used in practice?

  1. Enterprise knowledge: Microsoft Copilot, Google Workspace with Gemini, Notion AI, and Atlassian Intelligence all rely on vector databases.
  2. Customer support: Zendesk, Intercom, and Salesforce use vector databases for AI-driven support.
  3. AI search engines: Perplexity, You.com, and newer AI-first search engines use vector databases as core retrieval infrastructure.
  4. Financial services: Banks and asset managers use vector search for document analysis, compliance review, and research retrieval across filings and internal reports.
  5. Healthcare: Mayo Clinic and pharmaceutical companies use vector databases to search clinical literature and match patients to trials.
  6. Cybersecurity: CrowdStrike and Palo Alto Networks use vector databases to detect anomalous patterns across networks.
  7. AI agent memory: Emerging agent frameworks use vector databases as the memory layer for long-running autonomous agents.

What indexing algorithms do vector databases use?

Comparing a query against every stored vector, called exact or brute-force search, is accurate but scales poorly: a billion vectors means a billion distance calculations per query. Vector databases instead use approximate nearest neighbour (ANN) indexes that check a small fraction of the data and accept a small chance of missing the true closest match. Four families of index dominate.

Every index trades among three quantities: recall, meaning the share of true nearest neighbours found; latency; and memory or cost. Index parameters let operators move along that curve. Meta's open-source FAISS library, used inside many commercial products, implements most of these methods and made GPU-accelerated billion-scale search practical (Johnson, Douze, and Jégou, 2017).

In short: vector databases are fast because they do not compare a query with everything; approximate indexes such as HNSW, IVF, and product quantization search a small, well-chosen fraction of the data and trade a little recall for large gains in speed and cost.

Do you need a dedicated vector database?

Not always. Since 2023, vector search has become a feature of databases many organisations already run. PostgreSQL offers it through the pgvector extension, and MongoDB Atlas, Elasticsearch, OpenSearch, Redis, and the major cloud data warehouses all provide vector indexes. For many teams, the right first step is to use the database they already operate.

SituationUsually the better fit
Up to a few million vectors, data already in a relational databaseVector extension of the existing database, such as pgvector
Search must combine vectors with complex joins, transactions, or permissionsExisting operational database with vector support
Hundreds of millions to billions of vectors with strict latency targetsDedicated vector database built for distributed ANN search
Heavy real-time ingestion alongside high query volumeDedicated vector database or managed service
Prototype or single-developer projectLightweight embedded option or managed free tier

The deciding factors are scale, latency requirements, how often data changes, and operational capacity. A dedicated system adds another component to secure, monitor, and keep in sync with the source of truth.

Synchronisation is the cost teams most often underestimate. When a document is edited, deleted, or has its access permissions changed in the system of record, the matching vectors must be updated or removed too, or the AI system will keep retrieving content that no longer exists or that a user should no longer see. Keeping vectors inside the same database as the source data makes this consistency automatic. Keeping them in a separate system requires a reliable pipeline that propagates every change, plus monitoring to catch the cases where it fails. For many organisations, that operational burden, rather than raw search speed, is what decides the architecture. A practical pattern is to start with vector search inside the existing database, measure recall and latency on real traffic, and move to a dedicated system only when those measurements show the need.

How do vector databases serve as memory for AI agents?

The fastest-growing use of vector databases in 2026 is as the memory layer for AI agents. A large language model on its own remembers nothing between sessions and can only hold a limited amount of text in its context window. Agents that run for hours, serve the same user over weeks, or coordinate with other agents need somewhere durable to store what they learn.

Research systems established the pattern. Stanford and Google's "generative agents" kept a memory stream of observations and retrieved entries by a combination of recency, importance, and embedding similarity (Park et al., 2023). MemGPT treated the context window like a computer's main memory and a vector store like its disk, moving information between them as needed (Packer et al., 2023).

In production, agent memory typically has three tiers: the current context window, a vector store of past interactions and learned facts, and structured records in conventional databases. The vector store is often exposed to the agent as a tool it can query, following the patterns described in our guide to tool use and function calling, and increasingly through a standard server interface such as the Model Context Protocol. In multi-agent systems, a shared vector store lets specialised agents build on each other's findings, which makes the store's partitioning and access rules part of the orchestration design.

Memory also creates a new attack surface. Anything an agent retrieves enters its context and can influence its actions. Researchers demonstrated indirect prompt injection, in which instructions hidden in retrieved content take control of the model (Greshake et al., 2023). The PoisonedRAG study found that inserting five crafted texts per target question into a knowledge base of millions of documents achieved about a 90% attack success rate (Zou et al., 2024). For agents that can take real actions, write access to the vector store must be controlled as carefully as write access to code.

Agent memory also needs housekeeping that document search rarely does. Memories should be isolated per user or tenant so one customer's history never surfaces in another's session, deduplicated so repeated observations do not crowd out new ones, and given expiry or review rules so outdated facts do not keep steering decisions. Each stored entry should record its source and time, so that when an agent acts on a memory, an operator can trace where that belief came from.

In short: for AI agents, a vector database is long-term memory, which makes its retrieval quality a limit on agent reliability and its write permissions a security boundary.

How do you evaluate a vector database?

Vendor benchmarks are hard to compare because each tunes its own system for the test. Independent harnesses such as ANN-Benchmarks measure recall against queries per second across algorithms on standard datasets (Aumüller, Bernhardsson, and Faithfull, 2018), but the only reliable test is a workload that resembles yours. The metrics that matter are:

  1. Recall at k: The share of true top-k neighbours returned. Measure it against brute-force results on a sample of real queries.
  2. Tail latency: The 95th and 99th percentile response times under realistic load, not the average.
  3. Filtered performance: Speed and recall when queries include metadata filters such as user, date, or permission. Many systems slow down or lose recall sharply under restrictive filters.
  4. Ingestion and freshness: How quickly new or updated vectors become searchable, and whether heavy writes degrade query speed.
  5. Total cost: Memory, storage, compute, and operational effort at your target scale, including the cost of re-indexing when the embedding model changes.

Measure end-to-end quality too. For a RAG or agent system, a small drop in recall can matter less than adding a reranking step, or a great deal more if the missing documents are the ones that answer the question.

The future of vector databases

  1. Hybrid search becomes the default: The combination of vector similarity, keyword matching, and metadata filtering in a single query interface is emerging as the standard.
  2. Consolidation into existing databases: PostgreSQL with pgvector, MongoDB Atlas, and Elasticsearch now offer vector search, and for many workloads the question is shifting from which vector database to adopt to whether a separate one is needed at all.
  3. GPU-accelerated retrieval: GPU indexing, demonstrated at billion-vector scale in Meta's FAISS research (Johnson et al., 2017), is increasingly used to cut index build times and raise query throughput for very large collections.
  4. Multimodal storage: Vector databases are evolving to store and search across text, images, audio, and video.
  5. Agent memory: As autonomous AI agents proliferate, vector databases are becoming the memory layer for agent state.

The choice of vector database is no longer a tactical technology decision. It is an architectural decision that shapes what an organisation can build with AI for years to come.

Frequently asked questions

What is the difference between a vector database and a vector index?
A vector index is a data structure that enables fast similarity search. A vector database is a complete system including indexing, storage, query processing, concurrency, persistence, and replication.
Do I need a specialised vector database?
For collections up to a few million vectors with moderate query volume, PostgreSQL with pgvector is often sufficient. At hundreds of millions or billions of vectors with strict latency targets, dedicated vector databases are usually the better fit. Test with your own data before deciding.
Which vector database is best for RAG?
There is no single best choice. Pinecone excels at operational simplicity, Weaviate at hybrid search, Milvus at scale, Qdrant at performance, and pgvector at PostgreSQL integration.
How much does a vector database cost?
Open-source options are free to self-host. Managed services start with free tiers and scale to hundreds or thousands of dollars monthly for enterprise deployments.
Can a vector database replace my existing database?
No. Vector databases complement existing databases. Transactional data and structured records still belong in traditional systems. Vector databases add similarity search over unstructured data.
How do AI agents use vector databases?
AI agents use vector databases as long-term memory. They store past interactions, learned facts, and documents as embeddings and retrieve the relevant entries when needed, often by calling the database as a tool. Multi-agent systems use a shared vector store so agents can build on each other's work.
What is HNSW?
HNSW, or Hierarchical Navigable Small World, is a graph-based index for approximate nearest neighbour search. It links each vector to its near neighbours across several layers, letting searches move quickly from coarse to fine. It is the default index in most vector databases.

Sources and further reading

  1. Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Facebook AI Research, UCL, NYU, 2020. arxiv.org/abs/2005.11401
  2. Morris, J. X. et al. Text Embeddings Reveal (Almost) As Much As Text. Cornell, EMNLP 2023. arxiv.org/abs/2310.06816
  3. MLCommons. ANN Benchmarks for Vector Search. mlcommons.org
  4. European Commission. The EU AI Act. 2025. digital-strategy.ec.europa.eu
  5. Malkov, Y. and Yashunin, D. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. 2016. arxiv.org/abs/1603.09320
  6. Jégou, H., Douze, M., and Schmid, C. Product Quantization for Nearest Neighbor Search. IEEE TPAMI, 2011. doi.org/10.1109/TPAMI.2010.57
  7. Subramanya, S. J. et al. DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node. Microsoft Research, NeurIPS 2019. papers.nips.cc
  8. Johnson, J., Douze, M., and Jégou, H. Billion-scale Similarity Search with GPUs. Meta AI, 2017. arxiv.org/abs/1702.08734
  9. Aumüller, M., Bernhardsson, E., and Faithfull, A. ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms. 2018. arxiv.org/abs/1807.05614
  10. Park, J. S. et al. Generative Agents: Interactive Simulacra of Human Behavior. Stanford and Google, 2023. arxiv.org/abs/2304.03442
  11. Packer, C. et al. MemGPT: Towards LLMs as Operating Systems. UC Berkeley, 2023. arxiv.org/abs/2310.08560
  12. Greshake, K. et al. Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. 2023. arxiv.org/abs/2302.12173
  13. Zou, W. et al. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. 2024. arxiv.org/abs/2402.07867