If large language models are the engines of modern AI, embeddings are the language they speak. Every word, image, audio clip, or document that passes through a modern AI system is first converted into an embedding. This is the quiet, foundational layer that makes semantic search work, that allows ChatGPT to remember context, that powers recommendation systems on Netflix and Spotify, and that enables enterprise AI to reason over proprietary data.
Despite this centrality, embeddings remain one of the least-understood concepts in AI. This guide explains what embeddings are, how they work, why they matter, and where they are headed.
- What they are: Numerical representations of data, typically vectors of 384 to 3,072 dimensions, that capture meaning in mathematical form
- Why they matter: The foundation layer beneath semantic search, recommendation systems, RAG pipelines, and every major AI retrieval application
- How they work: Similar items are placed close together in high-dimensional vector space; dissimilar items are placed far apart
- Leading providers: OpenAI, Cohere, Voyage AI, Google, and open-source options from BAAI, Nomic, and Hugging Face
- Typical dimensionality: Production systems use 512 to 1,536 dimensions; research models can reach tens of thousands
- Core use cases: Semantic search, recommendation engines, retrieval-augmented generation, clustering, anomaly detection
- Primary limitation: Embeddings inherit biases from training data and can become stale as language evolves
What is an embedding?
An embedding is a numerical representation of data, typically a list of hundreds or thousands of numbers, that captures the meaning and relationships of that data in a mathematical form an AI system can process.
Text, images, audio, video, code, even graph structures can all be converted into embeddings. The process compresses complex, high-dimensional information into a compact vector where semantically similar items end up close together and dissimilar items end up far apart. This is the property that makes embeddings so powerful. They transform meaning into math.
How do embeddings work?
The process of creating an embedding follows a consistent pipeline regardless of the data type:
- Raw input: A piece of data is provided to the system. This could be a sentence, an image, a product description, or an entire document.
- Embedding model: The input is passed through a specialised neural network trained specifically to produce embeddings. These models are typically derived from or distilled from large foundation models.
- Vector generation: The model outputs a vector, typically between 384 and 3,072 dimensions. Each dimension captures some aspect of the input's meaning, though individual dimensions are not human-interpretable.
- Vector space mapping: The vector represents a point in a high-dimensional space. Similar inputs produce vectors that occupy nearby points in this space.
- Similarity comparison: When needed, vectors are compared using mathematical distance metrics such as cosine similarity or Euclidean distance. Closer vectors mean greater semantic similarity.
The breakthrough that made modern embeddings possible was the introduction of transformer-based architectures, published in the 2017 paper Attention Is All You Need.
The core idea behind embeddings
Embeddings turn meaning into geometry.
When text, images, or any other data is embedded, it is placed in a mathematical space where distance equals meaning. Two points close together represent two concepts close in meaning. This geometric property is what allows AI systems to reason about similarity at a scale and speed that keyword matching cannot match.
How do embeddings differ from related concepts?
| Concept | Difference |
|---|---|
| Embedding vs Token | A token is a unit of raw text. An embedding is the numerical meaning representation of that token |
| Embedding vs Vector | All embeddings are vectors, but not all vectors are embeddings. Embeddings are vectors designed to represent meaning |
| Embedding vs Keyword | Keywords require exact matches. Embeddings capture semantic similarity |
| Embedding vs Hash | A hash creates a unique identifier. An embedding creates a meaningful representation |
| Embedding vs Feature Vector | Feature vectors use hand-engineered attributes. Embeddings are learned automatically from data |
How do embeddings enable LLMs?
Large language models use embeddings at two critical points. Internally, every input an LLM processes is converted into embeddings before the model can operate on it. The transformer layers inside the LLM are sophisticated machinery for transforming these embeddings based on context.
Externally, when an LLM is connected to a knowledge base or search system, embeddings serve as the retrieval mechanism. The user's query is embedded, compared against stored embeddings, and the most relevant documents are returned to the LLM as context. This is the RAG pattern.
How do embeddings power vector databases?
Vector databases exist specifically to store, index, and search embeddings at scale. Traditional databases were built for exact matches on structured data. They cannot efficiently search across millions of high-dimensional vectors to find the closest match.
The category barely existed before 2020. Today vector search is offered both by dedicated vector databases and as a built-in feature of established systems such as PostgreSQL, Elasticsearch, and MongoDB, a measure of how quickly embeddings moved from research into standard infrastructure.
Why are embeddings important for AI systems?
- Semantic search: Google, Perplexity, and enterprise search tools use embeddings to understand what users actually mean, not just what they typed.
- Recommendation systems: Netflix, Spotify, YouTube, and Amazon embed user behaviour and content into the same vector space, then recommend items whose embeddings are close to the user's.
- RAG pipelines: Every retrieval-augmented generation system uses embeddings as the retrieval mechanism.
- Clustering and classification: Embeddings allow organisations to automatically organise millions of documents by meaning.
- Anomaly detection: Banks and cybersecurity firms use embeddings to detect behaviours that deviate from normal patterns.
In most production AI systems, embeddings form one of three foundational layers, alongside the foundation models that generate answers and the vector databases that store and search the embeddings themselves.
What are the limitations of embeddings?
- Embedding drift: As language and an organisation's own terminology evolve, embeddings from an older model represent new terms less accurately. Re-embedding is usually triggered by adopting a new model or by a measured decline in retrieval quality, not by a fixed calendar.
- Domain specificity: General-purpose embedding models perform poorly on specialised domains such as medical, legal, or highly technical content.
- Dimensionality trade-offs: Higher-dimensional embeddings capture more nuance but cost more to store and search.
- Bias inheritance: Embeddings trained on web-scale data inherit the biases present in that data. Researchers at Boston University and Microsoft Research showed word embeddings completing "man is to computer programmer as woman is to homemaker" (Bolukbasi et al., 2016), and a Princeton study in Science found embeddings reproduce human-like biases measured by psychologists (Caliskan et al., 2017).
- Interpretability: Individual dimensions of an embedding vector are not human-interpretable, creating challenges for auditing in regulated industries.
- Security: Research on arXiv has demonstrated that some embeddings can be partially reversed, raising privacy concerns.
Where are embeddings used in practice?
- Search and discovery: Google's MUM and BERT systems, Bing's vector search, and Perplexity's retrieval engine all use embeddings at their core.
- E-commerce: Amazon, Shopify, and Alibaba use embeddings for product recommendations, visual search, and personalisation.
- Healthcare: Mayo Clinic and academic medical centres use embeddings to organise clinical literature and medical imaging.
- Finance: Banks use embeddings for fraud detection, document processing, and research retrieval. McKinsey estimated in 2023 that generative AI could add $200 billion to $340 billion a year in value to the global banking industry (McKinsey, 2023).
- Cybersecurity: CrowdStrike and Palo Alto Networks embed network traffic patterns to detect anomalies.
- Enterprise knowledge: Microsoft Copilot, Google Workspace with Gemini, and Notion AI all use embeddings to make internal documents searchable by meaning.
How are embedding models trained?
Embedding models learn meaning by comparison. The dominant method is contrastive learning: the model is shown pairs of items that should be close together, such as a question and its correct answer, a sentence and its paraphrase, or an image and its caption, alongside items that should be far apart. Training adjusts the model so that matching pairs produce nearby vectors and mismatched pairs produce distant ones. Repeated across millions or billions of examples, this produces a vector space in which distance tracks meaning.
The idea has a clear lineage. In 2013, Google researchers introduced word2vec, which learned a vector for each word from the words around it and showed that simple arithmetic on those vectors captured relationships, with the vector for "king" minus "man" plus "woman" landing near "queen" (Mikolov et al., 2013). Word vectors, however, gave every word one fixed meaning regardless of context. Transformer models fixed that by producing context-dependent representations, and in 2019 Sentence-BERT adapted them to produce a single embedding per sentence. Its authors reported that finding the most similar pair in a collection of 10,000 sentences fell from about 65 hours with standard BERT to about 5 seconds (Reimers and Gurevych, 2019).
Contrastive training also works across data types. OpenAI's CLIP was trained on 400 million image and text pairs collected from the internet, learning to place a photo and its description near each other in a shared space (Radford et al., 2021). That shared space is what lets a text query retrieve an image, and it is the foundation of most multimodal search today.
A newer training refinement is Matryoshka representation learning, which arranges information so that the first dimensions of a vector carry the most important signal. The vector can then be truncated to a shorter length with limited loss of quality. Its authors reported up to 14 times smaller embeddings at the same accuracy on an ImageNet classification task (Kusupati et al., 2022), and OpenAI's text-embedding-3 models expose the same property through a parameter that shortens output vectors.
What are the different types of embeddings?
"Embedding" covers several distinct representations, and choosing the right type matters more than choosing the most popular model.
- Token embeddings: The vectors a transformer assigns to each token at its input layer. They are internal to every language model and rarely used directly by application developers.
- Sentence and document embeddings: One vector for a whole passage. These are the workhorse of semantic search and RAG, and what most "embedding APIs" return.
- Multimodal embeddings: Vectors for images, audio, or video placed in the same space as text, so that a sentence can retrieve a picture and vice versa.
- Dense embeddings: Vectors in which every dimension holds a value. They capture meaning well but can miss exact matches on rare terms such as product codes or names.
- Sparse embeddings: Very long vectors in which most values are zero and the non-zero values correspond to vocabulary terms. Learned sparse models such as SPLADE combine keyword precision with some semantic understanding (Formal et al., 2021).
- Multi-vector (late interaction) embeddings: Instead of one vector per document, systems such as ColBERT keep one vector per token and compare query and document tokens at search time, improving precision at the cost of storage (Khattab and Zaharia, 2020).
Production retrieval systems increasingly combine types. A common pattern is hybrid search, which runs dense and sparse retrieval together and merges the results, so the system catches both a paraphrased question and an exact part number.
How do you choose and evaluate an embedding model?
The standard public reference is the Massive Text Embedding Benchmark (MTEB), which at launch covered 8 task types across 58 datasets and 112 languages. Its central finding still holds: no single embedding method dominated across all tasks (Muennighoff et al., 2022). A model that leads on retrieval may trail on clustering or classification, so a leaderboard rank is a starting point, not a decision.
A practical selection process weighs five factors:
- Task fit: Check scores on the MTEB task category that matches your use, usually retrieval, rather than the overall average.
- Domain fit: Test on a sample of your own documents and real queries. General models can underperform on legal, medical, financial, or code-heavy text.
- Dimensions and storage: A 1,536-dimension vector stored as 32-bit floats takes about 6 KB. Shorter vectors, Matryoshka truncation, or quantization to 8-bit integers or single bits can cut storage by 4 to 32 times, with some loss of accuracy that should be measured.
- Language coverage: Multilingual models let a query in one language retrieve documents in another, but quality varies widely by language.
- Lock-in: Vectors from different models are not compatible. Switching models means re-embedding the entire corpus, so the choice carries a long-term cost.
The most reliable evaluation is a small, labelled test set built from your own data: a few hundred real questions, each paired with the documents that should be retrieved. Measure how often the right documents appear in the top results, then compare candidate models on that set.
How do embeddings power AI agents?
Embeddings are no longer only a search technology. They have become core infrastructure for AI agents, the systems that plan and take actions on a user's behalf, in three specific ways.
Memory. An agent working across many steps or sessions cannot keep everything in its context window. It stores past observations, decisions, and user preferences as embeddings and retrieves the relevant ones when needed. Stanford and Google researchers' "generative agents" scored stored memories on recency, importance, and relevance, with relevance measured by embedding similarity to the current situation (Park et al., 2023). MemGPT extended the idea into a tiered memory system modelled on an operating system, paging information in and out of the model's context (Packer et al., 2023).
Tool selection. Agents that can call external functions, the subject of our guide to tool use and function calling, may have access to hundreds of tools. Listing every tool description in the prompt wastes context and degrades accuracy. A common solution is to embed each tool's description and retrieve only the few that match the current task. The same approach helps agents navigate large catalogues of servers exposed through the Model Context Protocol.
Shared knowledge. In multi-agent systems, several specialised agents often read from and write to a common knowledge store, usually a vector database, so that one agent's findings are available to the others. How that store is partitioned and permissioned becomes part of the orchestration design.
The quality of the embeddings therefore sets a ceiling on agent reliability. An agent that retrieves the wrong memory or the wrong tool will act confidently on the wrong basis, one of the routes by which agents produce hallucinations that look like reasoning errors but begin as retrieval errors.
The future of embeddings
- Multimodal unification: OpenAI's CLIP placed images and text in one vector space, and Meta's ImageBind extended a single embedding space to six modalities: images, text, audio, depth, thermal, and motion sensor data (Girdhar et al., 2023).
- Cheaper inference: Stanford HAI's 2025 AI Index found that the cost of querying a model at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024 (Stanford HAI, 2025). Embedding models are far smaller than generative models, so the same trend has made it affordable to embed entire document archives.
- API commoditisation: High-quality embedding models are becoming available via simple APIs at continuously falling prices.
- Privacy-preserving embeddings: Cornell researchers showed a method called Vec2Text could reconstruct 92% of 32-token text inputs exactly from their embeddings (Morris et al., 2023), so stored embeddings need the same protection as the text they encode. Data governance obligations under the EU AI Act add to the pressure for privacy-preserving methods.
Organisations that treat embeddings as a strategic capability rather than a technical curiosity will build meaningful AI moats. Those that treat embeddings as commodity will find themselves dependent on whoever controls the embedding layer above them.
Frequently asked questions
Sources and further reading
- Vaswani, A. et al. Attention Is All You Need. Google Research, 2017. arxiv.org/abs/1706.03762
- McKinsey Global Institute. The economic potential of generative AI: The next productivity frontier. June 2023. mckinsey.com
- Stanford Institute for Human-Centered AI. AI Index Report 2025. 2025. hai.stanford.edu
- Bolukbasi, T. et al. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. Boston University and Microsoft Research, 2016. arxiv.org/abs/1607.06520
- Caliskan, A., Bryson, J., and Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science, 2017. doi.org/10.1126/science.aal4230
- Girdhar, R. et al. ImageBind: One Embedding Space To Bind Them All. Meta AI, 2023. arxiv.org/abs/2305.05665
- Morris, J. X. et al. Text Embeddings Reveal (Almost) As Much As Text. Cornell, EMNLP 2023. arxiv.org/abs/2310.06816
- European Commission. Regulatory framework on AI (EU AI Act). digital-strategy.ec.europa.eu
- Mikolov, T. et al. Efficient Estimation of Word Representations in Vector Space. Google, 2013. arxiv.org/abs/1301.3781
- Reimers, N. and Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. 2019. arxiv.org/abs/1908.10084
- Radford, A. et al. Learning Transferable Visual Models From Natural Language Supervision (CLIP). OpenAI, 2021. arxiv.org/abs/2103.00020
- Kusupati, A. et al. Matryoshka Representation Learning. 2022. arxiv.org/abs/2205.13147
- Muennighoff, N. et al. MTEB: Massive Text Embedding Benchmark. 2022. arxiv.org/abs/2210.07316
- Khattab, O. and Zaharia, M. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. Stanford, 2020. arxiv.org/abs/2004.12832
- Formal, T. et al. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. 2021. arxiv.org/abs/2107.05720
- Park, J. S. et al. Generative Agents: Interactive Simulacra of Human Behavior. Stanford and Google, 2023. arxiv.org/abs/2304.03442
- Packer, C. et al. MemGPT: Towards LLMs as Operating Systems. UC Berkeley, 2023. arxiv.org/abs/2310.08560
