Vector Database: What It Is and How to Choose One

Every piece of content, a paragraph, a memory, a product description, gets converted into a vector: a fixed-length array of numbers produced by an embedding model, positioned so that semantically similar content ends up close together in that vector space. A vector database indexes those vectors so that, given a new query vector, it can quickly find the nearest neighbors without comparing against every single stored vector one by one.

Most production vector databases use an approximate nearest-neighbor algorithm, commonly HNSW (hierarchical navigable small world graphs), to make that search fast at scale. "Approximate" is a deliberate tradeoff: exact nearest-neighbor search gets prohibitively slow past a few hundred thousand vectors, and approximate methods trade a small amount of recall for orders-of-magnitude faster queries.

Fully managed vector databases handle sharding, replication, and index tuning for you, at the cost of less control over index parameters and, usually, a higher bill at scale. They're the right choice when you want zero operational overhead and are willing to pay a premium for it, especially early on when engineering time is scarcer than budget.

Self-hosted options give you direct control over index parameters, quantization, and hardware, which matters once query volume is high enough that infrastructure cost starts to dominate. The tradeoff is operational: you're now responsible for scaling, backups, and uptime for a piece of infrastructure that, if it goes down, takes your retrieval layer down with it.

If you already run PostgreSQL, the pgvector extension adds vector similarity search directly to your existing database, which means one less system to operate and the ability to join vector search against your relational data in a single query. For moderate scale, that simplicity often outweighs the raw performance edge a dedicated vector database offers.

A dedicated vector database still wins once query volume or vector count gets large enough that a general-purpose database's indexing strategy starts to show its limits, or when you need features, hybrid search, multi-tenancy, filtered search, that a bolt-on extension doesn't fully support. The right choice genuinely depends on scale, not on which option is newer.

The two terms get used interchangeably, but they describe different layers of the same system. Vector embeddings are the numeric representations themselves, the output of an embedding model given a piece of text, an image, or any other content. The database is the storage and indexing layer that holds a large collection of those embeddings and answers similarity queries against them quickly, the infrastructure the embeddings live in rather than the embeddings themselves.

In other words, embeddings are the data; a dedicated index is where that data lives and gets searched efficiently. You could technically store vector embeddings in a plain array or a spreadsheet, but similarity search over more than a few thousand of them would be far too slow to be useful without purpose-built indexing, which is exactly the problem this category of system exists to solve.

Benchmarks that only report raw query latency miss the tradeoff that actually matters: latency versus recall. An index configured for maximum speed will return results faster but miss some true nearest neighbors; the same index tuned for higher recall will be slower but more accurate. Before choosing one, test recall@k on your own data and query patterns, not just a generic benchmark dataset, since recall behavior varies a lot depending on how tightly clustered your embeddings are.

Cost at scale is the other number worth modeling early. Some vector databases charge primarily for stored vectors, others for query volume, and quantization, storing lower-precision approximations of each vector, can cut storage and memory cost substantially at a small, usually acceptable, hit to recall. Running that cost projection against your expected vector count before committing to one option avoids an expensive surprise once a prototype turns into a production workload.

Most teams don't run a single flat index; they run one that has to stay correctly partitioned across many organizations, projects, or users at once. That's a harder requirement than it sounds. Filtering by a tenant id after the nearest-neighbor search runs can silently shrink the effective top-k below what a query actually needs, so the filtering ideally happens as part of the index traversal itself, not as a post-processing step layered on top of it.

Isolation matters just as much as filtering correctness. If one tenant's query can ever, even accidentally through a misconfigured filter or a bug in application code, retrieve another tenant's vectors, that's not a performance bug, it's a data leak. Any system holding vectors from multiple customers or teams should be evaluated on how strictly it enforces that boundary, not just on how fast its queries run, and that evaluation should include what happens when a filter is missing entirely rather than only when it's present and correct.

Similarity search is good at finding content that's semantically close to a query, but it's a poor fit for precise, structured questions: exact IDs, numeric ranges, or multi-hop relationships between entities. A vector database will happily return content that's topically similar to "which servers does Alex's config apply to" without ever answering the actual question, because that answer depends on following a relationship, not matching a similarity score.

This is why most serious retrieval systems pair vector search with keyword and graph-based lookup rather than relying on similarity alone, a pattern usually called hybrid search. A vector database is one essential component of a retrieval system, not the whole system by itself.

stored doesn't ask you to stand up or choose a vector database at all. Every memory your agents write is automatically indexed for vector similarity as part of the same write, alongside full-text and graph indexing, so a single query gets hybrid retrieval across all three without you operating a separate vector store or tuning HNSW parameters yourself.

That matters most for teams who'd rather ship a product than run infrastructure: the vector index, the graph, and the keyword index all live behind one remote MCP endpoint, so the underlying vector database is an implementation detail you never have to think about.