Do You Actually Need a Vector Database?

Adding retrieval to an application now comes with a default architectural suggestion: stand up a dedicated vector database. It is a reasonable option and it is not the starting point most teams need.

If your data already lives in Postgres, the relevant question is whether the workload has crossed a threshold that justifies a second datastore, with its own operations, backups, access model and failure modes.

What you give up by adding a store

The cost of a specialised database is rarely the licence. It is that your embeddings now live somewhere other than the rows they describe.

That means no transactional guarantee between them. A document updates, the embedding job fails, and the index now describes text that no longer exists. Every team that separates these eventually builds a reconciliation process, and the ones that build it early do so because they were bitten.

It also means no joins. Filtering results by tenant, permission, date or status — which almost every real application needs — becomes either a metadata filter in a system with a weaker query language, or a second round trip and a merge in application code. Both are more code than a WHERE clause.

Where Postgres is genuinely enough

With pgvector, an HNSW index and reasonable dimensions, Postgres handles corpora in the low millions of vectors with query latency in the tens of milliseconds. That covers most internal knowledge bases, product catalogues, support archives and document sets.

You also get things a dedicated store makes you rebuild: real transactions, familiar backup and restore, row-level security if you use it, the same connection pool and monitoring, and one system for your on-call to understand at 3am.

The operational simplicity is the argument. It is worth more than a benchmark delta on a workload that is not your bottleneck.

The thresholds that change the answer

Four, in our experience:

• Scale past roughly ten million vectors, where index build time and memory pressure start competing with your transactional workload on the same instance.
• Sustained high query concurrency — vector search is CPU-hungry, and on a shared primary that contends directly with the queries paying your bills.
• Frequent full re-embedding, as when you change model. Rebuilding a large index in place is disruptive in a way a purpose-built store handles better.
• Retrieval features you would otherwise build, such as native hybrid search fusion or learned re-ranking. If you would implement them anyway, buying them is legitimate.

Notice what is not on the list: recall quality on a small corpus, and "we might scale later". The first is dominated by chunking and re-ranking rather than by the index; the second is a migration you can do when the threshold arrives, from a system that was cheaper in the meantime.

The failure mode: optimising the wrong layer

Teams unhappy with retrieval quality often change datastore, and the quality does not move. It usually was not the index.

Retrieval quality is mostly decided before the vector search runs — how documents are chunked, whether chunks carry enough surrounding context to be interpretable alone, whether the embedding model suits the domain, and whether anything re-ranks the candidates. A different database performs the same nearest-neighbour lookup on the same embeddings.

Measure before migrating. If recall at ten is poor, a new store will give you poor recall faster.

When we would advise against Postgres

If your source data is not in Postgres, the argument weakens considerably. The case rests on co-locating embeddings with the rows they describe; if the rows are somewhere else, pick on merit.

If your team has no Postgres operational experience, adding vector workloads to a database nobody tunes is not the simple option it appears to be.

And if you are already past the thresholds above, do not fight it. Running a large vector workload on a transactional primary to avoid a second system is how you turn a retrieval problem into an availability problem.

The second store has a second operational bill

The comparison teams run is query performance. The comparison that decides the year is operational.

A separate store needs its own backup and restore, tested rather than assumed. Its own access control, which now has to agree with the permissions in your primary database. Its own upgrade path, monitoring, capacity planning and on-call knowledge. And its own failure mode during an incident, when someone has to reason about two systems disagreeing.

None of that is prohibitive. It is simply real, recurring, and absent from the benchmark that justified the migration. If your team is small, this is often the deciding factor rather than a footnote — one datastore that everyone understands beats two where the second is understood by one person who is on holiday.

A reasonable sequence

Start in Postgres. Instrument recall and latency against a fixed evaluation set, so you have numbers rather than impressions. Spend your first optimisation effort on chunking and re-ranking, which is where the quality is. Watch index build time and CPU share as the corpus grows.

Migrate when a threshold is actually crossed, and keep the embedding pipeline decoupled enough that the store is a swap rather than a rewrite. That decoupling is the durable decision here — more so than which database you picked first.

One caution on benchmarks you will read: most compare raw nearest-neighbour throughput on a synthetic corpus with no filtering. Real queries filter by tenant, permission and recency, and filtered vector search behaves very differently from unfiltered. Benchmark your own shape of query or the numbers are decoration.

We cover the cost side in what RAG costs in production, and the platform work in data platforms.