Filtered Vector Database Benchmark: Latency, Recall & Cost in 2026

Vector search benchmarks are finally becoming useful. The meaningful question in 2026 isn’t which database posts the highest QPS number; it’s which system preserves filtered recall and acceptable p99 latency while vectors are being written, tenants are unevenly distributed, and a node or zone fails.

Here’s the production version of that problem: a tenant asks for ten document matches, but an ACL filter removes results belonging to other customers. The system returns three. Was the tenant’s corpus genuinely sparse, or did the index search the wrong candidate pool? A benchmark that measures only unfiltered nearest-neighbor speed won’t tell you.

Key takeaways

• Filtered recall can change the ranking completely. Strong unfiltered recall does not guarantee useful results after tenant, permission, or category filters.
• Scale is expensive. Ten million 768-dimensional FP32 vectors require 30.7 GB before indexes, metadata, replicas, and headroom.
• Choose by workload, not QPS. pgvector, dedicated vector databases, storage-oriented services, and batch SQL engines solve different problems.

What benchmark numbers actually tell you

A published comparative study using SIFT1M—a million-vector benchmark dataset—reported the following single-node results:

These are external baseline figures, not a controlled procurement comparison. The published results are reported measurements from the study’s own hardware, dataset, and index settings; those conditions are not equivalent across products or necessarily representative of current releases. In particular, the table does not normalize CPU model, memory, storage, graph parameters, search parameters, replicas, or network overhead. Treat it as directional evidence, then reproduce the test with your workload.

FAISS, for example, avoids network hops, serialization, transactions, authentication, replication, and failure handling. A database measured through an application process has a different performance profile. Comparing FAISS with a managed service can still be useful, but only if you understand what each number includes.

Dimensionality is another major variable. Results on 384-dimensional embeddings are weak proxies for 1,536-dimensional production vectors. On GIST1M, which uses 960-dimensional vectors, throughput in comparative tests generally fell by two to five times versus lower-dimensional workloads. Distance calculations, memory bandwidth, cache behavior, and index size all become more demanding.

Vendor-reported results show why benchmark provenance matters:

• Google reports AlloyDB ScaNN serving more than 10 billion vectors at up to 95% recall and 51 ms p95 in an internal test.
• AWS says DynamoDB vector search supports up to 4,096 dimensions and can deliver single-digit-millisecond latency at 99% or better recall under specified conditions.
• Amazon S3 Vectors supports up to 2 billion vectors per index. One customer report describes low-hundreds-millisecond average latency across 50 million vectors and 30 parallel processes.
• Databricks examples show ten-million-record similarity joins completed in minutes and workloads involving up to one billion reference vectors.

Those claims may be accurate within their test conditions. They still describe different workloads: interactive RAG, a partitioned key-value lookup, object-backed retrieval, and a distributed batch join aren’t interchangeable.

Filtered recall is the metric that changes the decision

Imagine a shared index with 100 million vectors, of which 100,000 belong to one tenant. An unfiltered ANN search may find excellent global candidates. A post-filter then removes nearly all of them because they belong to other tenants or fail an access-control rule.

The application receives three results instead of ten. Worse, a low result count may look like a data-quality problem when the real issue is search strategy.

Pre-filtering narrows the search space before nearest-neighbor selection. Post-filtering finds approximate neighbors globally and applies metadata conditions afterward. Pre-filtering often makes correctness easier to preserve, but it doesn’t guarantee either recall or low latency: filter execution, candidate expansion, index support, and implementation details determine the outcome. Post-filtering can be fast for broad filters, but selective filters may return too few valid results.

For filtered recall, define a relevance set first. Run exact search within the filtered subset, then compare ANN results with that ground truth:

Filtered recall@k = valid relevant results returned after filtering ÷ valid relevant results in exact search over the same filtered subset

Without an exact-search ground truth for each subset, “filtered recall” is only a rough result-count metric. Test several selectivities, from broad filters that match 50% of the corpus to highly selective filters matching 0.01%.

How to benchmark realistic vector search workloads

A realistic benchmark starts with the embedding model and dimension used in production. If the application stores 1,536-dimensional vectors, don’t substitute a smaller public dataset simply because it runs faster.

Use metadata shaped like the real system: tenant IDs, document types, timestamps, ACLs, languages, and payload fields returned to the application. Then vary the pressure points that expose operational weaknesses:

1. Filter selectivity: 50%, 10%, 1%, 0.1%, and 0.01% of the corpus.
2. Concurrency: one, eight, 32, and 128 simultaneous queries.
3. Concurrent writes: ingest vectors while reads continue, and measure visibility delay.
4. Payload size: compare IDs-only responses with full snippets and metadata.
5. Tenant skew: make a few tenants much larger than the rest.
6. Cache state: measure warm, cold, and post-restart behavior.
7. Failure conditions: remove a node or zone while traffic continues.

Record p50, p95, and p99 latency separately. Also capture timeouts, empty-result rates, filtered recall, CPU, resident memory, index-build duration, and recovery time. A good median can hide a p99 that users experience as a timeout.

Run exact search on a smaller sample to establish ground truth, then compare approximate results against it. Keep the workload and reporting format identical across systems:

python benchmark.py \
  --dataset corpus-10m \
  --dimensions 1536 \
  --queries queries-100k \
  --filters filters/tenant-skew.json \
  --concurrency 32 \
  --top-k 10 \
  --duration 900 \
  --output results/qdrant-filtered.json

Don’t compare default settings and call the result fair. HNSW ef_search, graph connectivity, IVF probes, quantization, shard count, replicas, and consistency settings all change the recall-latency curve. Configure each system to meet the same minimum filtered recall, then compare latency, memory, throughput, and cost.

Memory, freshness, and cost shape the architecture

Raw vector storage is easy to underestimate:

Raw vector memory: number of vectors × dimensions × bytes per dimension

That’s before HNSW links, IVF structures, metadata, replicas, write-ahead logs, compaction space, operating-system memory, and failover capacity. Six terabytes is only the starting point for one billion 1,536-dimensional FP32 vectors that must remain memory-resident.

Freshness is a separate requirement. A product catalog or security-policy system may need new vectors searchable within seconds. A batch corpus may tolerate hours. Measure it directly:

Freshness delay = searchable timestamp − vector commit timestamp

Cost comparisons need the same discipline. A meaningful cost-per-million-queries figure must specify dimensions, filter selectivity, concurrency, result size, latency target, recall target, and freshness requirement. Include compute, storage, replicas, network, ingestion, backups, retries, and maintenance capacity.

The system with the lowest nominal query price may cost more if it times out on 3% of filtered requests or forces the application to maintain a second metadata store.

A concise architecture guide

The products in this space aren’t direct substitutes:

pgvector is often the best first move when avoiding dual writes matters. Vectors and source records share transactions, backups, and permissions. The trade-off appears when ANN queries, index builds, vacuum, and OLTP traffic compete for the same resources.

A dedicated vector database becomes more attractive when retrieval needs independent scaling, index maintenance affects transactional latency, or tenants require separate resource controls. The price is synchronization: you must handle delayed indexing, duplicate writes, and source-database commits that succeed while the vector service is unavailable.

For a multi-tenant RAG system, the practical rule is simple: keep vectors in Postgres while integration dominates and the workload fits comfortably. Move to a dedicated system when vector traffic needs isolation. Favor storage-oriented search when corpus economics matter more than interactive latency. Use lakehouse or distributed SQL engines for batch work, not as a forced replacement for an online retrieval path.

Frequently asked questions

What is the best vector database for multi-tenant RAG?

There’s no universal winner. Start with pgvector when tenant data and application records already live in Postgres; consider Qdrant or another dedicated system when retrieval needs independent scaling and isolation. Require filtered-recall and p99 results using your actual tenant distribution.

Is pre-filtering better than post-filtering?

Pre-filtering is often safer for tenant, permission, and compliance conditions because it limits search to the allowed subset. It still depends on filter execution and index support. Post-filtering can work well for broad filters but may return fewer than k results when the condition is selective.

Can pgvector handle 100 million vectors?

It can, depending on dimensions, index type, memory, update rate, query concurrency, and competing Postgres workloads. At 100 million 768-dimensional FP32 vectors, raw storage alone is about 307 GB; indexes, replicas, metadata, and operating headroom add substantially more.

How should I compare vector-search cost per million queries?

Use the same dimensions, filtered-recall target, p95 or p99 latency, payload size, concurrency, and freshness requirement. Count failed, retried, and over-latent requests as part of the cost rather than dividing only by attempted queries.