September 22, 2026

Estimating Retrieval Capacity from Storage and Query Demand

Build a defensible capacity workload model for enterprise retrieval by breaking down storage query cost drivers, ingestion spikes, and compute scaling.

Retrieval capacity depends on how documents are stored and updated as well as how users and applications query them. Estimating system requirements requires moving beyond raw corpus size or vendor node tiers to model four distinct operational dimensions: persistent storage, resident working memory, ingestion/update throughput, and query concurrency under target response thresholds.

Key takeaways

• Four Independent Capacity Dimensions: Corpus storage in gigabytes, vector counts, query rates (QPS), and concurrent active requests measure entirely different system resources. Storage volume alone cannot predict CPU or GPU query bottlenecks.
• Decoupled Disk and Memory Planning: Raw vector coordinate payloads represent only a fraction of total disk storage. Index structures, metadata indexes, replicas, backups, and temporary compaction space dictate disk footprint, while quantization and caching choices affect resident memory needs.
• Queueing Science for Concurrency: Using Little's Law (average requests in system = arrival rate x mean response time), average in-flight retrieval work is governed by stable completed throughput and mean queue-inclusive stage latency.
• Update and Ingestion Competition: Ingestion and re-indexing may compete with queries when they share compute, memory bandwidth or storage resources. Contention and locking depend on the database engine and deployment architecture; measure their effect on source-to-search freshness and query latency.

Capacity Worksheets and the Four Sizing Dimensions

Sizing enterprise retrieval-augmented generation (RAG) platforms requires separating persistent storage metrics from compute, memory, and concurrency metrics. Planning hardware or managed service tiers based solely on raw document gigabytes frequently leads to memory starvation during query traffic bursts or severe latency spikes during background document updates.

A defensible workload model evaluates four distinct capacity dimensions:

1. Persistent Storage (Disk/Object): Original source files, parsed text payloads, structured metadata indexes, database headers, durable backups, and temporary rebuild or compaction space.
2. Resident Working Memory (RAM/Cache): Vector index structures, metadata filter caches, and query execution workspaces.
3. Ingestion and Update Throughput: Document parsing pipelines, embedding model API or GPU processing rates, vector index modifications, and source-to-search freshness lag.
4. Retrieval Concurrency and Latency: Offered query rate (QPS), concurrent in-flight requests, metadata filter selectivity, and buyer-defined p50, p95, and p99 latency targets.

Distinct physical units govern these dimensions. User count, document count, raw corpus gigabytes, vector count, queries per second (QPS), and concurrent requests cannot be interchanged. Furthermore, vector retrieval capacity covers only the document search stage; downstream reranking models and LLM answer generation demand separate compute budgets.

Worked Example: Hypothetical Vector Storage and Coordinate Payload Math

To establish a baseline capacity calculation, consider an explicit hypothetical enterprise corpus with an assumed chunking distribution:

• Corpus Size: 100,000 source documents (hypothetical scenario).
• Assumed Chunking Ratio: Assumed sample mean of 10 chunks per document (after accounting for document sampling across text types, OCR density, tables, and chunk overlap).
• Vector Count: 100,000 documents x 10 chunks/doc = 1,000,000 vectors (assuming one embedding per chunk).
• Vector Representation: 1,536-dimensional dense vectors stored as standard 32-bit floating-point numbers (float32, 4 bytes per dimension).

Raw Coordinate Math

Raw coordinate payload = 1,000,000 vectors x 1,536 dimensions x 4 bytes = 6,144,000,000 bytes

In standard units, 6,144,000,000 bytes equals 6.144 GB (decimal) or approximately 5.72 GiB (binary). This figure represents strictly the uncompressed raw vector coordinate payload.

Sampling, Chunking Variations, and Overhead Disclaimers

Chunk counts cannot be inferred directly from compressed file sizes (such as PDF megabytes). Dense PDFs containing text scans, complex tables, or images require optical character recognition (OCR) and layout analysis, which alter chunk yield significantly compared to plain text. Furthermore, retaining historical chunk versions, maintaining multiple embeddings per chunk (such as combining dense semantic vectors with sparse lexical vectors), or handling re-embedding peaks during embedding model updates multiplies storage requirements.

Database engines also append per-vector headers and index metadata. For example, the official pgvector documentation notes that its vector storage format uses 4 * dimensions + 8 bytes per vector value. For a 1,536-dimensional vector, this adds an 8-byte internal header per vector coordinate array (6,152 bytes total per array). This 8-byte header is specific to pgvector's internal array representation; it excludes Postgres database tuple/row headers, page overhead, and vector index structures. This pgvector calculation is an illustrative example of database storage overhead and does not imply that Seahorse Cloud uses pgvector.

The Storage and Memory Ledger

A production vector storage architecture requires a comprehensive storage ledger that explicitly separates persistent disk capacity from resident RAM.

Storage Ledger Components

1. Source Documents and Extracted Text: Raw PDF/HTML source archives alongside parsed text chunks and metadata attributes.
2. Raw Vector Payloads: Uncompressed vector coordinate arrays (float32 byte arrays).
3. Index and Metadata Structures: Graph edge lists (such as HNSW neighbors), centroid directories, inverted list metadata indexes, and scalar payload fields.
4. Active Replicas: Additional node allocations for read availability. Adding one complete same-format read replica creates 2 active copies of replicated vector data and index structures, but does not double persistent object storage source archives or isolated disaster recovery backups.
5. Backups and Retained Versions: Durable point-in-time snapshots, historical chunk revisions, and write-ahead logs.
6. Temporary Rebuild and Compaction Space: Headroom required during background index builds, graph compaction, or corpus re-embedding.

Disk vs. Resident RAM Planning

Not all vector data must reside in memory, and system latency depends on measured index strategies and cache states rather than universal memory rules. While memory-resident indexes avoid disk I/O during graph traversals, modern platforms employ memory-mapped files (mmap), tiered storage, and vector quantization to balance cost and performance. Universal rules of thumb (such as assuming index RAM must always be a fixed multiplier of raw vector size) obscure actual workload behavior.

Quantization techniques reduce memory footprint by converting high-precision coordinates:

• Scalar Quantization (SQ8): Converts 32-bit floating-point coordinates to 8-bit integers (int8). This reduces the raw coordinate payload from 4 bytes to 1 byte per dimension (a 75% reduction in raw coordinate bytes), but does not reduce metadata, index graph overhead, or total system cost by 75%. Vector quality (ANN recall@k) and overall memory footprint must be measured under representative filter workloads.

Replicas, Shards, and Backups

• Shards: Horizontal partitions that split a vector dataset across nodes to scale storage capacity and index-building compute.
• Replicas: Duplicate instances of a dataset created to increase query throughput and provide failover availability. Replicas do not scale read throughput linearly due to load-balancing overhead, cache warmth differences, and coordination. High-availability configurations may use primary-replica or peer-to-peer replica topologies.
• Backups: Retained point-in-time copies support recovery and other offline uses. They do not ordinarily participate in live query serving or read scaling.

Query Demand, Queueing Science, and Little's Law

Query demand must be modeled using queueing science rather than naive concurrency estimates. System capacity depends on both the stable accepted query arrival rate and the average response time spent across execution queues and vector calculation stages.

Hypothetical Queueing Calculation

Consider a hypothetical enterprise application generating retrieval query load:

• Application Load: 30 application requests per second (hypothetical scenario).
• Retrieval Fan-out: 2 retrieval calls per application request (for example, searching dense semantic and metadata-filtered collections separately).
• Total Offered Retrieval Rate: 30 requests/s x 2 = 60 retrieval requests/second (prior to retries).
• Assumed Mean Retrieval Stage Time: Assumed mean response time of 0.20 seconds (200 ms) across the retrieval stage, including queue wait time and vector distance computation.

Assuming the system sustains 60 requests/second without request losses or queue backlog growth, Little's Law from queueing theory (see MIT Queueing Models Lecture) defines average in-flight work:

Average in-flight retrieval requests = 60 requests/sec x 0.20 sec = 12 requests

Critical Queueing Interpretations and Caveats

• What 12 Means: The value 12 represents the steady-state average number of requests in the system (active execution plus waiting in queue) under stable, sustained conditions.
• What 12 Does Not Mean: 12 is not worker thread pool capacity, maximum system throughput capacity, or proof of stability during traffic bursts.
• Configured Limits vs. In-Flight Concurrency: Configured limits cap work admitted to a specified stage or resource pool. Depending on implementation, a limit may cover active work or active plus queued work. Little’s Law describes average in-flight work within a defined system boundary under stable throughput; it does not determine a safe configured limit.
• Do Not Substitute Tail Latency: Little's Law applies strictly to long-term average arrival rates and average stage residence times. Substituting tail latencies (such as p95 or p99) into Little's Law produces mathematically invalid concurrency figures. Note that p50 represents median latency, while p95 and p99 measure tail latency percentiles. Keep latency percentiles separate when defining performance targets.
• Burst and Fan-out Effects: Peak query bursts, multi-tenant retry storms, and cold cache query variations create transient queues. Sizing must account for arrival spikes and cache state variations under production query loads.

Ingestion Stress and Source-to-Search Freshness

Corpus updates generate sustained compute and write stress that directly competes with active search queries for hardware resources.

Hypothetical Update Rate Calculation

• Document Change Rate: 1,000 changed documents per hour (hypothetical scenario; net additions and deletions modeled separately).
• Replacement Yield: Assuming 10 replacement chunks per changed document.
• Total Embedding Rate: 1,000 changed docs/hr x 10 replacement chunks/doc = 10,000 replacement embeddings/hour.
• Average Throughput: 10,000 / 3,600 seconds = 2.78 replacement embeddings/second.

Operational Impact of Updates

1. Additions, Deletions, and Replacement: Replacing modified documents requires versioned replacement and background cleanup according to the database engine's concurrency model, rather than universally requiring atomic write locks or deleting old chunks prior to new insertions (which can cause search availability gaps).
2. Backlog Spikes and Rate Limits: Sustained batch ingest spikes can exhaust document parser workers or hit embedding API rate limits, causing ingestion queue buildup.
3. Freshness Lag, Caches, and Revocation: Source-to-search freshness measures the time elapsed between writing a source update and its visibility in vector search results. Updating active collections requires explicit verification of stale chunks, retrieval query cache invalidation, and access control list (ACL) revocation lag under concurrent query load.
4. No Instant Synchronization Guarantee: Integrated document pipelines simplify architectural setup, but automated integration does not guarantee zero-lag index synchronization or automatic access revocation without explicit operational verification.

Technical Evidence: Metadata Filtering and Search Selectivity

Metadata filtering narrows search spaces but can affect vector index traversal efficiency and recall.

Filtering Mechanics and the pgvector HNSW Example

In vector databases utilizing HNSW graph indexes, applying restrictive metadata filters during or after graph traversal affects match yields. The official pgvector documentation provides an implementation-specific illustration: using default HNSW search breadth (ef_search = 40) with a metadata filter that matches only 10% of rows (0.10 selectivity) yields an average of approximately 4 returned matches:

In the documented pgvector illustration, default ef_search = 40 with a filter matching 10% of rows gives about 4 matching rows on average. This example is not a general prediction for arbitrary top-k settings, data distributions or filtering strategies.

Note that search breadth (ef_search) controls candidate search breadth during graph traversal, rather than graph traversal depth. This result illustrates how post-filtering on restrictive attributes reduces returned matches unless search parameters are increased or iterative scanning is performed (pgvector implements iterative HNSW scans up to documented scan limits). This is an illustrative pgvector operational detail, not a universal mathematical law, benchmark standard, or claim regarding Seahorse Cloud.

Benchmark Verification Requirements

Platform teams must evaluate filtering performance across distinct variables:

• Top-k: The requested number of final nearest-neighbor results returned to the application.
• Search Breadth (ef_search): The candidate pool parameter controlling search breadth during graph traversal.
• Filter Selectivity: The proportion of corpus records matching metadata predicates (for example, tenant ID or date range).
• ANN Recall@k: Approximate nearest neighbor recall evaluated against exact nearest neighbors within the same authorized filtered subset. Downstream answer relevance is evaluated separately.

When exact identifier lookups or metadata attributes are required, systems may utilize structured filters, keyword search, or hybrid retrieval (combining lexical and vector search).

Load-Test Acceptance Matrix and Cost Framework

System capacity must be verified through load testing using representative enterprise datasets and query distributions. Performance thresholds represent buyer-defined target objectives or service level objectives (SLOs), not pre-tested vendor guarantees.

Load-Test Acceptance Matrix

Unit Cost per Successful Retrieval

To evaluate infrastructure efficiency, calculate the total cost per successful retrieval request over a defined billing or operational period:

Total retrieval-stage cost = Allocated Storage + Indexing/Embedding Compute + Retrieval Runtime + Observability + Operations Labor

Cost per Successful Retrieval = Total retrieval-stage cost / Total unique completed successful retrieval requests

Cost Calculation Rules:

• Numerator Accounting: The numerator includes all allocated storage, indexing/embedding compute, retrieval runtime, observability, and operations labor spent during the period, including the costs incurred by failed or retried work within those categories.
• Denominator Accounting: The denominator counts unique completed logical retrieval requests that meet predefined quality and latency criteria. Count a request that succeeds after retries once.
• Undefined Ratio Condition: If zero retrieval requests complete successfully during the period, the unit cost ratio is undefined; report total spend and zero completed requests.
• Cost Boundaries: Declare the query embedding and reranking boundaries clearly, and explicitly separate retrieval-stage costs from downstream LLM answer-generation costs (context token processing and generation inference compute).
• Safety Headroom: Derive safety headroom and recovery margins from tested saturation points, anticipated growth, traffic bursts, and failure recovery times, rather than using unverified percentage rules of thumb.

Implementation Note: Seahorse Cloud Architecture

Seahorse Cloud provides an integrated managed platform for enterprise retrieval and agent orchestration. Public product documentation describes its document, vector database and agent capabilities (https://seahorse.dnotitia.ai/en/product/); the homepage also describes cross-session context memory for MCP agents (https://seahorse.dnotitia.ai/en/).

• Integrated Storage and Vector Synchronization: Combines S3-compatible persistent object storage with automated document parsing, semantic chunking, and vector database table synchronization.
• Managed Vector Database Tier: Features structured table and schema management, performance and capacity monitoring, multi-tenant isolation, and API-key authentication.
• AgentOps and Context Memory: Dedicated AgentOps capabilities manage AI agent lifecycles, Model Context Protocol (MCP) tool calls, inference APIs, and cross-session context memory.
• Deployment Boundaries: Supports cloud SaaS and on-premises deployments.

Technical Boundaries and Verification Principles

1. Authentication vs. Authorization: API-key authentication verifies client credentials but does not inherently establish fine-grained, row-level or document-level access control without policy engine enforcement.
2. MCP Security Execution: Model Context Protocol (MCP) standardizes agent tool integration, but explicit authorization policies must be verified across agent, tool, and data access layers.
3. Deployment Benchmarking: While Seahorse Cloud supports cloud SaaS and on-premises environments, operational capacity and latency depend on local hardware, network topology, and database configuration. Deployment targets must be benchmarked in the target environment rather than assumed to yield identical results across hosting models.

Frequently Asked Questions

How does persistent storage capacity differ from overall retrieval capacity?

Persistent storage capacity measures the total disk space required for raw source documents, parsed text chunks, vector coordinates, index structures, active replicas, and backups. Overall retrieval capacity measures the system's ability to process query arrival rates (QPS) and document updates within target latency thresholds, governed by CPU/GPU compute, memory bandwidth, and queueing dynamics.

Must vector indexes and chunk text reside entirely in RAM?

No. System latency depends on measured index strategies and cache states. Modern retrieval platforms use vector quantization (such as SQ8), memory-mapped files (mmap), and tiered storage to store chunk text and secondary indexes on high-speed disk, reducing RAM footprints.

What is the difference between query QPS, in-flight concurrency, and configured concurrency limits?

QPS (queries per second) measures a request rate; distinguish offered traffic from completed throughput. In-flight concurrency is the number of active and queued requests at a given time. Little's Law relates its long-term average to throughput and mean response time under stable conditions. Configured concurrency limits cap work admitted to a particular execution stage or resource pool; they do not establish sustainable capacity without load testing.

How do background document updates affect active query performance?

Background updates consume parsing, embedding, and indexing resources that may compete with queries. Locking, version replacement, cache invalidation, and cleanup depend on the implementation. Benchmark update bursts alongside queries and measure freshness, deletion visibility, and latency.

How should enterprise buyers validate a platform vendor's capacity sizing proposal?

Buyers should conduct a benchmark using a representative sample of their actual document corpus, vector dimensions, expected query arrival patterns, and metadata filters. Record completed QPS, median (p50) and tail latencies (p95, p99), ANN recall@k against exact filtered nearest neighbors, indexing lag, and resource usage during mixed query and ingestion loads rather than relying on unverified vendor tier sizing.