RAG Pipeline for SaaS: How to Build It for Many Tenants

The short answer

A RAG pipeline for SaaS providers has the same nine stages as any other: ingest, parse, chunk, embed, index, retrieve, rerank, generate and evaluate. What makes it a SaaS pipeline is that a tenant ID travels through all nine. Documents are ingested per tenant, stored in a per-tenant partition of the index, retrieved only inside that partition, and metered, rate-limited and evaluated per tenant.

The three decisions that shape everything else are the isolation model (how one customer's vectors are kept apart from another's), the permission model (who inside a customer may see which document) and the cost model (what a tenant may consume and what you charge for it). If you are new to the concept itself, start with what RAG is and come back.

The stages of a RAG pipeline, and what multi-tenancy changes

The table lists what each stage does in any AI RAG pipeline and what it has to do in addition once many tenants share it.

For a deeper look at the components themselves, see our guide to RAG architecture. The rest of this article covers the right-hand column.

Tenant isolation in the index: three models

This is the decision that is hardest to change later, because it determines how data is written.

The vendors' own documentation points the same way. Pinecone's multitenancy guide recommends one namespace per tenant, notes that each namespace is stored separately, and warns that with metadata filtering a search still scans the entire namespace regardless of filters, so it costs more and runs slower; offboarding a tenant is deleting its namespace. Weaviate's documentation says each tenant in a multi-tenant collection is stored on a separate shard and that every query must name the tenant. Qdrant's documentation says that creating a separate collection for each tenant is rarely the most efficient approach and reserves it for a limited number of tenants that need strict isolation; for everyone else it recommends one collection partitioned by a tenant field in the payload, with an option to promote large tenants to dedicated shards.

If your vectors live in PostgreSQL with pgvector, the pgvector documentation suggests list partitioning by tenant or separate tables for tenant isolation. PostgreSQL's row-level security can add a database-enforced boundary on top: according to the PostgreSQL documentation, once it is enabled on a table, every row access must be allowed by a policy, and if no policy exists the default is deny. The same page lists the catch: superusers, roles with the BYPASSRLS attribute and, by default, the table owner bypass the policies. If your application connects as the table owner, row-level security is protecting nothing until you force it or switch roles.

Two rules hold for all three models. The tenant ID always comes from the authenticated session on the server, never from a request parameter or from text the model produced. And the boundary has to cover every copy of the data, not only the vectors: the raw files, the parsed text, the keyword index, caches, logs with prompts, and evaluation sets built from customer questions.

Permissions inside a tenant

Tenant isolation stops customer A from seeing customer B. It does nothing about an intern at customer A asking the assistant about board minutes.

The mechanism is the same in most systems. At ingestion you store, with every chunk, who may read the source document: user IDs, group IDs or a reference to an access list. At query time the server resolves the current user's groups and filters retrieval by them before anything reaches the reranker or the model. A prompt instruction such as "only answer from documents the user may see" is not access control.

• Filter before retrieval, not after. If you retrieve ten chunks and then throw away the seven the user can't see, answer quality collapses for users with narrow access.
• Sync permissions on their own schedule. A document's text may not change for a year while its sharing settings change weekly. Treat access changes as events that update chunk metadata without re-embedding.
• Keep answers out of shared caches. A cached answer is a copy of the documents behind it. Cache keys need the tenant and the permission scope, or caching has to stop at the retrieval layer.
• Treat documents as untrusted input. Any user who can upload a file can put instructions in it. Within a tenant that is a prompt injection path from a low-privilege user to a high-privilege one, so the model's output should never trigger actions without checks in your code.

Ingestion and sync at scale

In a single-company project, ingestion is a script you run and watch. In SaaS it is a permanent job system that strangers feed with data.

A new customer connects a large source. Backfills and incremental updates need separate queues, and each tenant needs a concurrency limit, so that one customer importing 200,000 files doesn't make another customer's edited page wait for hours.

Documents change and disappear. Give every chunk a stable ID derived from the tenant, the document and its position, and store a content hash per document. Unchanged documents are skipped, changed ones replace their chunks, and deleted ones are removed from the vector index, the keyword index and the file store together.

You change the pipeline. A better chunking strategy or a new embedding model means re-processing everything. Vectors from different embedding models are not comparable, so a tenant has to be switched whole: build the new partition next to the old one, run that tenant's evaluation set against both, then flip a per-tenant pointer.

Per-tenant cost and rate limits

A RAG feature has four cost drivers, and all four scale with something the tenant controls: embedding tokens at ingestion (document volume and churn), index storage (document volume), reranking and generation tokens (question volume and context size), and re-indexing when you upgrade the pipeline.

So meter from day one. Every model call and every indexing job writes a usage record with tenant, feature, token counts and model. With that data you can enforce limits at three levels:

• Rate limits per user and per tenant, in requests and in tokens per minute, so a script or a looping integration can't exhaust your quota with the model provider.
• Budgets per tenant per month for queries and for indexed volume, with a soft threshold that notifies the customer's admin and a hard one that degrades gracefully: a smaller model, fewer retrieved chunks, or a clear "limit reached" message rather than an error.
• Ingestion quotas on documents, pages or storage per plan, checked before the work is queued, not after the embeddings are paid for.

Our RAG development cost guide breaks down build and running costs in more detail.

Pricing the feature

Once you can see cost per tenant, three pricing models are common, and they can be combined.

The arithmetic is simple and worth doing per plan. Take your measured average cost per answer, including retrieval, reranking and generation, multiply by answers per active user per month, and add the tenant's monthly indexing and storage cost. If that number is more than a small fraction of what the seat earns, the allowance is too generous or the pipeline is too expensive.

Evaluation per tenant

A single quality score across all tenants hides the problem you need to see. Tenant A has clean help-center articles; tenant B has ten years of scanned PDFs and three versions of every policy.

Build evaluation in three layers:

• A shared regression set on synthetic or licensed documents that you own, run on every change to chunking, retrieval, prompts or models. It catches pipeline regressions without touching customer data.
• Per-tenant test sets for larger customers: a few dozen real questions with expected source documents, agreed during onboarding and stored inside the tenant's boundary. Run them before switching that tenant to a new pipeline version.
• Production signals for every tenant: share of questions with no retrieved result above the relevance threshold, share of answers without a citation, thumbs-down rate, and parse failure rate. These need no labeled data and show which tenants are struggling.

Separate retrieval metrics (did the right chunk come back) from answer metrics (was the answer faithful to the chunks). When one tenant scores badly, the cause is usually in how that tenant's documents are parsed or chunked, not in the prompt. Our RAG evaluation guide covers the metrics and tooling, and RAG hallucinations covers the answer side.

Built by Gilzor

Results we’ve shipped

70+products launched

98%delivered on time

85%clients come back

5.0 ★on Clutch

Aviation33%faster baggage weigh-inWeb and Mobile application for Airlines→E-commerce · AITaking an AI-built e-commerce SEO platform to production→Healthcare+30%lab test ordersMedical Lab Web and Native Mobile Applications→

Talk to the people who build it. Tell us about your project and get a free estimate of scope, timeline and cost.

See how we’d approach yours →

A rollout plan

RAG pipeline development for an existing SaaS product goes best in stages, each testing something the previous one couldn't.

1. Choose the isolation and permission modelWrite down the tenant boundary for every data store, how access lists are stored on chunks, and how fast revoked access takes effect.
2. Build the pipeline for one source and your own tenantUse your company's workspace as tenant zero. Carry the tenant ID through every stage and every log line from the first commit, even with one tenant.
3. Add metering and the leak test before the second tenantUsage records per tenant, and an automated cross-tenant retrieval test in the release pipeline.
4. Onboard three to five design partnersPick customers with different document types and sizes. Agree test questions with each.
5. Put the feature behind a per-tenant flag with limitsEnable tenants in cohorts. Set conservative rate limits and budgets, and watch cost per tenant and the production quality signals for each cohort before the next.
6. Set pricing from measured usageUse the design-partner and cohort data to fix allowances, overage and the enterprise isolation tier. Add the usage page for customer admins.
7. Open to all plans and run it as a servicePer-tenant dashboards for ingestion lag, failures, cost and quality; a re-indexing routine; an owner who reviews struggling tenants every week.

FAQ

Where Gilzor fits

Most of a multi-tenant RAG pipeline is product engineering around the model: connectors, permission sync, job queues, metering, billing hooks and admin screens. Gilzor builds custom RAG systems and adds RAG to existing products. The team has launched 70+ products, among them an AI payroll platform that extracts data from PDFs and cut salary calculation errors by 17%. If you are deciding on an isolation model or a pricing approach for your own SaaS, our RAG implementation services start with a 2-week fixed-price Discovery Sprint that tests retrieval on your customers' kinds of documents and scopes the first version. The team works from Poland, with a partial overlap with the US working day.