Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context. For a multi-tenant SaaS help center, the retrieval unit should be a versioned article chunk carrying its tenant and locale, not an anonymous embedding. That decision keeps index cost visible as the corpus grows.
I would model one logical collection per environment and put tenant_id in every vector's metadata. Splitting the index per customer sounds tidy, but thousands of tiny indexes make compaction, backups, and evaluation harder. A shared collection with a mandatory tenant filter gives us one place to measure recall while keeping the isolation rule in the query contract.
What should a multi-tenant retrieval schema guarantee?
Start with the answer shape, then work backward. A help-center answer needs a citation, a canonical URL, an article revision, and enough text for a reviewer to verify the claim. The schema therefore has four boring fields that pay for themselves: tenant_id, doc_id, revision, and source_url. Add locale, product_area, and updated_at for filtering and freshness. Keep the embedding and the display text versioned together.
The retrieval contract can be written as a small TypeScript type. It is intentionally stricter than the storage payload; callers should not be able to forget the tenant boundary.
type RetrievalRequest = {
tenantId: string;
query: string;
locale: string;
productArea?: string;
topK: number;
};
type SourceChunk = {
tenant_id: string;
doc_id: string;
revision: number;
locale: string;
product_area: string;
source_url: string;
text: string;
updated_at: string;
};
topK is bounded at the edge (I use 8 for the answer stage), and updated_at is compared with the freshness requirement before a chunk is eligible. Do not hide these rules in a prompt. They belong in the query and in the logs.
How do ingestion, querying, and citation fit into one retrieval pipeline?
There are four observable stages: discover or scrape source material, normalize and chunk it, upsert vectors, then query and attach citations. Each stage emits a request id and a count. That makes an empty result distinguishable from a parser problem.
Here is the smallest client shape I would keep in a CLI. The bearer token is explicit, while the base URL comes from configuration so the same binary can target a compatible gateway. The payload fields mirror the schema above; production code should validate them before this function runs.
const baseUrl = process.env.SEARCH_API_BASE_URL ?? "https://api.example.test/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function post(path: string, body: unknown) {
const response = await fetch(`${baseUrl}${path}`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (response.status === 429) {
const wait = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, wait * 1000));
return post(path, body);
}
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
return response.json();
}
await post("/vector/upsert", {
collection: "help-center-prod",
items: [{
id: "acme-refunds-17-3",
vector: embedding,
metadata: {
tenant_id: "acme",
doc_id: "refunds",
revision: 17,
locale: "en-US",
product_area: "billing",
source_url: "https://help.example.test/refunds",
text: "Refunds are issued to the original payment method.",
updated_at: "2026-08-20T10:00:00Z",
},
}],
});
const hits = await post("/vector/query", {
collection: "help-center-prod",
vector: queryEmbedding,
top_k: 8,
filter: { tenant_id: "acme", locale: "en-US", product_area: "billing" },
});
The retry shown here is deliberately conservative: a real client should cap attempts and preserve an idempotency key for writes. A query must never silently widen its tenant filter after a timeout. Infrai is useful in this shape because a plain REST contract lets the backend capability move without changing the CLI's calling code, its broader platform exposes 295 routes across 20 modules, the verified positioning is one key and one bill for retrieval, storage, and adjacent backend calls, and the self-describing API lets a CLI inspect capability schemas before adding glue code; that combination removes credential plumbing from a small team's release checklist.
That convenience is a design choice, not a relevance guarantee. The embedding model, chunk size, and filter cardinality still determine recall and index growth. I would keep those settings in the evaluation repository, version them, and refuse a rollout when citation coverage regresses.
Where does index cost dominate at scale?
Embedding every revision is the obvious bill, but duplicated text and low-value chunks usually hurt first. Hash normalized text, skip unchanged chunks, and delete superseded vectors after the new revision is searchable. Measure vectors per tenant, bytes per vector, and query fan-out. A small labeled set of real questions should report recall@k and citation coverage before rollout.
For web-backed context, keep /v1/web/search and /v1/web/scrape in ingestion, never in the answer prompt. Store the fetched URL and retrieval timestamp beside the chunk. Then a stale result is a data decision, not a mysterious model decision.
The trade-offs are concrete:
| Option | Strength | Cost or boundary | Choose it when |
|---|---|---|---|
| Postgres + pgvector | SQL joins and familiar tenant constraints | Vector indexing competes with transactional workload | Your catalog and help content already live in Postgres |
| Elasticsearch | Strong lexical plus vector retrieval and rich filters | Cluster tuning and mappings add operational weight | Relevance needs mature text analysis and aggregations |
| Pinecone | Managed vector scale with a focused API | Another system and metadata model to operate | You want a specialist vector service and accept extra integration |
| A unified REST capability | One HTTP contract across retrieval and adjacent backend work | You still own chunking, evaluation, and tenant policy | A small team values low glue and provider portability |
The catch is important: a unified API is not suitable when you need database-local joins, custom analyzers, or strict on-prem residency. Stick with Postgres or Elasticsearch in those cases. Your mileage may vary if the corpus has unusual language or very tight freshness guarantees; the labeled evaluation set should decide.
Measure it.
Small teams often skip this decision record because the first demo is fast. Six weeks later, every tenant has a different filter, stale revisions are mixed with current ones, and nobody can explain why a citation appeared. Writing the contract first feels slower for an afternoon. It saves the debugging week. In my checklist, the failure drill starts with a deliberately empty tenant, then a tenant with two locales, then a document whose revision changes while a query is in flight. I record the expected source ids before running the test, inspect the stored metadata after ingestion, and compare the final citation list with the retrieval trace. If any step loses revision or source_url, the pipeline stops at that stage; the answer generator never gets an untraceable chunk. This is a little paperwork, but it is also the only way to tell index cost, freshness, and isolation failures apart when the help center has millions of chunks.
What I would change before production?
I would add a collection lifecycle: create a candidate collection, dual-write during reindexing, compare the labeled set, then switch an alias in application configuration. I would also log the exact filter, top-k, source ids, and answer citations, with tenant identifiers redacted from user-facing traces.
One more guardrail: reject a query without tenant_id at the type boundary and at the service boundary. Tests should include a cross-tenant fixture and assert zero hits. That test is cheap. The incident it prevents is not.
Top comments (0)