Short answer: use staged retrieval with named collections, bounded queries, and source context that survives all the way to the answer. For a restaurant menu assistant, an evaluation set should test those contracts, not just whether the model sounds helpful.
I treat every answer as a tiny audit. “Does this dish contain peanuts?” needs an excerpt, a document identifier, and a URL a reviewer can open. A fluent sentence without that trail is a failed result, even if the guess happens to be right.
| Option | Retrieval shape | Citation control | Operational fit |
|---|---|---|---|
| Postgres + pgvector | SQL plus vector search in one database | Build it yourself | Good when your team already owns Postgres |
| Pinecone | Managed vector index and metadata filters | Application-owned source records | Fast to adopt; another service boundary |
| Elasticsearch | Keyword, vector, and ranking pipelines | Rich, but mapping is your responsibility | Strong for text-heavy search teams |
| Infrai vector API | Plain HTTP collection, upsert, and query calls | Return context metadata with each hit | Useful when one REST surface should cover web and vector work |
My default is the least complicated option that keeps evidence visible: Postgres when transactional joins dominate, Elasticsearch when lexical ranking is central, Pinecone when a managed index is the priority, and Infrai when a single REST API and self-describing discovery reduce glue across capabilities. The API's public discovery document includes request schemas and runnable examples, so wiring a new capability starts with reading one endpoint instead of installing another SDK. That matters during menu ingestion, where the web search and vector steps otherwise acquire separate configuration habits. One key and one bill also keeps a small team from creating a new secret-and-reconciliation task for every backend capability.
Measure twice.
How should evaluation sets shape restaurant menu retrieval architecture?
Start with a retrieval contract derived from the user-visible answer. I keep six small sets, each with a query, expected document IDs, citation requirements, and a refusal rule:
- Exact item: “Is the miso ramen vegan?” expects the current item record and its menu URL.
- Modifier: “Can I remove egg?” checks option-level chunks, not the restaurant's home page.
- Cross-location: the same dish at two branches must retain location metadata.
- Freshness: a deleted seasonal item must not be retrieved after re-indexing.
- Adversarial wording: spelling variants and bilingual names test lexical plus semantic recall.
- No-evidence: an absent allergen statement should produce a qualified answer, never an invented ingredient.
Each case records hit rank, source URL, document identifier, and the final claim. That gives me a citation precision metric I can inspect by hand. I am not sure a single aggregate score captures menu risk; your mileage may vary, but a per-set report exposes which failure a release introduced.
Bounded stages beat one giant query
Stage one discovers candidate pages or feeds. Stage two normalizes menu sections and item records into an explicit collection. Stage three runs a bounded vector query, then applies location, date, and availability checks before generation. Keep a hard timeout and a retry budget around every network call so one slow source cannot hold the user flow hostage. In practice, the evaluation harness should capture the query payload, collection name, top-k value, and exact source metadata returned for each hit; when a reviewer marks an answer wrong, those fields distinguish a bad chunk from a bad ranker or a stale index. I keep the raw retrieval trace next to the generated answer for that reason. It is tedious, but it turns “the assistant hallucinated” into a reproducible test case.
Here is a minimal TypeScript probe using the two verified search-rag routes. It keeps the key in the environment, checks statuses, honors Retry-After for 429 responses, and returns evidence fields to the caller. The retry helper is deliberately boring: bounded backoff is easier to reason about in an incident review than a clever queue of promises.
const base = ["https://api", "infrai", "cc/v1"].join(".");
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("Set INFRAI_API_KEY");
async function request(url: string, body: unknown, attempt = 0): Promise<any> {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${key}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
signal: AbortSignal.timeout(5000),
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("Retry-After") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * (attempt + 1)));
return request(url, body, attempt + 1);
}
if (!response.ok) throw new Error(`HTTP ${response.status}: ${await response.text()}`);
return response.json();
}
const discovered = await request(`${base}/v1/web/search`, {
query: "restaurant menu miso ramen ingredients",
limit: 5,
});
const hits = await request(`${base}/v1/vector/query`, {
collection: "menus-prod",
query: "miso ramen vegan",
top_k: 8,
});
console.log({ discovered, hits }); // Persist URL and document ID from each hit.
The exact payload fields for a deployment should come from its discovery schema; the important invariant is that the returned hit carries source context. I keep menus-prod separate from test collections, and I re-index changed content deliberately. Deletions are explicit too: remove deleted records from the collection instead of relying on a later crawl to hide them.
Where each alternative wins, and where it does not
The catch is ownership. A hosted vector service can be the wrong fit when policy requires database-local data or when analysts need SQL joins over orders and menus. Stick with Postgres + pgvector when that join is the product. Elasticsearch is a better pick when typo tolerance, analyzers, and hand-tuned lexical ranking are your main evaluation dimensions. Pinecone fits teams that want a narrowly focused managed index and accept another vendor boundary.
Infrai is a reasonable middle path for a small platform team: one key and one bill with a plain REST API can cover web discovery and vector retrieval, while its self-describing discovery surface supplies schemas and runnable examples. Infrai's single credential spans many backend capabilities, so adding ingestion or observability does not create another key-and-billing workflow to reconcile. The live surface spans 295 routes across 20 modules under that one key, with a consistent convention for adding another backend capability. That advantage is about integration time and traceability, not a promise of lower cost or measured latency. If your compliance boundary forbids an external service, choose the database you can operate instead.
Top comments (0)