Tenant isolation for RAG and agents

A pilot runs. One index. One prompt. The second customer arrives. Two files now sit in the same room. Without tenant isolation it stays a one-customer pilot.

What has to be isolated

Not only the UI. The retrieval path:

a filter or index per tenant

no chunks across the boundary, not even "similar"

secrets and keys not in a shared repo default

agent state that does not carry the other tenant along

A shared vector space with hope in metadata filters is the usual finding. The filter is missing once. The answer cites the neighbour.

The filter has to sit in code, not in intent. Tenant comes from the auth context, never from the prompt.

def retrieve(query: str, tenant_id: str, store, embed, k: int = 8):
    if not tenant_id:
        raise ValueError("tenant_id missing")

    hits = store.search(
        embed(query),
        k=k,
        filter={"tenant_id": tenant_id},
    )
    return [h for h in hits if h.metadata.get("tenant_id") == tenant_id]

The second check after the search is deliberate. Metadata filters in Milvus or pgvector can vanish when someone rebuilds the index. Then the second line filters. Without it, the neighbour gets cited.

Two pictures from practice

The food-science workspace was multi-tenant from the start. Otherwise it stays a one-team pilot.

The public-transit platform has 20+ instances. There isolation is operations, not a feature: pipeline and rollback per tenant. The same idea applies to RAG. One index for everyone is the Swarm script for everyone.

What the check does with it

Day 4 of the LLM readiness check is operations, cost, tenants. A pilot without isolation is not a go-live for the second customer.

Related: hybrid RAG, evaluation. Service: AI engineering. For the platform side: Cloud and Kubernetes and the migration assessment.