Multi-tenant AI systems must guarantee that Tenant A's data never appears in Tenant B's responses — not in the LLM context, not in the vector store, not in the cache. A single isolation failure creates a cross-tenant data breach.
Select an isolation layer to see its failure mode and secure implementation:
Failure mode: Missing tenant filter → cross-tenant document retrieval
import chromadb
chroma = chromadb.HttpClient(host="localhost", port=8000)
def get_tenant_collection(tenant_id: str):
"""One collection per tenant — hard physical separation."""
return chroma.get_or_create_collection(
name=f"tenant_{tenant_id}",
metadata={"tenant_id": tenant_id},
)
def retrieve_for_tenant(tenant_id: str, query: str, n: int = 5):
collection = get_tenant_collection(tenant_id)
return collection.query(
query_texts=[query],
n_results=n,
where={"tenant_id": tenant_id}, # Double filter — defense in depth
)["documents"][0]
# NEVER query a shared collection without a tenant_id filter:
# WRONG: shared_collection.query(query_texts=[query])
# RIGHT: get_tenant_collection(tenant_id).query(...)| Layer | Isolation Mechanism | Failure Mode |
|---|---|---|
| Vector Store | Namespace/collection per tenant, filter all queries by tenant_id | Missing filter → cross-tenant document retrieval |
| Conversation History | Thread ID scoped to user, never shared across tenants | Shared thread → context bleed between users |
| Semantic Cache | Cache key includes tenant_id hash | Shared cache → Tenant B gets Tenant A's cached response |
| LLM Context | System prompt asserts tenant scope, user data tagged with tenant_id | Stale context → data from previous tenant's turn appears |
| Logs | Tenant ID on every log line, RBAC on log access | Shared logs → operational staff can read other tenants' queries |
import chromadb
chroma = chromadb.HttpClient(host="localhost", port=8000)
def get_tenant_collection(tenant_id: str):
"""One collection per tenant — hard isolation."""
return chroma.get_or_create_collection(
name=f"tenant_{tenant_id}", # Physical separation
metadata={"tenant_id": tenant_id},
)
def retrieve_for_tenant(tenant_id: str, query: str, n_results: int = 5) -> list[str]:
collection = get_tenant_collection(tenant_id)
results = collection.query(
query_texts=[query],
n_results=n_results,
where={"tenant_id": tenant_id}, # Double filter — defense in depth
)
return results["documents"][0]
def ingest_for_tenant(tenant_id: str, docs: list[str], ids: list[str]) -> None:
collection = get_tenant_collection(tenant_id)
collection.add(
ids=ids,
documents=docs,
metadatas=[{"tenant_id": tenant_id} for _ in docs],
)import anthropic
from pydantic import BaseModel
from typing import Literal
client = anthropic.Anthropic()
class TenantContext(BaseModel):
tenant_id: str
user_id: str
tier: Literal["free", "pro", "enterprise"]
def build_system_prompt(ctx: TenantContext) -> str:
"""Scoped system prompt — prevents cross-tenant context pollution."""
return f"""You are an AI assistant for tenant {ctx.tenant_id}.
You ONLY have access to information belonging to this tenant.
User tier: {ctx.tier}
You MUST NOT reference data, users, or context from any other organization.
If you are unsure whether information belongs to this tenant, do not include it."""
def tenant_chat(ctx: TenantContext, messages: list[dict], user_query: str) -> str:
# Retrieve only this tenant's documents
retrieved = retrieve_for_tenant(ctx.tenant_id, user_query)
context_block = "\n\n".join(retrieved)
# Tenant-scoped system prompt
system = build_system_prompt(ctx)
response = client.messages.create(
model="claude-haiku-4-5-20251001",
max_tokens=500,
system=system,
messages=messages + [{
"role": "user",
"content": f"Context (tenant {ctx.tenant_id} only):\n{context_block}\n\nQuery: {user_query}",
}],
)
return response.content[0].textimport hashlib
def make_cache_key(tenant_id: str, query: str) -> str:
"""
Cache key MUST include tenant_id — otherwise Tenant A's cached response
would be returned to Tenant B for the same query.
"""
tenant_hash = hashlib.sha256(tenant_id.encode()).hexdigest()[:12]
return f"cache:{tenant_hash}:{hashlib.md5(query.encode()).hexdigest()}"
# WRONG — never use query alone as cache key in multi-tenant systems:
# bad_key = hashlib.md5(query.encode()).hexdigest()
# RIGHT — always scope by tenant:
good_key = make_cache_key(ctx.tenant_id, user_query)Write a cross-tenant penetration test.Create two tenants (A and B). Ingest unique documents for each. Query tenant B's assistant asking about tenant A's documents by name. The correct response is denial of knowledge — not retrieval. Run this test in CI on every release.
LLM context windows have no access control.If two tenants' data is concatenated into the same context window — even accidentally — the model will use all of it to answer. Physical separation (separate collections, separate threads) is the only reliable guarantee.
Lesson 5 covers the compliance and governance layer: GDPR right-to-erasure for vector stores, HIPAA considerations, and AI-specific audit logging requirements.