GDPR and data residency once an LLM is in the loop

A lawyer checks whether a processing activity is lawful. I build the system that makes the answer technically possible in the first place. This piece is the second part, not the first. For the legal assessment you need someone licensed for that, not an architect.

What I see when an LLM prototype meets a real privacy review: the questions are almost always the same, and the answers are almost always missing, because nobody asked during the prototype phase.

The questions that come first

Where does the model physically run? Which region, which provider?

Where do the embeddings and the chunks that go into the model as context live?

Does a request ever leave the EU, even for an intermediate step like reranking?

Who has access to logs that might contain personal data from the prompt?

Every one of these questions has a technical answer, not a legal one. But without the technical answer, legal cannot answer their own question.

Where the answer usually stalls

Azure OpenAI in an EU region is a different case from OpenAI directly, and both are different from a self-hosted model. That is not a minor detail. A prototype built on the public API because it was the fastest path often needs an infrastructure change before rollout, not just a contract amendment.

For a vector store the question is just as concrete. pgvector on your own Postgres instance in the EU is a different answer than a hosted vector service with a data center outside the EU. Both work identically from a technical standpoint. The difference is entirely about where the data sits, not how the search runs.

I have solved this in projects with Azure OpenAI and pgvector, where both components ran in an EU region because the data could not leave the regulated environment. Where a hosted EU region was not enough, the alternative was self-hosted vectors instead of an external service.

What often gets missed

Logs are data too, and that gets missed often. A prompt that contains a personal request often lands unfiltered in the observability tool. If that tool sits outside the EU, the request left the region even if the main model stayed inside it. Logging infrastructure regularly gets less scrutiny in reviews than the model itself, even though it carries the same data.

Reranking and other intermediate steps count just as much. A reranker model running through an external API sees the same chunks as the main model. If the main call stays in the EU but the reranker does not, the chain breaks at its weakest link.

Multi-tenant sharpens the question further. On a platform with several tenants, "the data is in the EU" is not a sufficient statement on its own. It has to be clear which data from which tenant sits where, and that one tenant cannot reach another's data through a shared log stream. That is the same separation as in tenant isolation for RAG and agents, just triggered by privacy instead of security.

Deletion is not a simple delete in RAG

An access or deletion request touches one row in a table for a classic system. For a RAG system it touches a document that was split into chunks, each chunk with its own embedding in the vector store, possibly also present in a cache or in log entries from past answers. Deleting a document from the source system is not enough if the embeddings stay in the index and keep getting cited in answers.

That is why I build a deletion pipeline that traces back from the source document, through its ID, to every derived chunk, not just forward from the document. Without that back-reference, a deletion in the source system is an illusion as long as the copy in the vector store stays untouched.

Third countries and data processing agreements

The moment a provider outside the EU is involved, the question of the transfer mechanism comes up. I list which provider has a data processing agreement in place, which region is contractually guaranteed there, and whether that guarantee also covers intermediate steps like the provider's own error logs. This is again a technical and contractual inventory, not a statement on whether a given setup is legally permitted. That assessment sits with your data protection officer or your legal team, not with me.

What I put into a review for this

I list every component that touches data: model, vector store, reranker, logging, cache. For each component, the region, the provider, and whether a data processing agreement exists. This is a technical list, not a legal assessment. But it is the foundation on which the legal assessment can actually happen.

Without this list, legal gets a statement like "we use a GDPR-compliant model" that says nothing about the actual data chain. With the list, they get something they can actually check.

What the check does with it

In the LLM readiness check this shows up across several days, not just one: in retrieval, where the chunks sit, in operations, where logs land, in tenants, whether the isolation also covers the data location. I deliver the technical list. The legal assessment stays with your legal team or your lawyer.

The platform side of this, when infrastructure has to stay in the EU, sits under cloud and Kubernetes platforms. The service for the LLM side: AI engineering. Related: tenant isolation.