What Is Inference Residency?
Inference residency means the actual AI computation – when the model reads your prompt and generates a response – happens in a specific region, for example the EU. Data residency only covers where data is stored at rest. Most residency promises cover storage but not inference, so sensitive information can leave the region every time the model is called.
“All data is stored within the EU.” The sentence shows up in nearly every AI provider’s sales material, and it sounds like the GDPR question is solved. But storage is only half the picture. An AI service doesn’t just store data – it computes. Every prompt is sent to GPU servers running the model, somewhere in the world. Inference residency is about where that computation happens, and that’s exactly where most residency promises go quiet.
Data residency and inference residency – two different promises
Data residency means data at rest – files, databases, conversation history, indexes – sits in a specific region. That’s the promise cloud providers have made for years, and it’s usually what’s meant when someone says data “stays in the EU.”
Inference residency means the actual model run happens in the region: that the servers that read your prompt, hold it in working memory, and generate the response are physically located there. That’s a separate commitment, and it doesn’t automatically follow from the first.
| Term | Covers | Doesn't cover |
|---|---|---|
| Data residency | Storage at rest: files, history, indexes | Where the computation runs |
| Inference residency | The model run: prompt in, answer out | Surrounding services like authentication, routing, and analytics |
Why the distinction decides the GDPR assessment
The prompt is often the most sensitive part of the whole flow – it’s where the customer case, the personnel file, or the draft contract actually exists in plain text. GDPR’s rules on third-country transfers apply to all processing, not just storage. If inference runs in the US, every call is a transfer, regardless of whether anything is “saved” there.
A concrete example: an HR department uses an AI assistant to summarize rehabilitation cases. The provider promises EU storage, and the archive is indeed located in an EU region. But the model runs in the US. So every case – names, health information, assessments – passes through American servers the same second someone hits enter. From a GDPR perspective, that requires a legal basis, safeguards, and documentation, exactly as if the files had been placed on an American server.
Two examples showing how easily it goes wrong
The difference between storage and inference isn’t theoretical – it’s visible in the vendor landscape right now:
- OpenAI introduced in-region GPU inference for new EU tenants only in January 2026. Even with that in place, authentication, routing, indexing, and analytics can sit outside the region – residency covers the core flow, not necessarily the surrounding services.
- Claude via Azure AI Foundry runs on Anthropic’s infrastructure, even though the portal shows an EU region. The region selection in the interface describes where the resource sits – not where the model actually runs.
The point isn’t to single out individual providers – both offer other paths that keep data in the EU. The point is that an EU flag in a cloud portal isn’t proof of EU inference.
The rule: residency is configuration, not a product name
The same provider can give completely different residency depending on product, SKU, pricing tier, deployment type, and model version. So you can’t just “pick a GDPR-safe provider” and be done. What holds up is a written statement, per exact configuration, covering three legs:
- Storage – where data at rest sits, including history and indexes.
- Transit – which networks and nodes the traffic passes through on the way.
- Inference – where the model actually runs when the call is processed.
Demand that statement for exactly the SKU, tier, and model you intend to use. And request a new one at every model upgrade – a new version can have different conditions than the one you approved.
The wording in the contract matters too: don’t ask for a general assurance, ask for an appendix that names the model version, region, and which surrounding services are included. A provider that’s serious about its residency promise has no problem specifying it – hesitation about specifying it is itself an answer.
The questions to bring to the vendor meeting
Three questions that quickly reveal how well thought-out the residency promise is:
- “Where does inference happen for exactly the model and tier we’re buying – and can we get that in writing?”
- “Which surrounding services – authentication, routing, indexing, analytics – sit outside the region?”
- “What happens to the residency when you upgrade the model?”
A clear, written answer to all three is a good sign. A vague “all data is in the EU” is a reason to dig further. At Weapp we help companies weigh residency requirements against architecture choices when AI solutions are being introduced – get in touch if you’d like to talk through your landscape.
Frequently asked questions
Is it enough if the provider promises all data is stored in the EU?
No. Data storage only covers data at rest – files, history, and indexes. The computation itself can still happen on servers outside the EU, and then the information has left the region with every call. Always ask for confirmation covering storage, transit, and inference alike.
Is inference residency a GDPR requirement?
GDPR doesn't use the term, but the rules on third-country transfers apply to all processing – including what happens during the model run itself. If inference runs outside the EU, that's a transfer requiring a legal basis, safeguards, and documentation.
How do I find out where inference actually runs?
Ask the provider in writing, for exactly the SKU, tier, and model you intend to use. Public documentation is a start but changes on an ongoing basis and rarely covers the surrounding services. A dated, written statement is the only thing that holds up in an audit.
What does transit mean in this context?
Transit is the path data travels between you and the model – which networks and nodes the traffic passes through. Even with EU storage and EU inference, authentication and routing can go via systems outside the region, so ask the provider to account for all three legs separately.
Does the problem also apply to AI features in ordinary SaaS tools?
Yes. Many SaaS tools in turn call an external model provider, and then you inherit that chain's residency conditions. Map the subprocessor chain for every tool that processes personal data, not just for your own AI projects.