Choosing a Cloud for AI Workloads: The Comparison That Actually Matters
By Maplecode
Most cloud-for-AI comparisons are a feature grid: which provider has which accelerator, which managed vector store, which model garden. The grid is accurate for about a quarter. Then someone ships a new instance type and the ranking inverts.
That is a bad basis for a decision you will live with for five years. The questions worth arguing about are slower moving, and none of them are about model quality.
Data gravity decides more than model quality
Training and inference are elastic. Your data is not. If eight years of transactions sit in a warehouse on one provider, the practical choice for anything that reads that data at volume was made years ago by whoever picked the warehouse.
You can move the data. Budget properly for it: egress is charged per gigabyte, a full historical export of a mature estate runs to tens of terabytes, and the meter runs again on every re-sync while you operate in parallel. The cost that hurts is rarely the one-off migration. It is the eighteen months of dual-running while the old pipeline is decommissioned.
The honest framing: pick the cloud your data already lives on unless you have a specific, costed reason not to. "Their AI services look better" is not a costed reason.
The model layer is the least sticky part of the stack
This is the inversion most architecture decks miss. Two years ago the model was the differentiator. Now the frontier labs ship on multiple clouds, open-weight models run anywhere with a GPU, and the interface has converged on something close enough to a common shape that swapping providers is a config change plus an evaluation run.
What is sticky is everything around it: identity, networking, the data platform, your IaC, the compliance posture your auditors already accepted. Those take quarters to move. The model takes an afternoon and a regression suite.
So do not couple hard to the model layer, and do not choose a cloud because of it. Put a thin interface between your application and whatever is generating tokens, keep prompts and evaluations in your own repository, and treat the provider as replaceable — because it is.
Where the differences are real
Three places, in our experience, produce genuinely different engineering:
- Accelerator availability in your region. Not the published catalogue — actual quota, in the region your data residency rules allow. This varies enormously and it is the constraint that most often forces a redesign late.
- Identity and network integration. If your organisation runs on one directory, the provider that federates cleanly with it saves a category of work that never appears in a comparison table and never stops costing you if you get it wrong.
- The contract, not the console. Committed-spend discounts, support tiers and the data-processing terms your legal team will accept. These decide the bill more than instance pricing does.
The failure mode: optimising for the demo
The pattern we see most often is a proof of concept built on whichever provider's managed service made the demo fastest, then promoted to production because it already worked. Six months later the team is running one workload on a second cloud, with a second identity model, a second set of network rules and a second on-call runbook, to serve a feature that could have run anywhere.
Multi-cloud is a legitimate strategy when it is chosen. It is expensive when it accretes. The tell is that nobody can say what would have to be true for the second cloud to be shut down.
When we would advise against moving
If your AI workload is inference against a model you do not host, on data volumes measured in gigabytes rather than terabytes, the cloud choice barely matters and moving is not worth the disruption. Run it where your engineers already have credentials, alerting and a deployment pipeline. The operational familiarity is worth more than any pricing delta at that scale.
Reconsider when one of three things changes: you start fine-tuning on proprietary data, inference volume makes the accelerator bill a line item somebody asks about, or a data-residency obligation lands that your current provider cannot meet in the region you need.
How we run the decision
We treat it as a two-week exercise, not a strategy engagement. Establish where the data actually is and what it would cost to move. Confirm accelerator quota in the specific regions your compliance position allows, in writing, from the provider. Build the same workload twice, thinly, and measure the parts that surprised you rather than the parts the vendor benchmarked.
Then write down the reversal conditions: what would have to be true for this to be the wrong call. A decision nobody can describe how to unwind is a decision that will be defended long after it stops making sense.
Who makes this call is part of the problem
Cloud selection for AI usually lands between two groups with different incentives. The platform team owns the bill, the security posture and the 3am page, so it favours the provider already running. The product team owns a delivery date and favours whichever managed service removes the most work this quarter.
Both positions are rational and neither is complete. What resolves it is making the trade explicit: the product team names the weeks saved, the platform team names the ongoing cost of a second operating model, and someone with budget authority decides which is worth more. Left implicit, the decision goes to whoever ships first, which is how organisations acquire a second cloud nobody chose.
Write the outcome down with a date and a named owner. Cloud decisions get relitigated every time a bill arrives, and the absence of a record means the argument restarts from zero each time.
If you are working through this, our AI engineering and cloud modernization practices tend to run it together, because the answer is usually constrained by the platform rather than by the model.
