Concrete AI: GPU workloads on Exoscale with residency and cost control
The Concrete AI domain tests whether you can stand up AI workloads on Exoscale's GPU compute and data services, and whether you keep the same residency, security and cost discipline you would apply anywhere else.
Building blocks
AI workloads assemble from parts you already know: GPU instance types for training and inference, templates with drivers and CUDA preinstalled, block volumes for checkpoints and model weights, Object Storage for datasets and artefacts, private networks for multi-node traffic, and the NLB in front of inference endpoints. There is no magic: if the GPU type is not offered in your zone, the workload cannot run there - and residency requirements may then force a different design.
Data and model hygiene
Training data is often the most sensitive asset you hold. Keep it in a bucket in the zone you are allowed to use, restrict reads with bucket policies and IAM, and avoid copying it onto laptops. Model weights and checkpoints are large and stateful: put them on a block volume or push them to object storage, because local instance storage does not survive termination. Document provenance and lineage - it is both a quality and a compliance artefact.
Serving and cost
For inference, prefer a pool of GPU instances behind the NLB with health checks and a concurrency limit, so one stuck request does not take the endpoint down. GPUs bill by the hour whether busy or not: stop idle instances, snapshot the volume, and restart from the snapshot. Autoscaling on queue depth or latency beats static overprovisioning.
Worked example
A team fine-tunes a 7B-parameter model monthly and serves it to an internal app. Reasoning: curate the dataset into an Object Storage bucket in the approved zone; run a GPU instance with a block volume mounted for checkpoints, snapshotting after each successful run; push final weights to a versioned bucket path; then serve from a GPU pool behind an internal NLB with IAM controlling callers. Credentials never go into user-data; the serving instance reads them from a secrets store.
Traps
Assuming GPU capacity exists in every zone; losing a day of training because checkpoints lived only on local disk; exposing an inference endpoint publicly without authentication; shipping training data to a third-country service for convenience; and leaving GPU instances running overnight.