Architecting ML solutions
This domain tests architecture judgement before implementation: how to turn a business requirement into Google Cloud services, when to use prebuilt generative models instead of training your own, and how latency, cost, residency and security constrain the design.
Decide in this order
- Requirement first. Fix the serving mode (real-time, batch, streaming), the accuracy bar, the labeling budget, the residency region and the team's operational capacity. These eliminate most architectures immediately.
- Buy before build. Generic tasks (summarization, extraction, translation, classification, OCR) usually start with Gemini on Vertex AI or a prebuilt API. Custom training is justified by proprietary labels, strict accuracy targets or a need to control the model itself.
- BigQuery ML is the shortest path for tabular SQL users; AutoML when you want managed training on your own data with little code; custom containers on Vertex AI Training when you need a specific framework.
- Design the data path early. BigQuery for structured data, Cloud Storage for unstructured objects, Pub/Sub plus Dataflow for streaming.
Constraint checklist
Regional endpoints and VPC Service Controls for residency; CMEK when you must control encryption keys; least-privilege service accounts; quota for GPUs, TPUs and model throughput in the target region; egress and always-on endpoint costs.
Worked example
A retailer must summarize 2 million product reviews per month, keep data in an APAC region, and has a two-person team. "Good enough" quality is acceptable. Reasoning: the workload is asynchronous, so online serving adds cost and operational surface for nothing. Reviews already live in BigQuery; export to Cloud Storage and run Vertex AI Batch Prediction with Gemini through a regional endpoint. No custom model, no endpoint to autoscale, residency satisfied, and batch pricing sits far below online inference. Only if summaries fail review would supervised fine-tuning on labeled examples be evaluated.
Traps
- Defaulting to custom training before checking Model Garden and prebuilt APIs.
- Assuming every model, GPU or TPU exists in every region.
- Treating "managed" as automatically compliant with residency or VPC Service Controls.
- Building an online endpoint for a plainly batch workload.
- Omitting provisioned throughput and egress from cost estimates.