Prepare and Use Data for Analysis
ELT first, transform in place
The modern pattern is to land raw data in GCS or BigQuery and transform with SQL: Dataform or dbt for versioned, tested, dependency-aware models, with assertions feeding data-quality metrics. Dataplex and Data Catalog provide discovery, lineage, and quality scans. Use Cloud Data Fusion or Dataflow only when the transformation is genuinely code-shaped - unstructured files, complex parsing, or ML feature generation.
Making it usable
Understand BigQuery columnar, byte-scanned pricing: SELECT * on a wide table is expensive, LIMIT does not reduce bytes scanned, and ORDER BY on a huge result is a scalability smell. Use partition pruning, QUALIFY for deduplication, ARRAY_AGG and UNNEST for nested data, APPROX_COUNT_DISTINCT for high cardinality, and materialized views or BI Engine for repeated dashboard queries. BigQuery ML covers in-warehouse training for standard model types; Looker and Looker Studio sit on top.
Governance and security
Column-level security via policy tags, row-level security via row access policies, authorized views to expose subsets without copying, and dynamic data masking for analyst-facing surfaces. Sensitive Data Protection (DLP) handles inspection, de-identification, deterministic tokenization that preserves joins, format-preserving encryption, and k-anonymity or l-diversity for released datasets.
Traps
Copying data into a new dataset just to restrict columns - use policy tags or an authorized view. Tokenizing with a non-deterministic method and then expecting joins to work. A view whose definition is not authorized cannot be shared across datasets. Treating LIMIT, or a WHERE clause on a non-partition column, as cost control.
Rule of thumb: curate with views and policy tags, never with copies.