I85PREP
⌘ K
IBM watsonx Data Lakehouse Engineer Associate study materials
IBM watsonx Data Lakehouse Engineer Associate study resources

IBM watsonx Data Lakehouse Engineer Associate: watsonx.data Architecture and Provisioning

IBM watsonx Data Lakehouse Engineer Associate · Work through your exam, one topic at a time.

All guide sections

watsonx.data Architecture and Provisioning

This domain tests whether you can explain the lakehouse architecture and provision it correctly: storage registration, metastore and catalog creation, engine deployment, and the sizing and lifecycle choices that follow.

Separate storage, metadata, and compute

watsonx.data does not copy data into a proprietary store. It registers object storage (IBM Cloud Object Storage, Amazon S3, Azure Data Lake Storage Gen2, Google Cloud Storage, or MinIO on premises), points a metastore at it, and exposes the result as a catalog. Presto and Spark engines then read the same Iceberg or Hive tables without duplicating them. A catalog is therefore the pairing of a storage location with a metastore, not a bucket and not an engine.

Provisioning sequence

  1. Confirm the deployment: IBM-managed SaaS or self-managed on Red Hat OpenShift.
  2. Register storage with endpoint, bucket, and credentials; test connectivity.
  3. Register or select a Hive metastore.
  4. Create a catalog (Iceberg for new lakehouse tables; Hive for existing Hive data).
  5. Provision Presto and/or Spark engines, choosing node sizes, worker counts, auto-scale, and suspend settings.
  6. Grant users access to engines and catalogs.

Worked example

A team needs one shared lakehouse for SQL analysts and Spark engineers. You create an IBM COS bucket, register it as storage, associate the metastore, and create an Iceberg catalog named sales_lake. You provision a small Presto engine for interactive SQL and a Spark engine for ETL. Both point at sales_lake. Because storage and metastore are shared, a table written by Spark is immediately queryable in Presto. If you had created two catalogs for two buckets, the engines would see different table namespaces and you would need to manage two metadata stores.

Traps

  • Treating a bucket as a catalog; a catalog requires storage plus metastore.
  • Assuming provisioning an engine also creates storage or catalogs.
  • Choosing a Hive catalog type for new Iceberg tables.
  • Leaving engines always running; suspend idle engines to control cost.