watsonx Data Scientist - Associate cheat sheet

watsonx is a portfolio of three products: watsonx.ai (model build/tune/deploy), watsonx.data (open lakehouse), and watsonx.governance (risk and compliance); watsonx Orchestrate hosts agents and skills.

Core watsonx.ai runtime objects: project (collaboration + assets), space (deployment target), WML instance, and the Prompt Lab / notebook / AutoAI authoring paths that all resolve to the same deployment service.

A project stores assets (notebooks, models, prompts, connections); a deployment space stores deployable assets and is the boundary for serving endpoints and online/offline evaluation.

Service credentials are per-service; API keys are exchanged for IAM bearer tokens. The SDK reads credentials from config, environment, or explicit params, so the same notebook can target different regions.

Regional endpoints follow the pattern https://{region}.ml.cloud.ibm.com for the ML service and https://{region}.ds.cloud.ibm.com for data; the SDK requires an explicit URL for non-default regions.

Governance is cross-cutting: factsheets attach to assets in projects and spaces, and model evaluations can be tracked against both pre-production and production deployments.

Resource isolation: compute and storage are account- and region-scoped; a model deployed in one space cannot be called from another space without explicit promotion or re-deployment.

The platform is built on open standards: PMML, ONNX, and scikit-learn pipelines are portable, and watsonx.data exposes Iceberg tables queryable by Presto and Spark engines.

watsonx.data is an open lakehouse: object storage holds Parquet/ORC/Avro files, Iceberg adds table semantics (ACID, snapshots, schema evolution), and Presto/Spark engines query that storage.

Ingestion paths: a Presto CREATE TABLE ... AS SELECT from an external source, a Spark job writing Iceberg, or the console's ingestion wizard; ingested tables register in the metastore so all engines see them.

Bucketing is a physical layout choice that co-locates rows with equal hash values, reducing shuffle for joins on the bucket key; partitioning prunes files by column value and is the cheaper default for range filters.

Governance workflow: register the model or data asset, associate it with a use case, attach a factsheet, run evaluations, then record approvals; the use case is the audit anchor that groups related assets.

A model factsheet captures purpose, intended use, training data, performance metrics, and known limitations; a data factsheet captures source, schema, quality, and sensitivity classification.

Risk detection: governance can evaluate fairness (demographic parity, equal opportunity), drift, quality, and explainability; each evaluation produces a metric record attached to the asset version.

Governance requires an explicit service instance binding and a catalog; assets without a project or space connection cannot be governed, so binding precedes factsheet creation.

Lineage answers 'which data trained which model version'; it is derived from connections and asset references, so untracked manual uploads break the chain.

Prompt structure: system/instruction, context (retrieved or supplied), user input, and output-format instruction. Few-shot examples go before the target input and must mirror the expected output format exactly.

Decoding parameters: temperature scales randomness, top_p (nucleus) truncates the token distribution, top_k caps candidates, and repetition_penalty discourages loops; set temperature=0 or low top_p for deterministic extraction tasks.

max_new_tokens bounds generation length and directly drives latency and cost; it does not truncate the prompt, only the completion.

Stop sequences halt generation when the string appears; use them to end structured blocks (for example a closing tag) and reduce trailing chatter.

Model selection trade-offs: larger models improve reasoning but raise latency and cost; smaller/instruct-tuned models suit classification and extraction; code models suit code generation and explanation.

The Prompt Lab supports freeform, structured (chat roles), and chat modes; chat mode sends a list of role/content messages, which is what the SDK's chat APIs expect.

Tokenization: text is split into subword tokens, so token counts differ from word counts; prompts plus max_new_tokens must fit the model context window or the request errors.

Prompt variables (for example {input}) are declared in the prompt asset and filled at run time, enabling reuse of one prompt template across many inputs in a notebook or deployment.

Tuning options: prompt tuning (learns soft prompt vectors, cheap, small artifact), fine-tuning (updates all or adapter weights, higher cost and data need), and parameter-efficient methods such as LoRA.

Tuning requires a training data file in a supported format (JSONL with input/output fields, or task-specific schemas) uploaded to the project or referenced from cloud storage.

Deploying a tuned model creates a deployment in a space; the deployment exposes an inference endpoint that can be called with the task type (generation, chat, embeddings, classification).

Prompt-tuned models are served with the prompt tuning id in the request; fine-tuned models are served as a full model asset. Mixing the wrong task type returns a validation error.

Deployment assets: online deployment for real-time scoring with autoscaling, and batch deployment for scheduled scoring over a data file; both produce per-record outputs.

Evaluation before promotion: run a model evaluation on a test set to get metrics (accuracy, ROUGE, BLEU, perplexity) and store the result in the factsheet for the approval record.

Hardware sizing: choose GPU for foundation-model inference and tuning, CPU for classical ML scoring; the deployment's hardware spec affects throughput and cost, not model weights.

Version and promotion: a tuned model is a new versioned asset; promote by deploying the specific version, and record the version id in governance to keep lineage intact.

AutoAI automates preprocessing, algorithm selection, feature engineering, and hyperparameter optimization, ranking pipelines so the user can compare and pick a model.

Required inputs: a training data asset, a target column, and a prediction type (binary, multiclass, regression); the positive class matters for binary metrics and must be chosen explicitly.

Pipeline ranking uses an optimized metric (for example accuracy, ROC AUC, root mean squared error); the leaderboard shows the metric per pipeline, and the top pipeline is not always the best for production.

AutoAI-generated pipelines can be saved as a notebook or a model; saving as a model enables deployment, while the notebook exposes the generated Python for review.

Feature engineering options include hyperparameter optimization, feature scaling, and text/date feature extraction; enabling them increases training time and pipeline count.

AutoAI experiments consume compute for the duration of training; stopping an experiment early keeps completed pipelines but halts new ones.

Watson Studio / watsonx ML pipelines are DAGs of nodes (data, transform, train, evaluate, deploy) that can be scheduled or triggered; pipelines make retraining reproducible.

When a pipeline is deployed, the model is wrapped with preprocessing, so raw records can be scored; skipping the wrapper requires replicating the exact transformations.

RAG flow: embed the query, retrieve top-k chunks from a vector store, insert them into the prompt as context, generate an answer, optionally cite sources; retrieval quality bounds answer quality.

Chunking rules: chunk size trades context completeness against precision; overlap preserves boundary sentences; chunk on semantic boundaries (headings, paragraphs) rather than fixed characters when possible.

Embeddings map text to vectors; similarity is usually cosine or dot product. The embedding model used at index time must match the one used at query time.

Vector store options: watsonx.data Milvus, Elasticsearch, and external stores; the retriever returns documents with scores that can be filtered by threshold to avoid injecting irrelevant context.

Prompt template for RAG: instruct the model to answer only from the provided context and to state when the answer is absent, which reduces hallucination.

An agent combines a model with tools (functions, APIs, retrievers) and a reasoning loop; the model selects a tool, receives the result, and iterates until it produces a final answer.

watsonx Orchestrate hosts skills and agents; tools must have typed inputs and clear descriptions because the model chooses tools from their descriptions.

Evaluation of RAG: measure retrieval (recall@k, context precision) separately from generation (answer faithfulness, groundedness, answer relevance).

Fairness metrics: demographic parity compares positive-prediction rates across groups; equal opportunity compares true positive rates; disparate impact is the ratio of favorable outcomes between groups.

Drift types: data/covariate drift (input distribution shifts), concept drift (input-output relationship shifts), and prediction drift (output distribution shifts). Monitoring compares production data to a reference/training set.

Monitoring requires a feedback or ground-truth dataset to compute quality metrics; without labels only drift and fairness on inputs/predictions are computable.

Explainability: global feature importance summarizes the model overall, local explanations (for example SHAP values) explain a single prediction; governance records both for transparency.

A factsheet's evaluation section stores metric name, value, threshold, and dataset used, so an auditor can reproduce the assessment.

Responsible AI checks: bias, explainability, robustness/adversarial, and privacy; each can be run as an evaluation and attached to the model version before approval.

Alerting: monitoring thresholds trigger notifications when drift or quality crosses a limit; the threshold and window are configuration, not model properties.

Model health in production includes latency, error rate, and throughput alongside accuracy and drift; governance tracks the model lifecycle state (development, evaluation, approved, retired).

Typical setup: from ibm_watsonx_ai import Credentials, APIClient; Credentials(url=..., apikey=...) then APIClient(credentials) to reach projects and spaces.

ModelInference(model_id=..., credentials=..., project_id=...) exposes generate_text, generate_text_stream, chat, and embed_text; the project_id or space_id scopes the call.

generate_text returns a dict with generated_text and token counts; chat returns a response with choices[].message.content, so parsing differs between the two APIs.

Parameters go in a params dict (decoding_method, max_new_tokens, temperature, top_p, stop_sequences); passing an unknown key raises a validation error.

Uploading assets: client.data_assets.create(name, file_path) or the project's data asset API; deploying a model uses client.deployments.create(asset_id, space_id).

The watsonx.data SDK (ibm_watsonx_data / lakehouse clients) manages engines, catalogs, and schemas; queries run through Presto or Spark endpoints, not through the ML client.

Secrets: read the API key from environment variables or project credentials; never hardcode it in notebook cells that are saved into the project.

Reproducibility: pin the SDK version, record model_id and parameters in the notebook, and store the evaluation result as an asset so runs can be compared later.