CWIDP study plan
CWIDP · Give each week a clear focus.
8 weeks · About 6 hours per week
Data Collection and Web Acquisition
Map the full web acquisition pipeline (request, render, parse, extract, stage) and identify where each CWIDP domain touches it
Configure an HTTP client with headers, timeouts, retries, and a user agent you can justify to a site owner
Compare static HTML parsing against headless browser rendering and decide when each is required
Handle pagination, rate limiting, and robots.txt directives in a single acquisition script
Your checkpoint
A working Python acquisition script that crawls 50 paginated pages with retry/backoff and writes raw responses plus a fetch log (status, latency, bytes) to disk
Data Collection and Web Acquisition
Read and apply robots.txt, terms of service, and crawl-delay conventions before writing any scraper
Implement session handling, cookie persistence, and form submission for an authenticated source
Extract structured records from semi-structured pages using CSS selectors and XPath, with fallbacks
Design a schema contract for acquired records before extraction begins
Your checkpoint
An extraction module that converts 3 page templates into a single normalized record schema, validated against a written field contract, with a README documenting legal and rate-limit decisions
Data Quality, Cleaning, and Validation
Profile a raw dataset for nulls, duplicates, type mismatches, and out-of-range values
Write validation rules that separate hard failures from soft warnings and log both
Resolve duplicate records using deterministic matching plus a fuzzy fallback with a confidence threshold
Standardize dates, currencies, and categorical values into documented canonical forms
Your checkpoint
A validation report showing before/after defect counts per rule, plus a reusable validation function library with tests covering at least 10 edge cases
Data Modeling and Storage
Choose between normalized relational, star schema, and document store models for three given access patterns
Design a star schema with conformed dimensions and document grain for each fact table
Implement slowly changing dimension type 2 handling for a changing customer attribute
Write DDL with primary, foreign, and uniqueness constraints that enforce your data contract
Your checkpoint
An ER diagram plus executable DDL for a star schema, and a load script that correctly versions at least one SCD2 dimension with effective dates
Data Analysis and Statistical Methods
Select the correct summary statistic and distribution check for a given business question
Run and interpret hypothesis tests (t-test, chi-square) including assumptions and p-value caveats
Build a regression model and explain coefficients, residuals, and fit quality to a non-technical reader
Distinguish correlation from causation and identify confounders in a supplied dataset
Your checkpoint
An analysis notebook on a real dataset answering 3 business questions, each with the method chosen, assumptions checked, and a plain-language interpretation of results
Data Visualization and Reporting & Privacy, Ethics, and Regulatory Compliance
Match chart type to data relationship and remove chart elements that do not carry information
Apply accessibility practices: colorblind-safe palettes, alt text, and sufficient contrast
Classify data fields as personal, sensitive, or anonymous and apply minimization before sharing
Map a data workflow to GDPR/CCPA obligations including lawful basis, retention, and subject rights
Your checkpoint
A two-page stakeholder report with 3 justified visuals plus a data-handling note listing every personal field used, its lawful basis, and its retention period
Security and Data Governance
Apply least-privilege roles and column-level masking to a multi-team data store
Design an audit logging scheme that records who accessed which records and when
Write a data classification and retention policy covering at least 4 data categories
Identify security risks in an ingestion pipeline and propose controls for each
Your checkpoint
A governance package containing a role/permission matrix, a classification table, and a threat-to-control mapping for the pipeline you built in weeks 1-4
Data Infrastructure and Automation & Data Collection and Web Acquisition & Data Quality, Cleaning, and Validation & Data Modeling and Storage & Data Analysis and Statistical Methods & Data Visualization and Reporting & Privacy, Ethics, and Regulatory Compliance & Security and Data Governance
Orchestrate the acquisition, validation, and load steps as a scheduled DAG with failure alerts
Add idempotency and rerun safety so a failed task can restart without duplicating data
Complete two timed 60-question mixed-domain practice sets and log every miss by domain
Re-drill your three weakest domains using targeted flashcards and one hands-on redo each
Your checkpoint
An end-to-end scheduled pipeline that runs unattended, plus a scored practice log showing per-domain accuracy and the specific remediation done for each weak area
Milestones
Week 2: You can explain, without notes, why a given site should be scraped statically versus rendered, and your acquisition script runs within documented rate limits
Week 4: Your star schema loads clean validated data end to end, and you can defend each modeling choice against an alternative
Week 6: You score at least 70% on a mixed practice set covering domains 1-5, and your report passes a self-review for accessibility and data minimization
Week 8: You score at least 80% overall on a full timed practice exam with no domain below 65%, and your pipeline runs unattended for 3 consecutive scheduled runs