Data Management, Quality, and Reproducibility
Data lifecycle
Candidates should understand the data lifecycle: planning, collection, entry, cleaning, storage, sharing, and archiving. Know the value of a data management plan that specifies formats, naming conventions, storage, access controls, and retention. Understand the difference between source data and derived datasets, and the importance of a documented, reproducible pipeline from raw to analytic data.
Quality control
Learn the elements of quality assurance and quality control: standard operating procedures, double entry or verification, range and logic checks, and regular audits. Understand the role of version control for data and code, and why code review and scripted analyses reduce error. Know the concepts of data provenance and audit trails.
Reproducibility and openness
Understand the difference between reproducibility (same data and methods yielding the same result) and replicability (new data yielding consistent findings). Learn the value of pre-registration, data sharing, and code sharing. Know that de-identification reduces but does not eliminate re-identification risk, and that sharing must respect consent and governance.
Traps
- Assuming cleaned data need no documentation of transformation steps.
- Treating spreadsheet-based manual editing as adequate provenance.
- Confusing reproducibility with replicability.
- Believing removal of direct identifiers guarantees anonymity.
Worked rule of thumb
Apply the two-person, two-file test: could an independent analyst, given your raw data, code, and documentation, arrive at your reported numbers without asking you anything? If not, the workflow is not yet reproducible.