VCE Applied Computing Data Analytics cheat sheet

Data life cycle stages: capture/collection, storage, preparation and cleaning, analysis, visualisation and communication, archiving and disposal; each stage can introduce or remove error.

Data integrity dimensions: accuracy, completeness, consistency, timeliness, uniqueness and validity; a dataset can be complete but still inaccurate.

Validation checks data on entry against rules (type, range, list, format, presence); verification confirms data matches its source (double entry, cross-checking against original documents).

Referential integrity requires every foreign key value to match an existing primary key in the related table; orphan records violate this rule.

Data types constrain valid values: integer, floating point, Boolean, date/time, string; storing a date as text prevents date arithmetic and sorting.

Garbage in, garbage out (GIGO): analytical outputs cannot be more reliable than the input data, so error rates propagate through every later stage.

Data management roles: data owner (accountable for a dataset), data steward (maintains quality and standards), data custodian (implements storage and security), end user.

Metadata describes data about data (field names, units, formats, provenance, collection date); missing metadata limits reuse and interpretation.

Primary data is collected firsthand for the current purpose (surveys, interviews, sensors, experiments); secondary data was collected by others for another purpose (census, government open data, company records).

Sampling methods: random (equal selection probability), stratified (random within defined strata), systematic (every nth item), cluster (whole groups selected), convenience (whoever is available, non-representative).

Census collects every member of a population; a sample collects a subset and risks sampling error, which decreases as sample size increases.

Survey question design faults: leading questions, double-barrelled questions, ambiguous wording, overlapping response categories and non-exhaustive options all bias results.

Automated collection methods include web scraping, APIs, sensor/IoT feeds, log files and transaction records; each has different accuracy, cost and refresh rates.

Response bias and non-response bias occur when those who answer differ systematically from those who do not; low response rates reduce representativeness.

Acquisition considerations: cost, time, accuracy, currency, coverage, format compatibility and legal permission to use the source.

Quantitative data is numeric and measurable; qualitative data is descriptive and often categorical; both can be structured, semi-structured or unstructured.

Cleaning operations: remove duplicates, fix inconsistent formats (dates, capitalisation, units), handle missing values, correct outliers and typos, and standardise categories.

Missing value options: deletion (listwise or pairwise), imputation (mean, median, mode, regression, last observation carried forward) or flagging as unknown; each choice changes results.

Outliers may be genuine extreme values or errors; detect with boxplots, z-scores (|z| > 3) or the IQR rule (below Q1 − 1.5×IQR or above Q3 + 1.5×IQR) before deciding to keep or remove.

Wrangling/transformation operations: filtering rows, selecting and renaming columns, sorting, joining/merging tables on keys, aggregating (sum, count, average), pivoting, and deriving new calculated fields.

Normalisation: min-max scaling maps values to 0–1 via (x − min)/(max − min); z-score standardisation gives mean 0 and standard deviation 1.

Data validation after cleaning includes range checks, consistency checks across fields, and reconciliation of totals against source control totals.

Document every cleaning step in a change log with date, rule applied and records affected so the transformation is reproducible and auditable.

Encoding and formatting issues: character encoding mismatches, delimiters inside text fields, inconsistent units (kg vs g) and mixed date conventions (DD/MM vs MM/DD).

Chart selection rule: comparison across categories → bar chart; trend over time → line chart; part-to-whole → stacked bar or pie (few slices only); relationship between two numeric variables → scatter plot; distribution → histogram or boxplot.

Truncated y-axis exaggerates differences; starting a bar chart axis at zero is the default for honest comparison of magnitudes.

Pie charts should be used only for parts of a single whole, with few categories, and where slices sum to 100%.

Data-ink ratio principle: maximise ink used for data and remove unnecessary gridlines, 3D effects, shadows and decoration that do not aid interpretation.

Label axes with variable name and units, provide a title stating the finding, cite the data source and date, and use a legend when more than one series is shown.

Colour should be accessible (avoid red–green only contrasts), consistent across charts, and used to encode meaning rather than decoration.

Audience adaptation: technical audiences accept detailed tables and statistical notation; general audiences need plain-language headlines, annotations and simplified scale.

Misleading techniques to avoid: inconsistent scales between compared charts, cherry-picked time ranges, dual axes implying causation, and area/volume scaling that distorts ratios.

Descriptive statistics: mean (sensitive to outliers), median (robust central value), mode (most frequent, works for categorical data), range, IQR and standard deviation (spread).

Correlation coefficient r ranges from −1 to +1; r near 0 indicates little linear relationship; correlation does not establish causation.

Confounding variables affect both the supposed cause and the outcome; controlled comparison or randomisation is needed to support causal claims.

Trend analysis uses moving averages or regression lines to smooth seasonal noise and estimate direction; extrapolation beyond the data range carries high uncertainty.

Decision rules: define the metric, threshold and action before analysis (e.g. reorder stock when average weekly sales exceed 500 units) to avoid post-hoc justification.

Benchmarking compares results against a target, historical baseline or peer group; without a baseline a number has no decision value.

Confidence intervals and sample size affect reliability: small or biased samples produce unstable estimates that should be reported with their uncertainty.

Interpretation must connect findings to stakeholder objectives, state assumptions and limitations, and recommend a specific action with expected impact.

Privacy principles (Australian Privacy Principles) require collection only for a lawful purpose, notice of purpose, limits on use and disclosure, data quality, security and access/correction rights.

Personal information is information about an identified or reasonably identifiable individual; sensitive information (health, biometric, racial origin) attracts stricter handling.

De-identification removes identifiers; re-identification risk remains when combined quasi-identifiers (age, postcode, date of birth) uniquely single out a person.

Informed consent requires the purpose, what data is collected, who receives it and the right to withdraw; consent obtained by deception or pressure is invalid.

Security controls: authentication, role-based access control, encryption in transit and at rest, audit logs, backups and least-privilege access.

Legal instruments include privacy and health-records legislation, copyright (data may be protected as a compilation), and contractual licence terms for purchased datasets.

Ethical obligations go beyond legality: fairness and absence of bias, transparency of method, accountability for harms, and honest reporting of inconvenient results.

Data breaches require assessment and notification where serious harm is likely; incident response includes containment, impact assessment and remediation.

Project life cycle phases: initiation/definition, planning, execution, monitoring and control, and closure with evaluation.

Scope defines deliverables and boundaries; a scope creep control process requires formal approval of changes to requirements.

Work breakdown structure decomposes deliverables into tasks that can be scheduled, assigned and estimated.

Gantt charts show tasks against time and dependencies; critical path is the longest dependent sequence and determines minimum project duration.

Milestones are zero-duration checkpoints marking completion of a deliverable; they carry no effort but anchor progress reporting.

Collaboration tools: version control for scripts and datasets, shared repositories, issue trackers, code review and documented naming conventions.

Risk management: identify, assess likelihood and impact, plan mitigation and assign an owner; include data unavailability and privacy breach as risks.

Evaluation criteria for a completed solution include accuracy against requirements, usability for stakeholders, timeliness, and documented limitations and next steps.