PCED common mistakes
PCED · Understand the traps and how to avoid them.
Loading a CSV with csv.reader and treating every field as a number, then calling float() on a header row or on a genuinely empty field and crashing (or silently skipping the row).
Candidates learn csv.reader produces lists of strings but forget that real files contain header rows, blank lines, and missing values; they also forget that float('') and float('N/A') raise ValueError rather than returning None.
What to do instead
Read the header explicitly with next(reader) or csv.DictReader, and convert each field inside a try/except ValueError that assigns None or a documented sentinel. Validate with a quick loop that prints type(field) for the first few rows before writing analysis code.
Opening files without a context manager or without specifying encoding and newline, then wondering why rows are double-spaced on Windows or why accented characters are mangled.
Plain open() works in simple examples, so candidates drop the with statement under time pressure; the default platform encoding and universal newline translation are invisible until the data contains non-ASCII text or embedded newlines inside quoted fields.
What to do instead
Always use with open(path, newline='', encoding='utf-8') as f: and pass the handle to csv.reader/csv.DictReader. If the source is not UTF-8, detect it once and state the encoding in a comment or in the summary.
Computing statistics over a list that still contains None, empty strings, or string numerals, and reporting a mean or median that is wrong or raises TypeError.
Cleaning and analysis get treated as separate phases, so the candidate filters for display but passes the raw list to statistics.mean() or numpy.mean(); mixed-type lists also fail late, after most of the work looks correct.
What to do instead
Build one cleaned list of floats up front, filtering with a comprehension such as [float(x) for x in raw if x not in (None, '')], and assert len(cleaned) equals the expected count. Reuse that single cleaned structure everywhere instead of re-deriving it per calculation.
Using statistics.median() on an even-length list and expecting the middle element, or confusing median with mode, then mislabeling the result in the written summary.
Candidates memorize the definitions but not the edge behaviour: median of an even count is the average of the two central values, and mode returns the first most-common value in a tie (or raises StatisticsError when every value is unique).
What to do instead
Sort the data manually once to confirm the middle values, and use collections.Counter.most_common() when ties matter so the tie-breaking rule is explicit. State in the summary which measure was used and why it suits the distribution's skew.
Mixing datetime strings and datetime objects, or subtracting a naive datetime from an aware one, and getting a TypeError or an off-by-one-day result.
Dates arrive as strings such as '2024-03-05' and candidates compare them lexically, which works until formats vary; timezone-aware values from parsing sometimes carry offsets that break subtraction against datetime.now().
What to do instead
Parse once with datetime.strptime or datetime.fromisoformat into a single list of date objects, keep all values naive or all aware consistently, and use (end - start).days for day differences. Test with a known pair of dates whose gap you can verify by hand.
Relying on NumPy broadcasting or boolean masking without checking shapes, so a (n,) array is combined with an (n,1) array and produces an unexpected (n,n) result that flows silently into a summary.
Broadcasting rules are convenient but non-obvious, and the resulting array still has valid numbers, so nothing raises; candidates also forget that comparing a list to an array elementwise needs np.array() first.
What to do instead
Print arr.shape after every reshape or slice, prefer explicit axis= arguments in sum/mean, and use np.array_equal or a small assert on shape before computing. When a result looks too large, check for accidental outer broadcasting.
Producing a chart or text summary in which the y-axis does not start at zero for a bar chart, or a line chart connects categories that are not ordered, and then drawing a causal conclusion from the visual.
Default plotting behaviour plus the desire for a dramatic visual leads to truncated axes; candidates also treat any sequence as a time series and interpret the resulting slope as a trend.
What to do instead
For bar charts set the baseline to zero and label units on both axes; sort categories deliberately or use a categorical chart type when order is meaningless. In the caption, state the comparison being made and avoid causal language unless the design supports it.
Reporting an aggregate such as an overall mean while ignoring a small subgroup with a tiny sample size, or dropping rows with missing values without disclosing how many were removed.
Aggregation is easy to compute and looks authoritative, so sample sizes and missingness are never surfaced; Simpson's-paradox-style reversals and selection effects then go unnoticed.
What to do instead
Always report n alongside every statistic, break results down by the key grouping variable, and include a one-line data-quality note stating rows read, rows excluded, and why. If a subgroup is too small, say so rather than reporting a precise-looking number.
Treating personally identifying columns as ordinary features: printing raw names, emails, or IDs into a summary, or joining datasets on an identifier without considering re-identification.
The analysis pipeline is the focus and ethics is treated as a separate essay topic, so de-identification never enters the code; small groups combined with quasi-identifiers such as age, ZIP, and date make 'anonymous' rows uniquely identifiable.
What to do instead
Before analysis, classify each column as identifying, quasi-identifying, or safe, drop or hash identifiers at load time, and aggregate small cells to a minimum group size. Keep a short note in the output describing what was removed and why.
Writing one long script with global variables and no functions, so a bug in cleaning silently corrupts the analysis and the candidate cannot tell which stage failed.
Notebook-style incremental execution encourages copy-paste and re-running cells out of order; without function boundaries or assertions, intermediate state is invisible and stale results persist.
What to do instead
Structure the work as small functions (load, clean, summarize) that take inputs and return outputs, add assert statements on row counts and value ranges at each boundary, and restart the kernel or re-run the whole script top to bottom before trusting any number.