Transformation and Analysis with Built-ins and Essential Libraries
What this domain tests
Practical analysis with the standard library and NumPy: reading CSV, summarizing numbers, handling dates, counting and grouping, and doing vectorized arithmetic on arrays. Expect code-reading items ("what does this print?") and items asking which tool fits a task.
Standard library essentials
- csv: open(path, newline="") for reading and writing; csv.reader yields lists, csv.DictReader yields dicts keyed by the header; DictWriter needs fieldnames plus writeheader().
- math: sqrt, ceil, floor, isnan, fsum (accurate float totals).
- statistics: mean, median, mode, variance/stdev for a sample (n-1), pvariance/pstdev for a population (n), quantiles(data, n=4). The mean is dragged by outliers; the median resists them.
- datetime: date, datetime, timedelta; strptime parses text into a datetime (%Y %m %d %H %M %S), strftime formats back to text; subtracting dates gives a timedelta, and timedelta(days=7) adds a week.
- collections: Counter tallies with most_common(n); defaultdict(list) groups without KeyError; namedtuple for light records; deque for fast work at both ends.
NumPy essentials
- np.array([[1,2,3],[4,5,6]]) has shape (2,3) and one dtype for all elements.
- Vectorized operations: arr*2 scales every element; arr.sum()/arr.mean() reduce everything, axis=0 reduces rows (per column), axis=1 reduces columns (per row).
- arr[arr>3] filters with a boolean mask; np.where(cond, a, b) chooses elementwise.
- np.nan is a float and spreads through ordinary sum/mean, so use np.isnan and np.nanmean.
Worked example
ages = [23, 41, 35, 29]: statistics.mean and statistics.median both give 32.0, while statistics.stdev uses n-1. Change the last value to 200: the mean jumps to 74.75 but the median is only 38, so a report on skewed data should quote the median.
Traps
- stdev vs pstdev; forgetting CSV fields arrive as strings.
- Mixing up strptime and strftime, or a wrong format code raising ValueError.
- Swapping axis=0 and axis=1; letting NaN poison a total.