Describe Core Data Concepts
Know the vocabulary first
Core data concepts questions are almost always vocabulary and classification questions. Learn three axes:
- Structure: Structured data fits a fixed schema (rows and columns, e.g. a customer table). Semi-structured data has tags/keys but flexible schema (JSON, XML, YAML, Avro). Unstructured data has no schema (images, audio, video, PDFs, free text).
- Workload type: OLTP (transactional) handles many small read/write operations, is normalized, and optimizes for write speed and concurrency. OLAP (analytical) handles few, large, read-heavy queries over historical data, is often denormalized, and optimizes for aggregation.
- Processing style: Batch processes chunks on a schedule; streaming processes events continuously as they arrive.
Non-relational families
Be able to match a scenario to a database type: key-value (fast lookups by key), document (JSON-like documents, flexible schema), graph (entities and relationships, traversals), and column-family (wide rows, sparse data, very large scale).
Roles and file formats
Know who does what: a database administrator manages availability, security, and performance; a data engineer builds pipelines and integrates data; a data analyst builds reports and visualizations; a data scientist builds models and experiments. Know that CSV is text and row-oriented, JSON/XML are semi-structured text, and Parquet/ORC/Avro are binary and compact (Parquet is columnar and compresses well, so it suits analytics).
Traps and a rule of thumb
Traps: calling JSON "unstructured" (it is semi-structured); confusing ETL (transform before load) with ELT (load raw, transform later); assuming OLTP and OLAP use the same schema design.
Rule of thumb: ask what shape is the data and what will be done with it. Example: a folder of JSON order documents queried by analytics is semi-structured data in an analytical workload - load it into a columnar store, do not model it as an OLTP table.