Initial Load and Repeatable Delta Import
A retailer is migrating to Neo4j 5.26 Enterprise on a single 64 GB, 16-core server. Source CSVs contain 12 million Customer rows, 45 million Order rows, and 90 million OrderLine rows, about 8 GB total. A nightly delta adds or updates roughly 300,000 customers and orders. The current Cypher script runs one LOAD CSV with MERGE for the entire file in a single transaction. It runs out of memory, creates duplicate Customer nodes because no uniqueness constraint exists, and fails on some rows with 'Cannot merge node using null property value' because email is sometimes blank. Dates arrive in two formats: ISO-8601 strings and epoch milliseconds. The team needs an initial load that finishes overnight and a repeatable delta job that can be re-run safely after a failure.
1. Choose the initial load mechanism and justify it against LOAD CSV + MERGE. What must be true of the CSV files?
Read the worked answer
For 8 GB and 147 million records, use neo4j-admin database import full against a stopped database. It reads separate node and relationship CSV files, builds stores directly, and does not execute Cypher or MERGE per row, so it avoids the transaction memory and duplicate-merge cost. Requirements: consistent headers, unique IDs per entity, relationship files that reference those IDs, correct field terminators and quoting, and no null IDs. The importer does not enforce your uniqueness constraints during load; validate IDs before import. If the database must stay online, LOAD CSV is the fallback, but then batch it and create constraints first. For this size, offline import is the judgement call. After import, create constraints and indexes before the delta job; they are not a substitute for clean IDs.
2. Write the delta pattern to be idempotent and memory-safe. Address constraints, batching, and null emails.
Read the worked answer
Create CREATE CONSTRAINT customer_id IF NOT EXISTS FOR (c:Customer) REQUIRE c.id IS UNIQUE and the equivalent for Order. Normalize the CSV before load: trim blanks, convert email '' to null, and convert both date formats consistently. Use batched transactions, not one transaction:
CALL {
LOAD CSV WITH HEADERS FROM 'file:///customers_delta.csv' AS row
WITH row WHERE row.id IS NOT NULL
MERGE (c:Customer {id: row.id})
SET c.name = row.name,
c.email = CASE WHEN row.email = '' THEN null ELSE row.email END,
c.updatedAt = datetime(row.updatedAt)
RETURN count(*) AS ignored
} IN TRANSACTIONS OF 5000 ROWS
MERGE is idempotent on the unique id; a re-run updates rather than duplicates. Never MERGE on a property that can be null; filter null IDs or use a separate match/create path. Check the plan uses the unique index. For orders, merge Order and then match Customer and OrderLine by ID; do not create relationships to null nodes.
3. How would you handle the bad rows and verify the delta is safe to re-run? Include date parsing and validation.
Read the worked answer
Pre-validate with a staging query or external script: count rows by id, detect duplicate IDs, null IDs, and malformed dates before MERGE. In Cypher, branch on format: if the date matches ^\d+$, use datetime({epochMillis: toInteger(row.date)}); otherwise use datetime(row.date). Use coalesce only if a fallback is valid; do not silently set a wrong date. Load rejects into a quarantine file or label with the raw row and error. After a run, compare distinct source IDs vs distinct Customer IDs, and relationship counts vs expected. Re-run the same file in a test database and confirm no duplicate nodes and unchanged counts. Use SHOW TRANSACTIONS during the job to confirm batch sizes, and check dbms.listQueries for long transactions. Finally, run a sample of 100 known records through the same query and compare properties.