Public Web Scraping Meets GDPR Deletion Requests
A lead-generation company collects publicly available professional profiles and company websites, producing 2.1 million contact records. About 18% are EU residents; records include name, job title, work email, company, profile URL, and sometimes a personal mobile number found in page text. The company relies on legitimate interest and keeps records for three years. Its privacy notice is linked only in the website footer and does not mention profiling or third-party sale. It receives about 5,000 deletion requests per month, handled manually by two contractors via email. A vendor stores a weekly snapshot in an S3 bucket configured for public read for 36 hours while a downstream job runs; access logs are incomplete. A journalist has asked how the company handles EU deletion requests.
1. Assess the lawful basis, transparency, and data-minimization weaknesses in this operation. What would you prioritize?
Read the worked answer
Public availability does not by itself make the processing lawful or fair. Legitimate interest can be a basis, but it requires a documented balancing test: the company must show a real interest, necessity, and that individuals' rights do not override it. For 2.1 million records, including EU residents and personal mobile numbers, that assessment must be specific to the data categories and sources. Scraping a personal mobile number from page text is harder to justify than collecting a published work email, because it may exceed reasonable expectations and increase risk.
Transparency is weak. A footer-only notice that omits profiling and third-party sale is unlikely to meet the requirement to inform people about purposes, categories, sources, recipients, retention, and rights. The company should provide a layered notice at the point of collection where possible, a clear source list, and an easy opt-out. Data minimization is also weak: keeping every record for three years needs a field-level justification. Job titles and companies go stale quickly; personal mobile numbers may not be necessary for the stated purpose. I would prioritize a data inventory and lawful-basis assessment, then a retention schedule by field, then notice and opt-out, before expanding collection.
2. Design a deletion workflow that is auditable and propagates to backups and vendors. Include how you would handle re-scraping and legal exceptions.
Read the worked answer
The workflow should start with a single intake channel that creates a case ID, timestamps the request, and records the requester's identity and the scope. Identity verification should be proportionate: confirm control of the email or profile in question without demanding excessive ID. The system should search all stores by stable identifiers and hashes, not just by name, because names are ambiguous. Matches should be reviewed for legal exceptions, such as a record needed for a contract, defense of legal claims, or a statutory retention obligation; the exception and reviewer must be logged.
Deletion should include the primary database, analytics copies, marketing automation, and vendor snapshots. The vendor contract must require deletion within a defined SLA and provide a certificate or API confirmation. Backups need a documented approach: either delete from backups when technically feasible or suppress the record so it cannot be restored into production, with a defined maximum backup lifecycle. A suppression list should store a hashed identifier to prevent re-scraping from re-adding the person; the crawler must check it before insertion. The case should be closed only when all stores confirm, and the company should track request volume, median completion time, exceptions, and failures. Under GDPR, the default response period is one month, extendable by two months for complex requests, so two contractors handling 5,000 requests per month manually is not sustainable.