GDPR as Cron Jobs: Compliance You Can Read in the Source

Compliance Pages Are Promises. Schedulers Are Facts.

Most analytics vendors handle GDPR with a trust page: a reassuring paragraph about taking privacy seriously, a link to a DPA, and an email address for deletion requests. The problem is structural. A policy document can promise anything. The only thing that actually deletes data on a schedule is a scheduler.

This essay is about GDPR analytics engineering the way we practice it: retention as a cron job that runs every five minutes, erasure as an authenticated endpoint that propagates through five storage systems, and PII scrubbing as a function that runs at the edge before anything is stored. Every mechanism below is a specific job, route, or table in the ClickStream platform — including the genuinely hard parts, like data already written to immutable storage.

A privacy policy tells you what a vendor intends. A cron schedule tells you what actually happens at 3 a.m. when nobody is watching.

Retention Presets That Expand into Code

Every ClickStream site carries a compliance profile built from one of five presets: standard, gdpr_strict, hipaa, ccpa, or custom. A preset is not a label on a settings page — it is a function that expands into a concrete configuration object consumed by the collector, the SDK, and the purge jobs:

// packages/shared-types — expandPreset('gdpr_strict'), verbatim
{
  preset: 'gdpr_strict',
  consentMode: 'opt_in',
  scrubLevel: 'aggressive',
  allowThirdPartyIdentity: false,
  allowMarketingIdentity: false,
  allowDeviceFingerprint: false,
  allowThirdPartyEnrichment: false,
  hipaaMode: false,
  retentionDays: 180,
  autoPurge: true,
}

Every preset ships with auto-purge on — there is no preset that retains visitor data forever by default. And the profile is replicated to the collector's edge configuration, so consent mode and scrub level are enforced server-side from the first request — not by JavaScript that a slow page load might skip. The same profile also carries identity toggles (third-party identity, marketing identity, device fingerprinting), which interact directly with how first-party cookie persistence works on a site, and an excludedPaths list: path prefixes like /checkout for which the collector drops form values, replay frames, and behavioral summaries entirely, enforced server-side.

Session replay follows the same philosophy with its own tiered windows — we wrote about that separately in the honest retention ladder.

The Retention Purge Cron Runs Every Five Minutes

A retention setting is only a promise until something enforces it. On our platform, the enforcement is a purge job triggered by the export worker's scheduled handler every five minutes. Each run:

• Iterates sites with auto-purge enabled (up to 50 per run) and finds visitors whose last activity predates the site's retention window;
• Deletes the visitor and every related row — devices, IP records, consent receipts, identity links, merge history, and campaign attributions — in bounded parallel batches;
• Writes an erasure tombstone for every deleted visitor, so the collector will not silently re-create the profile when the same browser returns. Deletion and tombstoning always happen in the same run — no visitor is purged without its tombstone;
• Writes an audit row per site recording the retention window, the cutoff date, and exactly how many visitors, records, and tombstones the run produced.

The cadence itself shows what treating compliance as code actually means. The job originally rode an hourly cron. Each run has a hard budget of 200 visitors — pinned by a platform subrequest limit that cannot be raised — so hourly runs capped the entire platform at 4,800 purged visitors per day. Measured inflow of purge-eligible visitors averaged around 5,150 per day with peaks over 10,000: the queue was quietly falling behind, a compliance bug even though nothing was "broken." Since the budget could not grow, cadence was the only lever — the job moved to the five-minute tick, raising the ceiling to 57,600 per day.

And because a purge that silently falls behind is the failure mode that matters, each run also probes the backlog of the least-recently-purged site and logs a warning when it exceeds ten runs' worth of budget. The drain-versus-inflow signal is part of the job, not a dashboard someone has to remember to check.

Article 17 as an Endpoint, Not an Email Alias

The GDPR right to erasure is implemented as an authenticated dashboard endpoint. It requires a session, site ownership, and an explicit confirmation flag — the request fails without it:

POST /api/privacy/delete
{
  "siteId": "site_abc123",
  "identifierType": "email",        // email | phone | hem | visitor_id
  "identifierValue": "subject@example.com",
  "confirm": true                   // schema-enforced literal; omitting it rejects the request
}

One request propagates through every system that might hold the data subject's records:

• Dashboard database: the visitor row plus nine related tables, including form submissions and hashed-email history;
• Erasure tombstones: a database row that suppresses the visitor from dashboard reads and from every future raw export;
• Collector tombstones: keys in the edge KV store (90-day TTL, matching analytics retention) that block visitor re-creation, identity resolution, and enrichment dispatch if the same browser comes back;
• Identity graph: a tenant-scoped POST /v1/erasure call that deletes the person's signals, edges, and signal logs from the graph service;
• Enrichment purge by hash: a DELETE /v1/enrichments/by-md5-hash call that soft-deletes every enrichment row matching an HMAC of the source hash — the encrypted payload and its IV are overwritten with NULL, so the data is unrecoverable while the row remains addressable for audit. The audit trail itself keeps only an 8-character hash prefix, never a full identifier.

Each request is recorded in a data_subject_requests table, so there is a durable record that the DSAR happened, when, and what it covered. This is the same machinery that keeps the promotion path from anonymous visitor to CRM contact clean — covered in the compliant path from visitor to contact.

The Honest Part: What Erasure Cannot Do Instantly

Two of our storage layers are append-only, and pretending otherwise would be marketing, not engineering. The erasure endpoint states its own limitations in its response:

• Analytics event rows are immutable — they cannot be selectively deleted, and they age out of storage within 90 days;
• Already-written Parquet export files cannot be selectively rewritten. The erased visitor is excluded from every future export, and existing objects are removed by the daily 90-day storage retention cleanup.

In other words: live systems forget immediately, append-only systems forget within their bounded retention window, and the endpoint tells you which is which.

Erasure-Aware Exports Fail Closed

Raw Parquet exports (available on request on Scale and higher plans) are the place where a deleted visitor could most easily leak back out. So every export run starts by loading the erased-visitor set for the tenant:

-- export worker, verbatim: the exclusion set loaded before any export
SELECT DISTINCT et.visitor_id
FROM erasure_tombstones et
JOIN sites s ON s.id = et.site_id
WHERE s.client_id = ? AND et.expires_at > ?

That set is passed into the Parquet writer as an exclusion list, so erased visitors are filtered out of every future export window. And the failure mode is the right one: if the export worker cannot resolve the tenant's encryption key or its exclusion set, the export aborts rather than shipping data it cannot vouch for. Exports are also encrypted per tenant with the customer's own key — the subject of its own essay, Your Data, Your Key.

PII Is Scrubbed at the Edge, Before Storage

Data you never store is data you never have to purge. Before any event reaches analytics storage, the collector sanitizes free-text fields — URLs, referrers, element selectors — server-side at the edge. The intensity follows the site's scrubLevel:

• Standard: email and SSN patterns are redacted from free text, and known PII query parameters (email, phone, name, password, token, and similar keys) are stripped from URLs;
• Aggressive (the gdpr_strict and hipaa default): additionally redacts phone-number patterns everywhere, strips all query strings from page URLs and referrers, and drops captured element text entirely.

Payment card numbers get special treatment: they are redacted unconditionally at both levels. A candidate is matched broadly (13–19 digits in any script, any common separator), then confirmed by digit count and a Luhn check after Unicode normalization — so real card numbers are caught even in fullwidth digits, while legitimate long numbers like invoice and tracking IDs pass through untouched. That backstop exists specifically for the case where an SDK-side blocklist misses or a stale cached bundle is still running.

The Identity Graph Cleans Up After Itself

Person-level resolution creates its own privacy obligations, so the identity graph service runs its own maintenance cron every six hours:

// identity-graph scheduled handler — cron: 0 */6 * * *
pruneExpiredEdges()             // expired edges + household clusters
garbageCollectMergedNodes(90)   // merged nodes older than 90 days
scrubExpiredPii(90)             // raw PII in audit records older than 90 days
processResolutionTasks(100)     // reprocess ambiguous graph states

The third step is the one that matters most for this essay: any raw PII held in the graph's audit records is scrubbed once those records are 90 days old. Audit trails are kept; the personal data inside them is not kept a day longer than the window requires. How the graph is structured — and why edges expire at all — is covered in Inside the Identity Graph.

Consent Receipts: The Audit Trail That Survives

Consent is recorded the same way everything else here is: as rows, not vibes. Every consent decision is written to a consent_receipts table tied to the visitor. Those receipts then follow the data subject through every right they exercise:

• Access and portability: the data export bundles the visitor record with their devices, IP records, consent receipts, and identity merge history — the receipts are part of the subject's file, not a separate system;
• Erasure: consent receipts are deleted with the visitor, and the deletion count is itemized in the purge's audit row;
• Accountability: retention purges write system audit rows, and DSARs are logged in data_subject_requests — so the record that compliance happened outlives the personal data itself.

What This Essay Is Not Claiming

It is worth being explicit about boundaries. This essay makes engineering claims, not certification claims — we are describing jobs and endpoints, not waving a badge. The operational constants it quotes (per-run budgets, cadences, measured inflow) are current design decisions that will evolve, not service-level guarantees. And it does not claim erasure is instantaneous everywhere — the append-only layers forget on their bounded 90-day schedule, and we would rather tell you that than pretend.

The same posture runs through the rest of the platform: API keys are scoped and domain-gated so a leaked key is useless (covered here), and the whole pipeline is built so that you own your analytics data rather than renting access to it.

The Bottom Line

GDPR compliance in an analytics platform is not a document problem. It is a distributed-systems problem: retention windows that must be enforced against real inflow, erasure that must propagate through databases, edge caches, graphs, and export pipelines, and audit trails that must outlive the data they describe. The mechanisms above — a five-minute purge cron with a backlog alarm, an Article 17 endpoint that fans out to five systems, exports that fail closed, and PII scrubbing that runs before storage — are how we turned that problem into running code.

If a vendor's deletion story is an email alias, their retention story is a shrug. Ask to see the scheduler.