Ask any analytics vendor where your raw event data lives and you'll get a hand-wavy answer about "secure cloud storage." Ask what the file looks like at rest — whose key encrypted it, what happens when encryption fails mid-job, whether an erased visitor can reappear in an old export — and the conversation usually ends.
This post answers those questions for ClickStream's export pipeline, with the actual source. It's the same philosophy as owning your analytics pipeline: your raw events are yours, in an open format, and the way they're protected should be inspectable, not asserted.
The Threat Model for an Encrypted Analytics Export
Raw exports are the most concentrated artifact an analytics platform produces. A single Parquet file can hold tens of thousands of events: page paths, referrers, session and visitor IDs, hashed identifiers, geo fields. Dashboards show aggregates; export files hold the underlying reality.
So the export path gets a stricter design than the rest of the platform:
- Per-tenant keys, not a bucket key. Every tenant's exports are encrypted under a key derived from that tenant's own encryption key. An object that leaks without your key is ciphertext, the same way a stolen analytics key is useless without your domain.
- Fail closed. If encryption throws — or the tenant's key can't even be resolved — the export aborts. There is no code path that falls back to writing plaintext.
- Authenticated encryption. AES-256-GCM includes a 16-byte authentication tag, so a tampered or truncated file fails to decrypt instead of silently producing corrupt rows.
Key Derivation: HKDF From Your Tenant Key
Each site has an encryption_key stored in the platform's D1 database. The export worker never uses it directly as a cipher key. Instead it runs the tenant key through HKDF-SHA-256 to derive a dedicated AES-256-GCM key, scoped to export encryption by a fixed salt and info string. This is the derivation, verbatim from apps/export-worker/src/services/r2.ts:
async function deriveEncryptionKey(tenantKeyHex: string): Promise<CryptoKey> {
const keyBytes = new Uint8Array(
tenantKeyHex.match(/.{2}/g)!.map(byte => parseInt(byte, 16))
);
// Import the tenant key as HKDF input
const baseKey = await crypto.subtle.importKey('raw', keyBytes, 'HKDF', false, ['deriveKey']);
// Derive AES-256-GCM key using HKDF with a fixed info string
return crypto.subtle.deriveKey(
{
name: 'HKDF',
hash: 'SHA-256',
salt: new TextEncoder().encode('clickstream-r2-export-v1'),
info: new TextEncoder().encode('aes-256-gcm-export'),
},
baseKey,
{ name: 'AES-GCM', length: 256 },
false,
['encrypt', 'decrypt']
);
}Three details worth noticing:
- The derived key is non-extractable. The
falseflag means the Workers runtime holds aCryptoKeyhandle that can encrypt and decrypt but never export its raw bytes — not to logs, not to a debugger. - Domain separation is explicit. The salt
clickstream-r2-export-v1and infoaes-256-gcm-exportmean this derived key is only ever the export key. If the same tenant key ever feeds another feature, that feature derives a different key. - One tenant, one key. A key from tenant A mathematically cannot decrypt tenant B's objects. There is no shared master key whose compromise opens every export in the bucket.
There's also a deliberately paranoid edge case: if one account has multiple sites with different site-level keys and no account-level key, the export worker throws rather than guessing which key to use. An ambiguous key situation is treated as a failure, not a coin flip.
Unencrypted Writes Abort by Design
The most important property of the pipeline isn't the cipher — it's the failure path. Each Parquet chunk is encrypted before the R2 write, and if that step throws, the entire export dies with a structured error log:
if (encryptionKey) {
try {
outputBuffer = await encryptData(outputBuffer, encryptionKey);
isEncrypted = true;
} catch (encErr) {
// ...structured encryption_failure log with clientId, chunk, window...
throw new Error(
`[Export] Encryption failed for ${clientId} chunk ${index}: ${errorMessage}. ` +
`Aborting export — sensitive data must not be written unencrypted.`
);
}
}The same fail-closed rule applies one step earlier: if the worker can't resolve a tenant's encryption key at all, it aborts with "cannot determine encryption requirements" before a single row is queried. Much of ClickStream is deliberately fail-open — the pixel and Signals degrade gracefully because a lost pageview is a small loss. Exports invert that: a plaintext file at rest is a large loss, so the pipeline prefers no file to an unprotected one.
On disk, each encrypted object is a simple, documented layout — a random 96-bit IV generated per file, then the ciphertext, then GCM's authentication tag:
[12-byte IV][ciphertext][16-byte auth tag]Encrypted objects get an .enc suffix, an application/octet-stream content type, and R2 custom metadata recording encryptionAlgorithm: AES-256-GCM, the event count, and the export window — so you can audit what a file contains without being able to read it.
Hive-Partitioned Parquet Your Warehouse Already Understands
Underneath the encryption, the payload is Apache Parquet with Snappy compression, encoded from Arrow tables. Object keys follow Hive-style partitioning, one hour per leaf:
acme-corp/year=2026/month=07/day=27/hour=14/events.parquet.enc
acme-corp/year=2026/month=07/day=27/hour=14/events.parquet.part1.encDesign choices that matter when you're the one loading the data:
- Deterministic keys. The same whole-hour window always produces the same object key, so a retried export overwrites itself instead of duplicating data. Ad-hoc windows get the full time range stamped into the filename so they can never collide with the hourly series.
- Bounded file sizes. Chunks cap at 50,000 events per file (streamed from the event store 10,000 rows at a time), with
.partNsuffixes for overflow — no gigabyte-scale files that OOM your loader. - Strict column typing. String columns always encode as UTF-8, never as null-typed columns — a subtle Arrow inference trap that the code guards against specifically because null-typed columns break downstream Parquet consumers.
- Real columns, not blobs. Typed fields like
timestamp,event_type,page_path,referrer,session_id,visitor_id,device_type,country,scroll_depth,is_bot,bot_score,utm_params, andclick_ids. IP addresses ship only asip_hash— the raw IP is not in the export.
The format is compatible with BigQuery, Athena, Spark, and DuckDB. To be precise about what that means: there is no BigQuery integration. ClickStream doesn't hold credentials to your warehouse or push data into it. You decrypt with your key, then load the files with your own tooling — bq load, an Athena external table over the partition tree, or DuckDB directly:
-- After decrypting, DuckDB reads the partition tree natively
SELECT event_type, count(*) AS events
FROM read_parquet('exports/**/*.parquet', hive_partitioning = true)
GROUP BY event_type
ORDER BY events DESC;Decrypting With Your Key
Because the file layout and derivation parameters are documented above, decryption is ~20 lines of standard WebCrypto in Node — no ClickStream SDK required:
import { webcrypto as crypto } from 'node:crypto';
import { readFile, writeFile } from 'node:fs/promises';
// Layout: [12-byte IV][ciphertext + 16-byte GCM auth tag]
async function decryptExport(path, tenantKeyHex) {
const data = new Uint8Array(await readFile(path));
const keyBytes = Uint8Array.from(
tenantKeyHex.match(/.{2}/g).map((b) => parseInt(b, 16))
);
const baseKey = await crypto.subtle.importKey('raw', keyBytes, 'HKDF', false, ['deriveKey']);
const key = await crypto.subtle.deriveKey(
{
name: 'HKDF',
hash: 'SHA-256',
salt: new TextEncoder().encode('clickstream-r2-export-v1'),
info: new TextEncoder().encode('aes-256-gcm-export'),
},
baseKey,
{ name: 'AES-GCM', length: 256 },
false,
['encrypt', 'decrypt']
);
const iv = data.slice(0, 12);
const ciphertext = data.slice(12);
const plaintext = await crypto.subtle.decrypt({ name: 'AES-GCM', iv }, key, ciphertext);
await writeFile(path.replace(/\.enc$/, ''), Buffer.from(plaintext));
}If the file was tampered with in storage or transit, that decrypt call throws — GCM's auth tag makes corruption loud instead of silent.
Erased Visitors Never Ride Along
An export pipeline can undo a privacy program: a visitor exercises their right to erasure, the platform deletes them from its live stores, and then a raw export happily re-emits their events from the event archive. ClickStream closes that hole with erasure tombstones. Before any export runs, the worker loads every erased visitor for the tenant:
SELECT DISTINCT et.visitor_id
FROM erasure_tombstones et
JOIN sites s ON s.id = et.site_id
WHERE s.client_id = ? AND et.expires_at > ?Every event row is filtered against that set before it ever reaches the Parquet encoder. An erased visitor doesn't get redacted in the file — they're excluded from it. The broader machinery behind those tombstones is covered in GDPR as cron jobs: compliance you can read in the source.
A 90-Day Retention Window, Enforced Daily
Raw exports live in R2 for 90 days, then a daily cleanup job (midnight UTC) deletes anything older, parsing the date straight out of each object's Hive partition path. The window is a working buffer for you to pull files into your own warehouse — not a shadow archive that accumulates forever. It's the same honest-expiry stance as the session replay retention ladder: data that has a reason to exist has a date it stops existing.
What's Available Today, Honestly
Marketing pages love to describe pipelines in the perpetual present tense. Here's the actual availability:
| Export | Plan | How it works today |
|---|---|---|
| CSV export | Growth and above | Self-serve from the dashboard |
| Encrypted Parquet export | Scale and above | Fulfilled on request — contact us with a time window |
| BigQuery / Athena / warehouse push | — | Does not exist. The format is compatible; the loading is yours |
The plan gate is enforced in the worker itself — a client below Scale is skipped with an explicit log line, not silently exported. Automatic hourly Parquet delivery is built and intentionally not yet enabled as a self-serve feature; today each Parquet export run is triggered on request against the window you need. Plan details are on the pricing page.
The Bottom Line
- Per-tenant AES-256-GCM — every export encrypted under a key derived (HKDF-SHA-256) from your tenant key; no shared master key.
- Fail closed — encryption failure or key ambiguity aborts the export; plaintext never reaches storage.
- Open format — Hive-partitioned, Snappy-compressed Parquet that BigQuery, Athena, Spark, and DuckDB read natively once you decrypt.
- Erasure-aware — tombstoned visitors are excluded before encoding, so an export can't resurrect deleted people.
- Honest retention — 90 days in R2, enforced by a daily cleanup job.
An export you can't decrypt without the vendor is a hostage. An export encrypted so the vendor's storage can't betray you — sealed under your key, in a format any warehouse reads — is actually your data.