Operations
By the end of this page you know how redfish retains and anonymizes data, what the worker does, and how to back up and restore the database.
Retention and anonymization
Section titled “Retention and anonymization”redfish keeps personal data only as long as it is needed. A retention job runs once a day inside the worker. For each organization, the targets whose campaigns are all finished and older than that organization’s retention window are anonymized: the first and last name are cleared, the email is replaced by redacted-<id>@redacted.invalid, and an anonymizedAt timestamp is set.
Reports do not suffer from this. The events behind a report hold no personal data, and campaign recipients are kept, so the funnel statistics stay the same after anonymization. See Campaigns.
The worker process
Section titled “The worker process”redfish runs a second process next to the application: the worker. It drives a database-backed job queue and is responsible for:
- Sending campaign email from the queue.
- Running scheduled campaign launches.
- Running the daily retention job. The worker starts the retention job at launch, and each run schedules the next one 24 hours later.
The worker is a separate Compose service (worker in development, worker-prod with the prod profile). In production it waits until the web service is healthy before it starts.
Backup
Section titled “Backup”The database is a single SQLite file at data/redfish.db. The simplest backup is to copy the data/ folder while the application is stopped, or to snapshot it at the storage layer. Keep REDFISH_SECRET_KEY with the backup, since the sending-profile secrets are encrypted with it.
Continuous backup with Litestream
Section titled “Continuous backup with Litestream”SQLite runs in WAL mode, which lets Litestream replicate data/redfish.db continuously to S3-compatible storage. This is optional.
-
Set the
LITESTREAM_*variables in.env(see Configuration). Secrets are never committed. -
Start the service:
Terminal window docker compose --profile backup up -d litestream
Restore
Section titled “Restore”With the application stopped and the database absent:
docker compose --profile backup run --rm --entrypoint litestream litestream \ restore -config /etc/litestream.yml -o /app/data/redfish.db /app/data/redfish.dbTest the restore from time to time: restore to a temporary path (-o /app/data/restore-test.db) and check it with sqlite3 ... "pragma integrity_check".