Data control
Where every store lives on disk, what each retention window governs, how backups work, and how to move an instance to another host.
Self-hosting SendSets means holding mailbox credentials, message bodies and contact records on your own disk. This page is the whole answer to where that data is, how long it stays, and how you get it off this machine.
Three questions, and they have different answers:
| Question | Answer |
|---|---|
| Where is it? | Six stores, each pointed at by one variable |
| How long does it stay? | Retention windows, all editable in the admin panel |
| How do I move it? | sendsetsctl backup for the instance, workspace export for one workspace |
Where the data sits
Every store's location is one variable in .env. Compose reads a source starting with / as a bind mount and anything else as a named volume, so the same variable covers both and there is no second compose file.
| Variable | Holds | Default |
|---|---|---|
SENDSETS_PG_DATA | Postgres: organizations, users, mailboxes (credentials sealed), contacts, campaigns, the audit trail | <data root>/postgres |
SENDSETS_BLOBS | Message bodies, attachments, avatars, logos and email body images | <data root>/blobs |
SENDSETS_NATS_DATA | The event bus's JetStream state | <data root>/nats |
SENDSETS_REDIS_DATA | Cache and rate-limit counters. Disposable | <data root>/redis |
SENDSETS_WORKER_STATE | A worker's own id and sync cursors. Disposable | <data root>/worker |
SENDSETS_UPDATER_STATE | The last update job's log. Disposable | <data root>/updater |
An install from install.sh points all six under one data root, so /opt/sendsets/data is the whole of it and you can rsync that path. A clone-and-build install keeps Docker named volumes unless you set the same variables.
Only the first two carry anything you cannot rebuild. The other four are state a fresh container reconstructs, which is why a backup does not include them.
Encryption
Two keys, and they are not interchangeable.
| Key | Opens | Read by |
|---|---|---|
CREDENTIALS_ENCRYPTION_KEY | Mailbox SMTP and IMAP credentials, and OAuth tokens | Backend and workers, without an organization context |
KMS_LOCAL_MASTER_KEY | The per-organization data keys, which in turn open everything else | Backend and consumer |
Both are unrecoverable. A database backup without them restores an instance whose mailboxes authenticate against nothing, and there is no way back from that. They are in .env, and install.sh also writes them to keys-backup.txt next to the install, which is on the same disk as the database and therefore not a backup either. Copy them somewhere else.
External stores
Postgres, Redis and blob storage each accept an external target, set at install time or by editing .env:
PRIMARY_DB=postgres://user:pass@db.internal:5432/sendsets?sslmode=require
REDIS=redis://cache.internal:6379
BLOB_PROVIDER=s3
BLOB_BUCKET=sendsets
AWS_ENDPOINT_URL_S3=https://<account>.r2.cloudflarestorage.comFilesystem blobs stop working when workers run off-host
A remote worker writes bodies to its own disk, so the dashboard on the control-plane host finds nothing. Any deployment with workers on more than one machine needs S3-compatible blob storage. This is the one storage choice that is not just a preference.
What is kept, and for how long
Every window below lives in the database, not the environment, and is edited under Instance > Configuration > Settings in the admin panel. A sweep runs a few times a day and reads the current value on every pass, so a change takes effect without a restart.
Mailbox import and sync
| Setting | Default | Range | What it governs |
|---|---|---|---|
sync.backfill_days | 90 | 1 to 730 | How far back the initial import reaches when a mailbox is connected, newest first |
sync.backfill_messages | 5,000 | 1 to 100,000 | The most messages that import stores per mailbox |
sync.daily_messages_per_mailbox | 2,000 | 1 to 100,000 | New mail one mailbox may store per UTC day |
sync.daily_messages_per_org | 25,000 | 1 to 2,000,000 | New plus imported mail across one workspace per UTC day |
Mail over a daily budget is deferred, never dropped: the provider cursor is held and the mail is re-offered on the next pass. Replies to the mailbox's own outreach ride a separate budget of the same size and keep landing regardless.
Automatic inbox tagging
Automatic inbox tagging is off unless both TYPESAFE_API_KEY and INBOX_TAGGING_ENABLED=true are set. When enabled, the consumer sends the inbound subject and bounded plain-text body, plus the previous outbound message in the thread when one exists, to TypeSafe for classification. The result and raw probabilities are stored in inbox_tag_results. Completed results travel with the Inbox group in a workspace export; temporary processing claims do not. See Automatic inbox tagging.
Event history
| Setting | Default | What it holds |
|---|---|---|
retention.engagement_event_days | 365 | Per-event open and click logs: client, device, approximate location |
retention.form_event_days | 180 | Form funnel events: views, starts, field-level drop-off |
retention.audit_log_days | 90 | The audit trail: actor, IP address, user agent, change payload |
Each is between 1 and 3,650 days. These are the three settings a retention or privacy policy applies to, because each window is also how long the personal data in that log is held.
None of them change a number anyone reads. Campaign progress keeps its own summary of opens and clicks that outlives the per-event log, so counts, filters and branching are unaffected by shortening any of these. What gets shorter is what a contact's timeline can show, how far a funnel report reaches, and how far back an admin can audit.
Shortening a window deletes on the next sweep
There is no grace period and no copy. Take a backup first if you are not sure.
The minimal retention preset in the admin panel and in the installer sets all three to 30 days.
Warmup standing by address
Warmup keeps the standing of every penalised mailbox (score, state, and any block) against its address as well as its row, so removing the mailbox or dropping out of the pool and coming back does not start it clean. The copy is cleared as soon as the mailbox recovers, and once the mailbox is gone it is kept for 90 days after the block ends or after the removal, whichever is later; a block that requires review is kept until an operator lifts it. A mailbox in good standing has no such copy. The window is fixed rather than a setting, and the rows are purged by the same sweep.
Pending inbox verification
Mail whose warmup verification is temporarily unavailable waits in the unibox_pending_emails Postgres table. It holds the sync event, including message metadata and bounded searchable body text, until verification succeeds or the mailbox is deleted. The consumer retries it automatically after restarts. Full bodies remain in blob storage. This pending queue is included in instance backups but excluded from workspace exports, whose destination re-syncs provider mail.
Disconnecting a mailbox
Disconnecting a mailbox, or deleting the workspace holding it, removes its rows from Postgres straight away through the database's own cascades. Two things cannot be done in that instant and are queued instead, then worked off within about a minute:
| Owed | What happens |
|---|---|
| The OAuth grant | A Gmail mailbox's refresh token is sent to Google's revocation endpoint, which invalidates it and removes the instance from the customer's third-party access list. Microsoft publishes no per-application revocation endpoint, so an Outlook mailbox's tokens are destroyed locally and the customer removes the app themselves |
| The message bodies | Everything the mailbox wrote under users/<user>/emails/<mailbox>/ in the blob store is deleted, whether that is a filesystem path or an S3 bucket |
While an erasure is outstanding its row holds a copy of the mailbox's OAuth refresh token, still sealed under CREDENTIALS_ENCRYPTION_KEY, because the row it was read from goes with the mailbox and the token is what the revocation sends. It is destroyed with the rest of the row when the erasure finishes.
The queue is the mailbox_erasures table, and it holds only work still owed: a finished erasure is removed rather than kept, because the row names the address the customer asked to have forgotten. A failure is retried with a widening backoff, capped at six hours, and never abandoned, so an unreachable provider or store delays the erasure instead of losing it. Anything still outstanding after a day is reported by the mailbox_erasure job, visible under scheduled jobs in the admin panel.
This runs on the control plane, which is the only place holding both the credential key and a credential for the blob store. A worker cannot do it: it signs one object at a time through the control plane and holds no bucket credential at all.
Set them at install time
The installer writes the answers into .env as one document, applied on the first boot of a fresh database:
SENDSETS_SETTINGS_BOOTSTRAP={"sync":{"backfill_days":30},"retention":{"audit_log_days":30}}It is read only while the settings row has never been written. From the first save in the admin panel onwards the panel is authoritative, so leaving the line in .env never undoes a later edit.
Backups
sendsetsctl backup writes one bundle holding the three things that only restore together:
- the database, as a
pg_dump - the blob root, when blobs are on the filesystem
- the encryption keys, unless you pass
--no-keys
docker compose -p sendsets exec backend sendsetsctl backup --out /data/blobs/sendsets.tar.gz
docker compose -p sendsets cp backend:/data/blobs/sendsets.tar.gz ./sendsets.tar.gz \
&& docker compose -p sendsets exec -T backend rm -f /data/blobs/sendsets.tar.gz/data/blobs is a hand-off, not a destination: it is the one path the container and the host both see. The && matters twice over. backup leaves its own output out of the archive, but a bundle left there is swept into the next run, so it has to be deleted; and a cp that failed must not be followed by deleting the only copy that exists.
The bundle is written 0600 and holds every mailbox credential on the instance plus the keys that open them. Treat the file as you would the instance itself.
An install from install.sh --wizard can schedule this for you: backup.sh next to the install, a systemd timer, a retention count, and an optional aws s3 cp to somewhere off the host. A backup that only exists on the machine it backs up is not one.
Restore
On the destination host, with the same keys in place:
docker compose -p sendsets exec backend sendsetsctl restore --file /data/blobs/sendsets.tar.gz
docker compose -p sendsets restartThe restore empties the schema and replays the dump, so it replaces everything currently on that instance and asks you to type restore first.
Before it does anything it compares the bundle's CREDENTIALS_ENCRYPTION_KEY and KMS_LOCAL_MASTER_KEY against the destination's and refuses to continue when they differ, printing the two lines to put in .env. That check is the point of the command: without it a restore looks like it worked and every mailbox fails to authenticate days later, with no error that names the cause.
Moving an instance
Two ways, and they answer different questions.
The whole instance, to a new host
Every workspace, every user, the platform admins, the API keys.
Install SendSets on the new host
curl -fsSL https://sendsets.com/install.sh | sh -s -- --host <new-hostname>Put the old keys in the new .env
Copy CREDENTIALS_ENCRYPTION_KEY and KMS_LOCAL_MASTER_KEY from the old install, then recreate the containers so they take:
docker compose -p sendsets up -dRestore the bundle
docker compose -p sendsets exec backend sendsetsctl restore --file /data/blobs/sendsets.tar.gz
docker compose -p sendsets restartCheck it
docker compose -p sendsets exec backend sendsetsctl statusMailboxes should be connected, not needing a reconnect. If they need one, the keys did not match.
The rsync alternative works too and is sometimes simpler, as long as the stack is stopped first: a running Postgres data directory copied file by file is not a consistent snapshot and can restore as a corrupt cluster. Stop it, copy the data root and the .env to the new host, start it there. It moves the same bytes; the bundle exists because it is the version that survives a different host layout, a different Postgres, a live instance, and a partial copy.
One workspace, to another instance
Workspace export and import moves a single organization's data between two running instances, re-sealing its secrets for the destination's keys. That is the per-customer tool; the bundle here is the instance-level one. They are not interchangeable: a bundle cannot be applied to one workspace, and a workspace archive cannot restore an instance.
Outbound calls
A self-hosted SendSets makes no outbound call of its own except one, and it is off with a single setting.
| Call | When | Turn it off |
|---|---|---|
| GitHub releases API | Every 30 minutes, to tell the admin panel a newer version exists | UPDATE_CHECK_ENABLED=false |
Everything else is you: mail through the mailboxes you connect, DNS lookups for the domains you check, and whatever integrations you configure. There is no telemetry, no phone-home, and no license check.
Error reporting
An instance reports errors nowhere unless you point it somewhere. Every service reads its own credentials and none of them ship with any, so a default install sends no crash, no stack trace and no browser error to anybody, including us.
Two backends are supported and you choose either, both, or neither. PostHog is the default one; Sentry works exactly as it always has.
| Service | PostHog | Sentry |
|---|---|---|
| Backend, consumer, worker | POSTHOG_KEY | SENTRY_DSN |
| Forms service | POSTHOG_KEY | SENTRY_DSN |
| Tracking service | POSTHOG_KEY | SENTRY_DSN |
| Realtime service | POSTHOG_KEY | SENTRY_DSN |
| Dashboard and admin containers | SENDSETS_POSTHOG_KEY | SENDSETS_SENTRY_DSN |
| Public form pages | SENDSETS_POSTHOG_KEY on the forms service | SENDSETS_SENTRY_DSN on the forms service |
POSTHOG_HOST decides which PostHog receives them, including one you run yourself, and POSTHOG_ERROR_TRACKING=false turns error reporting off while leaving the key configured for the product analytics below. A Sentry DSN names whatever Sentry Cloud project, self-hosted Sentry or Sentry-compatible server you like.
Leave them unset, which is the default the installer writes, and nothing leaves the process: there is no host to contact and nothing to opt out of. Errors still reach that service's own log, as they always did. The dashboard, the admin panel and form pages go further and load no reporting code at all, so there is not even a script to block.
What a reported error carries is the exception, its stack trace, the service, the environment and the build, plus enough context to answer it:
- Which workspace and user. A browser exception carries the organization id and user id of whoever hit it, and a server-side one carries them for the request that panicked, along with its route and request id. That is what makes "this customer says campaigns are broken" a search rather than a guess. On the server side that is a property and nothing more: no person is identified. In the dashboard and the admin panel, with a browser key set, the signed-in user is identified to PostHog (a person profile with their email and name, and an identifier in local storage and a cookie), so the exception belongs to that person; see usage analytics below.
- What led to it. The last routes visited and the last API calls that failed, with their method, path, status and request id.
- Nothing else about the request. No headers, no bodies, no query strings. Session replay exists only in the dashboard and the admin panel, only with a browser key set, and
SENDSETS_POSTHOG_SESSION_REPLAY=falseturns it off; the marketing site and public form pages are never recorded.
Server-side reports that belong to no request, a scheduled job or a worker, are attributed to the process that raised them and to nobody.
The sendsets CLI runs on your own machine and reports nowhere, ever. It has nothing to set.
Usage analytics
There are none. A self-hosted instance loads no analytics script and sends no usage data, not aggregated, not anonymised, not "to help us improve the product".
The hosted service at sendsets.com does measure its own marketing site, dashboard and admin panel, including session replay in the two apps, and the code for that ships in the same images you run. It is inert without a key: the installer never asks for one, the .env template does not mention it, and a build without one contains no analytics script for you to block. If you want to check, grep posthog the served assets of your own dashboard.
If you do want product analytics on your own instance, the variables are on configuration and they accept a self-hosted PostHog. The server-side POSTHOG_KEY carries the backend's error tracking and its product events; the browser-side SENDSETS_POSTHOG_KEY is what turns on the dashboard's and the admin panel's analytics, session replay and browser errors, and SENDSETS_POSTHOG_ERROR_TRACKING and SENDSETS_POSTHOG_SESSION_REPLAY switch those last two off separately. That is your decision to make, and nothing about it points at us.
See also
- Install: the wizard that asks all of this up front
- Configuration: every variable and every database-backed setting
- sendsetsctl:
backup,restore, and the operator commands - Workspace export and import: the per-workspace story
sendsetsctl
The CLI for a SendSets instance. The operator commands for accounts, health and recovery, and the API commands that let scripts and AI agents drive campaigns, contacts, mailboxes and the inbox with an API key.
Configuration reference
Every environment variable SendSets reads, what it does, its default, and whether changing it needs a restart.