sovrn

Experimental AT protocol email hosting

Colmena keys via secrets decrypt; retire ansible-vault

closed
#4ce0593 opened by agent Sep 29

Replace the ansible-vault files and the Python vault scripts (deployment/scripts/*, _vault_lib.py, vault-password-client) with Colmena keys fed by the secrets tool (~/projects/secrets).

Sketch: - deployment.keys.<name>.keyCommand = [ "secrets" "decrypt" "sovrn/<scope>/<name>" ] with user, group and permissions per key. - Naming scheme in the secrets store: - sovrn/shared/*: OIDC key, OAuth attestation key, backup-domains token, relay API/SMTP, Resend key - sovrn/<host>/*: Stalwart recovery admin, sovrnd service-account password, R2 keys, ZDS secrets - Keys vanish on reboot: Colmena’s default destDir is /run/keys (tmpfs), so an unattended reboot leaves Stalwart, sovrnd and Litestream without secrets. Recommend a persistent root-only destDir (e.g. /var/lib/sovrn/keys) for the mail boxes. Decide. - Order consumers on Colmena’s generated <name>-key.service units (After=/Wants=). - Pre-generate the Stalwart recovery admin and the sovrnd service-account password with secrets encrypt, instead of generating them on the box and fetching them back. That deletes ensure-bootstrap-secrets, persist-bootstrap-secrets and the sentinel/slurp logic. - One-off migration: decrypt the current vault files and re-encrypt each value into the store. - Decide who deploys (laptop vs CI) and how CI gets an age identity.

Done when a scratch box receives its keys with the correct owner and mode, and they survive a reboot (if we choose a persistent destDir).

1 Comment

agent 4ecbe50 Sep 30

Progress: ansible-vault retired; Colmena keys come from secrets

Decisions (with the user) - Identity: one age identity. CI gets a copy of the operator’s ~/.config/age/identity.txt; there are no per-environment recipients. - Store location: SECRETS_DIR=~/projects/data. It becomes a git repo on the private server before CI is integrated. - CI scope: tests, plus automated deploys of staging (mx99.eu.sovrn.at) only. Production is deployed from the workstation after staging looks right. - Key directory: keys land in /var/lib/sovrn/keys (persistent), not /run/keys.

Migration (done, verified by round-trip) - Store naming: sovrn/shared/<file> and sovrn/hosts/<fqdn>/<file>. Each value is stored in its on-host form; the OAuth attestation key is base64-decoded to 32 raw bytes. - relay-api-secret, pds-resend-api-key and metrics-password are symlinks to the existing smtp2go_api_key, resend_api_key and sovrn_metrics_password entries. Those entries carry a trailing newline. sovrnd trims when it reads them; check vmagent and Alertmanager when their roles are ported (dd3c64a, a6edca3). - mx99: 8 per-host values migrated. sovrnd-stalwart-password and ssh_host_ed25519_key are not generated yet; new-host / gen-host-secrets will create them.

Nix - nix/modules/secrets.nix adds the sovrn.secrets.<name> = { scope = shared|host; user; group; mode; } option. It maps each entry to deployment.keys with keyCommand = secrets decrypt <storePath>. Root-owned keys upload pre-activation; keys owned by service users upload post-activation, because those users are created during activation. - The role modules declare their keys: cell 16, relay 3, metrics 2.

Justfile.nix - gen-host-secrets HOST evaluates the host’s declared host-scoped secrets and generates the missing ones (SSH host key, 32-hex passwords, ZDS secrets, secp256k1-valid PLC key). It lists the R2 tokens for manual entry and never rotates existing values. - check-secrets [HOST] is a pre-deploy gate that deploy depends on. - new-host now runs gen-host-secrets. - New test and ci-staging recipes (test → known-hosts → deploy mx99). - Tested against a copy of the store: an existing host (idempotent), a fresh cell, and the check failing and then passing.

Removed - The vault files and deployment/scripts/ (Python vault tooling, test-deploy). - vault_password_file from ansible.cfg. - The Ansible run/provision recipes in the Justfile. - ansible and apacheHttpd from devenv.

Found: deployment/playbooks/.local/stalwart-secret.sovrn.at was committed in plaintext (in the history of origin/main). It is a Stalwart recovery credential for the old sovrn.at box and does not match mx99’s current value. Removed from the tree; the history still contains it.

Open: a real deploy that verifies owners, modes and reboot survival (needs a NixOS host from bc4b527).