T2 follow-up: comail/crawler provisioning for per-domain ZDS env
closedFollow-up discovered during T2 implementation (bug 2f8a863, verified in final review).
Sovrnd reads SOVRN_PDS_COMAIL_API_KEY / SOVRN_PDS_COMAIL_DID / SOVRN_PDS_CRAWLERS via the new PDSConfig bindings (config.go), but nothing sets them: sovrn.toml.j2 has no [pds] section and no sovrn_comail_* vars exist anywhere in deployment/ (fleet vault generator mints only OIDC + OAuth keys). email_from / operator_handle resolve via viper defaults; plc_directory is intentionally optional.
Current behavior (by design, tested): empty comail/crawlers fail per-record RenderEnv validation loudly and Reconcile skips that unit’s env — degraded, not silent. So no cell can serve a domain until this lands.
Scope:
- Storage decision: fleet-shared (vault-shared.yml) vs per-cell (host_vars/<cell>/vault.yml) for comail key/DID; crawler list is non-secret (group vars).
- Recipe support alongside provision-shared-secrets (fleet) or provision-zds-secrets (per-cell) for generation + structural asserts.
- Group-vars (sovrn_comail_*-shaped) + [pds] block in sovrn.toml.j2 (or env-file equivalent for sovrnd).
- First-cell converge proving Reconcile renders env end-to-end with real values.
Acceptance: just update <cell> --tags zds,sovrnd on a real cell renders a complete per-domain env (no validation skips) and the domain’s ZDS starts healthy behind Caddy.
3 Comments
PDS mail via Resend (replacing comail) — Implementation Plan
Goal: Replace comail with one fleet-shared Resend API key so
just update <cell> --tags zds,sovrndrenders complete per-domain ZDS envs withZDS_MAIL_PROVIDER=resend.Architecture: Hardcode
resendinRenderEnv(same ascomailwas hardcoded — no provider knob, YAGNI).PDSConfigswapscomail_api_key/comail_didforresend_api_key. Secret travels vault-shared.yml →sovrn.toml [pds]value (0640 root:sovrn) →SupervisorConfig→ per-instance<slug>.env(0600). Non-secrets via group vars.Tech Stack: Go (viper config, pdslifecycle), Ansible (sovrnd/sovrn_secrets roles, vault), ZDS 0.3.1, Resend API.
Review verdict (why Resend works): Pinned ZDS 0.3.1 supports
--mail-provider comail|resend+ZDS_RESEND_API_KEY(verified viazds --help, binary strings, upstreamdocs/operations.md). Resend needs only key + verified sender domain — strictly simpler than comail (key + DID). Decisions: replace comail entirely (no dual support); fleet-shared key invault-shared.yml; sender[email protected]; crawlers staybsky.network,vsky.network(vsky proxies requestCrawl to all known public relays — no need to enumerate atproto.at/relays).Task 0: Operator pre-req (no code, out-of-band, blocks Task 4)
notify.sovrn.atin Resend — Resend dashboard → Domains → addnotify.sovrn.at→ publish TXT/SPF/DKIM → wait for Verified.secrets add resend_pds_api_key. Never paste into chat or group vars.Task 1: Go — replace comail with Resend (TDD)
Files: - Modify:
config.go(PDSConfig, setDefaults, bindEnv) - Modify:pds_supervision.go(wire ResendAPIKey) - Modify:internal/pdslifecycle/env.go(EnvParams, RenderEnv, validateEnv, secretKeys) - Modify:internal/pdslifecycle/supervisor.go(SupervisorConfig, ActivationParams, envParamsForRecord, Activate) - Modify:pds_supervision_test.go,internal/pdslifecycle/env_test.go,supervisor_test.go,reconcile_env_test.go,testdata/env-basic.goldenstatic env valuessubtest inpds_supervision_test.goto setSOVRN_PDS_RESEND_API_KEY=re_test123and expectcfg.PDS.ResendAPIKey+EmailFrom == "[email protected]".go test -run 'TestPDSSupervisionEnvGate/static' . -v, expect FAIL.config.go—PDSConfig{Supervision, ResendAPIKey, EmailFrom, Crawlers, OperatorHandle, PLCDirectory}; default[email protected]; bindEnvpds.resend_api_key(drop comail keys).env.go/supervisor.go/pds_supervision.go—RenderEnvemitsZDS_MAIL_PROVIDER=resend+ZDS_RESEND_API_KEY, dropsZDS_COMAIL_*; validatorresend api key is required; secretKeys swap; supervisor config/activation params swap.SOVRN_INTEGRATION=0 go test ./...— expect PASS.jj commit -m "feat(pds): send PDS mail via fleet-shared Resend key" <files>.Task 2: Deployment —
[pds]block + fleet-shared vault + assertsFiles: - Modify:
deployment/roles/sovrnd/templates/sovrn.toml.j2(append[pds]) - Modify:deployment/inventory/group_vars/all/sovrn.yml(sovrn_pds_*vars) - Modify:deployment/roles/sovrn_secrets/tasks/main.yml(assert,no_log) - Operator:deployment/inventory/group_vars/all/vault-shared.yml(ciphertext only)[pds]block — supervision/resend_api_key/email_from/crawlers/operator_handle/plc_directory.sovrn_pds_supervision: true,sovrn_pds_email_from: "[email protected]",sovrn_pds_crawlers: "https://bsky.network,https://vsky.network", operator/plc defaults, vault-var indirection for the key.sovrn_secretsassert —sovrn_pds_resend_api_keydefined, non-empty,^re_[A-Za-z0-9_-]+$,no_log, gated on supervision.ansible-vault editvault-shared.yml, add key, re-encrypt, commit ciphertext (separate operator commit).jj commit -m "feat(deploy): wire fleet-shared Resend key into sovrnd [pds]" <files>.Task 3: Docs touch-ups
_precheckneeds nothing (shared-vault-encrypted check already covers the fleet file).provision-shared-secretsdocstring that Resend keys are pasted manually (relay-style), not generated.Task 4: First-cell converge — DEFERRED until smoke-test host is provisioned
Implementation complete (Tasks 1-3). Task 4 deferred pending smoke-test host.
Deferred Task 4 (first-cell converge + Resend mail-path proof) moved to tracking issue 28c7d1f, consolidated with the relay split verification from 9bec37e. Both blocked on smoke-test server provisioning.