sovrn

Experimental AT protocol email hosting

[C2] stalwartsync: Registry-client spike — schema, object lifecycle, two-phase reload

closed
#24dcaa6 opened by agent Aug 24

Goal

Prove the control plane ↔ Stalwart integration works exactly as designed before any product features: a Go package internal/stalwartsync that authenticates to Stalwart and performs the full provisioning object lifecycle over Registry/set|get|query on POST /jmap, verified against the C1 harness.

Context: docs/05-stalwart-integration.md (§1 API surface, §2 mechanics, §3 credentials, §6 ops), docs/03-provisioning.md §6.

Tasks

  • [ ] JMAP envelope client (request/response, error mapping incl. trc error shapes)
  • [ ] Schema discovery: resolve /api/schema/{sha256} redirect → download gzipped JSON → codegen or runtime typing decision (ADR note)
  • [ ] Auth as service account (basic/bearer per what C1 bootstrap provisions)
  • [ ] Domain lifecycle: Registry/set Domain create → get → update → delete; verify DKIM DNS-publication task fires
  • [ ] Account lifecycle: Registry/set Account with quotas + EmailAlias list → get → alias add/remove → delete → poll async destruction task (TaskDestroyAccount) to completion
  • [ ] ApplySettings() helper: settings object set + explicit Action::ReloadSettings, verifying hot reload took effect
  • [ ] Integration tests in CI against C1 (dockerised)

Acceptance criteria

  • Full lifecycle tests green in CI against real container
  • Replaying every write twice produces identical end state (idempotency proof for crash-retry safety)
  • Masked-password path proven: updating account fields does NOT require resending secrets

Risks this should surface

  • Registry API error semantics/edge cases not visible from reading source
  • Async destruction polling semantics
  • Whether schema drift between Stalwart versions breaks our client (feed the CI schema-diff job decision)

4 Comments

agent 264fd9c Aug 25

Provisioning packages: internal/jmap + record-kind lifecycle packages

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development or superpowers:executing-plans. Steps use - [ ] tracking.

Goal: Prove control-plane ↔ Stalwart integration: a pure JMAP plumbing package (internal/jmap), a Stalwart-Registry semantics layer (internal/registry), and one package per provisioning record kind (tenant, domain, account, settings) whose public API reads like the domain actions they perform — ready to be driven by AT proto record handlers later.

Architecture: Three layers, strict downward deps only:

jmap      RFC-8620 protocol plumbing — Client{Endpoint}, iterative Request builder,
          generic Send[T], typed spec errors            (no Stalwart knowledge)
  ▲
registry  "Stalwart is assumed" lives here — x:<Type>/<fn> dispatch, accountId:"a",
          SetResult/SetError parsing, schema-hash pin, env+secret-file config
  ▲
tenant · domain · account(+password) · settings     policy & orchestration per record kind

Every write path is exposed as both strict Create (typed ErrAlreadyExists) and idempotent Ensure (query→diff→set-or-noop) — crash-retry safety per acceptance criteria. Integration tests env-gated against the C1 harness (devenv up).

Tech stack: Go 1.26 stdlib only. Libraries evaluated and rejected (recorded in doc.go): rockorager/go-jmap (mature but session-centric, no Registry methods), pr0ton11/jmap-go (has stalwart sub-package but v0.x/single-maintainer/zero importers, wrong ergonomics).

Decisions locked with owner (2026-08-25): - hostname = mail domain being managed; connection via explicit (ctx, c *jmap.Client, ...) params, no globals - Both strict Create and idempotent Ensure exported - Tenancy required (decision A): domain.Create(ctx, c, name, tenantID) — tenant assignment mandatory & positional; infra/service zones live under a designated platform tenant (tenant.PlatformTenantName), never in the global namespace. Account likewise takes required tenantID (server FK-cross-checks account↔domain agreement) - Schema typing = hand-typed structs + pinned /api/schema hash (ADR 0003); deliverable test-only (no CLI); IMAP-wire login stays in C5

Verified ground truth (Stalwart @ rev 2add611e + C1 comments c7cfd93 / 345b461)

Fact Source
Registry methods wire as x:<ObjectType>/<get\|set\|query> under capability urn:stalwart:jmap; envelope field methodCalls; accountId:"a" accepted jmap-proto/src/request/method.rs:239,358
Method-level error entry: ["error",{type,description},callId]; types: forbidden invalidArguments serverFail unknownMethod unsupportedFilter requestTooLarge stateMismatch anchorNotFound accountNotFound … jmap-proto/src/error/method.rs:80–160
Set-level errors: notCreated/notUpdated/notDestroyed maps {type, description?, properties?} jmap-proto/src/error/set.rs:127–146
Recovery-admin Basic auth ⇒ full admin; dev secret at /data/service-secret (user sovrn-admin) authentication.rs:90–127, serve-stalwart.sh:41–51
Secondary credentials cannot be inline on Account create; app passwords are standalone x:AppPassword/set scoped by target accountId; server generates secret, returned once as created.<id>.secret principal.rs:322, account.rs:346–470
Masked-password placeholder "****" preserves stored secret on credential update principal.rs:109–176, prelude.rs:58
Domain create with dkimManagement:{@type:"Automatic"} schedules DKIM task; DNS publication additionally requires a DnsServer object (absent in dev) mapping/domain.rs:100–139
Action::ReloadSettings = x:Action/set create {"@type":"ReloadSettings"}; actions reject update/destroy mapping/action.rs:38–60, structs_impl.rs:549
Account destroy ⇒ scheduled TaskDestroyAccount (OSS: immediate); completion must be polled principal.rs:441–475
Delete integrity: destroying an object with linked members ⇒ CannotDeleteLinked ⇒ notDestroyed store/src/registry/write.rs:372–385
Tenancy: Tenant{name,…} free-form label, top-level; Domain.member_tenant_id: Option<Id> + Account.member_tenant_id; FK enforcement rejects nonexistent tenant refs AND account→domain tenant mismatch at write time structs.rs:5802,2713,6200; write.rs:179,206–231; structs_impl.rs Domain index foreign_key(Tenant,…)
Tenant-less objects land in platform-global namespace: visible to top-level tokens only; invisible to every tenant-scoped token query.rs:174–176, set.rs:295–320
Tenant quotas stored but NOT enforced for top-level-token writes (only tenant-scoped callers) principal.rs:370–433
GET /api/schema/{x} (auth required) redirects to hash URL; pin @ rev: 25gnm1_xB9bPk-puX9VTFMP1W5V-1muUWuK964-_uEw http/src/api/mod.rs:97–115
Objects serialize @type tags: Account "User", DKIM/DNS mgmt "Automatic"/"Manual"; quota keys enum strings (maxAccounts maxDiskQuota maxEmails …) structs.rs:12–15, enums.rs:2660,2784

File structure & API contract

go.mod                                  module sovrn, go 1.26
devenv.nix                              += pkgs.go pkgs.golangci-lint
docs/adr/0003-stalwart-schema-typing.md ADR (schema decision)
docs/adr/0004-required-domain-tenancy.md ADR (decision A)
internal/jmap/
  client.go    Client{ Endpoint string; HTTP *http.Client; Auth Authenticator }
               type Authenticator interface{ Authorize(*http.Request) }
               Basic{User, Secret}; New(endpoint string, auth Authenticator) *Client
  envelope.go  wire types: Request{Using []string; methodCalls [][]any} (custom marshal),
               Response{MethodResponses []RawResponse; SessionState string},
               RawResponse{Name string; Args json.RawMessage; CallID string}
  request.go   NewRequest(capabilities ...string) *Request   // defaults core+stalwart URNs
               (*Request) Invoke(method string, args any) (callID string)  // chainable builder
  send.go      func Send[T any](ctx context.Context, c *Client, r *Request) (T, error)
               // POSTs ONE envelope; decodes the response matching r's FINAL callID into T;
               // ["error",…] entries become *Error. SendAll returns []RawResponse for
               // multi-call envelopes. Decode[T](RawResponse) (T, error) exported too.
  errors.go    Error{Kind Kind; Description string}   // method-level, implements error
               const (KindForbidden, KindInvalidArguments, KindServerFail, …)
               SetError{Type string; Description string; Properties []string}
internal/registry/
  consts.go    CapabilityURN="urn:stalwart:jmap"; MASKED_PASSWORD="****"; accountID="a"
  options.go   Options{URL, Username string; SecretFile, Secret string}; FromEnv() (Options, error)
               // env: STALWARTSYNC_URL (def http://127.0.0.1:8080), STALWARTSYNC_USERNAME
               // (def sovrn-admin), STALWARTSYNC_SECRET_FILE (def $SOVRN_DATA_DIR/stalwart/service-secret)
  registry.go  Get(ctx,c,objType,ids,props)/Query(ctx,c,objType,filter Filter)/
               Set(ctx,c,objType,create,update,destroy)/Destroy(...)
  result.go    SetResult{ Created map[string]json.RawMessage; NotCreated map[string]SetError;
               Updated,NotUpdated map[string]SetError; Destroyed,NotDestroyed []string }
               Filter{Attribute,Operator,Value string}  // operator "equals"
  schema.go    PinnedSchemaHash const; VerifySchema(ctx,c) error
internal/tenant/
  tenant.go    const PlatformTenantName = "platform"
               type Tenant struct{ ID, Name string; Quotas map[string]uint64 }
               Create(ctx,c,name string, quotas ...) (Tenant, error)        // ErrAlreadyExists
               Ensure(ctx,c,name string, quotas ...) (Tenant, changed bool, error)
               Get / Destroy(ctx,c,id)
internal/domain/
  domain.go    type Domain struct{ ID, Name, TenantID string; DKIMAuto bool }
               Create(ctx,c,dnsName,tenantID string) (Domain, error)  // tenancy REQUIRED
                 // nonexistent tenant ⇒ typed ErrTenantNotFound (server InvalidForeignKey)
               Ensure(ctx,c,dnsName,tenantID string) (Domain, changed bool, error)
                 // absent → create; matching → noop; tenant-less/foreign → ErrAssigned{Current}, zero writes
               AssignTenant(ctx,c,domainID,tenantID string) error      // explicit reassignment
               ListForTenant(ctx,c,tenantID) ([]Domain, error)
               Delete(ctx,c,id)                                        // notDestroyed while accounts linked
internal/account/
  account.go   type Account struct{ ID, Host, User, TenantID string; Quotas…; Aliases []Alias }
               Create(ctx,c,host,user,tenantID string, opts…)          // tenancy REQUIRED
               Ensure(...) same ErrAssigned semantics
               UpdateMasked(ctx,c,id,fields,cred CredentialRef)         // secret:"****"
               Destroy(ctx,c,id) / AwaitDestroy(ctx,c,id,timeout)
  password.go  IssuePassword(ctx,c,id,description string,ttl) (secret string, error)
               // x:AppPassword/set {accountId:id, create:{apX:{description,expiresAt}}}
               // returns server-generated one-time secret from created.apX.secret
internal/settings/
  settings.go  Apply(ctx,c,objType string, patch map[string]any) error
               // phase 1: x:<obj>/set update; phase 2: Action create {"@type":"ReloadSettings"}
internal/integration/
  story_test.go   env-gated journey test (skips without STALWARTSYNC_URL)

Import graph: jmap ← registry ← {tenant,domain,account,settings}; story test imports all four record packages. Future AT proto mapper imports record packages only.

Tasks

Task 1: Scaffold

  • [ ] go.mod (module sovrn, go 1.26); devenv.nix += pkgs.go pkgs.golangci-lint; doc.go files stating package contracts + rejected-library note.
  • [ ] Verify: go build ./...; commit feat: go module scaffold.

Task 2: internal/jmap — envelope + typed errors (TDD)

  • [ ] Failing tests first (httptest fake server): round-trip envelope, method-error entry → *Error{Kind}, HTTP status mapping (401→ErrUnauthorized, 500→serverFail), multi-call Invoke/SendAll binding final callID.
  • [ ] Implement client.go/envelope.go/request.go/send.go/errors.go.
  • [ ] Green; commit feat(jmap): envelope client with generic Send and typed errors.

Task 3: internal/registry — semantics + config (TDD)

  • [ ] Failing unit tests w/ stub RoundTripper asserting exact payloads: Query filter shape, Set create/update/destroy, SetResult parse incl. notCreated SetError, Options.FromEnv precedence (Secret > SecretFile > error), schema redirect follow + drift error.
  • [ ] Implement consts.go/options.go/registry.go/result.go/schema.go.
  • [ ] Green; commit feat(registry): stalwart registry ops, config, schema pin.

Task 4: internal/tenant + internal/domain (TDD)

  • [ ] Payload tests: tenant create minimal body incl. quotas; duplicate Create → ErrAlreadyExists; Ensure replay → changed=false, zero mutations.
  • [ ] Domain: memberTenantId ALWAYS sent; Ensure wrong/absent tenant fixture → ErrAssigned with zero mutation calls; fresh-create w/ bogus tenant → typed ErrTenantNotFound from server set-error; AssignTenant patch payload; ListForTenant filter shape.
  • [ ] Implement both packages (Ensure algorithm: Query equals-name → Get → diff desired subset → noop | update-patch | create).
  • [ ] Green; commit feat(tenant,domain): lifecycle, required tenancy, Create/Ensure duality.

Task 5: internal/account + password (TDD)

  • [ ] Unit tests: account create {"@type":"User","name":"dave","domainId","memberTenantId","aliases":[…],"quotas":{…}}; Ensure ErrAssigned semantics; IssuePassword asserts x:AppPassword/set shape + extracts created.apX.secret; UpdateMasked emits credentials [{credentialId,secret:“****”}]; AwaitDestroy polls until notFound.
  • [ ] Implement; green; commit feat(account): lifecycle, masked updates, app passwords, async destroy.

Task 6: internal/settings (TDD)

  • [ ] Unit: Apply sends singleton update then Action create {"create":{"act0":{"@type":"ReloadSettings"}}}; fails if either phase errors. Property discovered live from x:Jmap/get during Task 8.
  • [ ] Green; commit feat(settings): two-phase ApplySettings helper.

Task 7: ADRs

  • [ ] docs/adr/0003-stalwart-schema-typing.md: hand-typed + pinned hash, revisit codegen >~10 object types.
  • [ ] docs/adr/0004-required-domain-tenancy.md: decision A — 1..1 mandatory assignment, platform-tenant convention, raw-registry escape hatch.
  • [ ] Commit docs(adr): 0003 schema typing, 0004 required domain tenancy.

Task 8: Integration journey (the spike proper)

Skip guard: no STALWARTSYNC_URL ⇒ t.Skip. - [ ] TestIntegrationProvisionUserStory, literal names, rerun-safe via Ensure: 1. Setup: registry.VerifySchema; tenant.Ensure(PlatformTenantName) 2. tenant.Ensure(“atproto”, quotas) → id; replay unchanged 3. domain.Ensure(“atproto.test”, t.ID) → changed; poll x:Task/get for DkimManagement task (log observed); replay → changed=false 4. Negative: domain.Ensure(“atproto.test”, otherTenant) → ErrAssigned; domain.Create(“fresh.test”,“bogus”) → ErrTenantNotFound (wire shape recorded); premature tenant.Destroy(atproto) → typed notDestroyed/CannotDeleteLinked 5. account.Ensure(“atproto.test”,“dave”,t.ID, quotas, alias) → replay unchanged 6. Anonymous GET /api/discover/[email protected] resolves 7. account.IssuePassword(daveID,“imap-device”,24h) → Basic auth AS [email protected] via jmap Core/echo succeeds (IMAP wire = C5) 8. Masked-path subtest: scratch account WITH Password credential → UpdateMasked description → old secret still authenticates 9. settings.Apply on benign JMAP knob → value reflected; re-apply noop 10. Full replay of steps 2–5 → all changed=false (crash-retry idempotency proof) 11. Teardown order enforced: account.Destroy+AwaitDestroy (<30s) → domain.Delete → tenant.Destroy; discover stops resolving - [ ] Findings recorded whichever way they fall: Tenant-on-OSS reality; query filter operator names accepted; DKIM task visibility window; app-password-auth-without-primary-password behavior; singleton-id conventions; exact invalidForeignKey SetError wire type; platform-tenant behaviour. - [ ] Run: devenv up, then STALWARTSYNC_SECRET_FILE=data/dev/stalwart/service-secret go test ./internal/integration -v. Commit test(integration): signup story journey.

Task 9: Close-out verification

  • [ ] go vet ./… && golangci-lint run && go test ./… green; integration suite green after sovrn-reset && devenv up.
  • [ ] Post findings comment to bug 24dcaa6 (Phase-0 protocol f89287c: surprises recorded first); leave open for owner review.
agent 2745dbc Aug 25

Progress checkpoint — Tasks 1–4 complete, paused for owner review — 2026-08-25

Plan (comment 264fd9c) executed through Task 4 via subagent-driven development; each task passed spec-compliance + code-quality review gates.

Commits (jj, oldest first)

Change Description
qulpwsty feat: go module scaffold + toolchain (git.kilimanjaro.io/sovrn; devenv gains go + golangci-lint; 6 package-contract doc.go files)
rxvwtrwz ci: selfci pipeline config (test+lint) — split out of jmap work
puplsvow feat(jmap): envelope client with generic Send and typed errors (27 tests)
svskvltx feat(registry): stalwart registry ops, config, schema pin (27 tests)
zzwlqknk feat(tenant,domain): lifecycle, required tenancy, Create/Ensure duality (31 tests)

API now real

jmap.Send[T] envelope client → registry.Get/Query/Set/Destroy/SetResult → tenant.Create/Ensure/Delete, domain.Create/Ensure/AssignTenant/ListForTenant/Delete. Decision A enforced: tenancy required & positional; ErrTenantRequired, precheck ErrTenantNotFound, conflict AssignedError{Current} with provably-zero writes on mismatch.

Design decisions made during execution

  1. KindNotFound→KindServerPartialFail rename (wire-faithful to method.rs); Decode hard-fails type-less error entries
  2. nil invocation args coerce to {} (RFC 8620 conformance); Error.HTTPStatus added for future retry policy
  3. registry.Query returns total → ListForTenant fails loud on pagination truncation
  4. Verb convention: Delete = synchronous removal (tenant/domain); Destroy+AwaitDestroy reserved for account’s async Stalwart teardown (Task 5)
  5. Tenant quota Ensure diff compares desired keys only (server-defaulted extras never re-patch)

Queued for Task 8 live verification

primaryKeyViolation/cannotDeleteLinked wire casing; tenant-on-OSS create; DKIM task visibility window; quotas merge-vs-replace server behavior; memberTenantId query filter under top-level token; app-password auth without primary password; singleton-id conventions.

Remaining: Tasks 5–9 (account+password, settings, ADRs 0003/0004, integration journey, close-out)

agent 2644d5c Aug 25

Implementation complete — full journey green against live C1 harness (2026-08-25)

Plan (comment 264fd9c) executed end-to-end via subagent-driven development; every task passed spec-compliance + code-quality review gates. Mid-flight owner review (bugs d73c3db/19e419e) merged internal/jmap+internal/registry into internal/stalwart (comments df7b3cc/1596ef4).

Commits (oldest first)

qulpwsty scaffold → rxvwtrwz selfci CI → puplsvow feat(jmap) → svskvltx feat(registry) → zzwlqknk feat(tenant,domain) → xzmvnpzr refactor(stalwart): merge → oomwrwow feat(account) → zylwxsns feat(settings) → xospuolq docs(adr): 0003+0004 → mlmqvoqz test(integration): signup story journey.

Acceptance criteria — all met (live, go test -race ./internal/integration -v)

  • Full lifecycle green against real container: tenant/domain/account create→get→update→delete + settings two-phase ✓
  • Idempotency: replaying every Ensure twice yields changed=false and identical end state ✓
  • Masked-password path proven: account.Update omitting credentials → original secret still authenticates (hard-asserted) ✓
  • Auth as recovery admin (basic) ✓; schema hash pin matches live rev ✓

Surprises (the point of Phase 0) — plan’s source-reading was wrong in 8 places

# Plan assumed Live server actually
1 query filter = RFC array [{attribute,operator,value}] object {"name": v}, ordered ops {"nameIs<Op>": v}; array → HTTP 400 notRequest
2 Account.name = full address local part; emailAddress is server-computed (unindexed)
3 List (aliases/credentials) = JSON array object keyed by u32 index {"0":{…}}; array → invalidPatch
4 linked-delete error = cannotDeleteLinked objectIsLinked (+ objectId/linkedObjects)
5 Tenant.name queryable not indexed (full-text only); Ensure/Create list+match in memory
6 query response has total only with calculateTotal:true
7 masked password = resend "****" in account credentials account set is a shallow merge — omit credentials to preserve; "****" belongs in x:AccountPassword/set only
8 /api/discover/{email} resolves per-address returns the OIDC discovery doc for any input (does not resolve addresses)

Confirmed-correct from source: app-password = standalone x:AppPassword/set with target accountId, server-generated one-time secret (works via JMAP basic-auth); Action::ReloadSettings = {"@type":"ReloadSettings"}; settings singleton id is literal "singleton"; async TaskDestroyAccount (account destroy is read-synchronous in practice); DKIM auto-management materializes DkimSignature objects + DkimManagement tasks; destroy ordering must remove linked DkimSignature before the domain.

Follow-ups worth filing (not blocking this spike)

  • Tenant lookup is O(n) list+match with undetected truncation; revisit pagination/calculateTotal when tenant count grows.
  • domain.ListForTenant pagination guard is now live (total populated) but does not yet paginate.
  • docs/05-stalwart-integration.md §2 wire shapes contradict findings 1–8 above; needs an update pass (deferred per f89287c “ADR/issue first, docs later” posture).
  • DKIM task lifecycle edge: deleting a domain with an in-flight DKIM task orphans it (observed Failed “Invalid foreign key” task in harness).

Left open for owner review.

agent 294bdfc Aug 25

Follow-up issues filed: pagination/tenant-enumeration → 9d84518; DKIM-task orphan → cf71374.