Phase 0: critical-path risk reduction (tracking)

closed
#f98ccdc opened by agent Aug 24

Purpose

Overarching tracker for Phase 0 — critical-path risk reduction: build the thinnest possible walking skeleton through the three riskiest integrations first, so unknown unknowns surface in week one rather than after months of feature work:

  1. Control plane ↔ Stalwart deployment integration (Registry API over real deployment)
  2. Credential model: how the control plane is authorized to act on behalf of arbitrary users (masquerade/impersonation)
  3. atproto OAuth flow for authn/authz against a user’s PDS at signup

Design basis: docs/00-overview.md … docs/09-open-questions.md (decisions locked 2026-08-24).

Child issues & dependency graph

ID Issue Depends on
ccd3c37 [C1] Deployment harness: local Stalwart OSS + Postgres + spaces-alpha PDS —
24dcaa6 [C2] stalwartsync: Registry-client spike — schema, object lifecycle, two-phase reload C1
3541d85 [C3] Control-plane credential & impersonation model (masquerade as any user) C1
a589d34 [C4] OAuth broker walking skeleton — indigo ClientApp vs live PDS C1
c1c7e19 [C5] OIDC issuer → Stalwart OpenId directory wiring + app-password round-trip C2, C3, C4
0d541aa [C6] End-to-end risk probe: signup→login→read-mailbox + failure injection C1–C5
ccd3c37 ──► 24dcaa6 ─┐
        ├──► 3541d85 ├─► c1c7e19 ──► 0d541aa
        └──► a589d34 ─┘

C2/C3/C4 are parallelizable once C1 lands.

Phase exit criteria

  • 0d541aa passes end-to-end against real Stalwart + real PDS containers, including one injected failure per dependency.
  • Every surprise discovered during C1–C6 is recorded as a new bug issue (link from the discovering issue) and triaged into later phases.
  • Decision records exist for: impersonation mechanism (3541d85 ADR) and any deviation from design docs.

6 Comments

agent f59f84c Aug 24

Scope change on C1 (ccd3c37) — 2026-08-24

Per design review, C1 was revised before implementation:

  1. SQLite everywhere (ADR-0002 in sovrn docs): no Postgres container anywhere in Phase 0 or v1. Sovrn app DB = SQLite (modernc.org/sqlite); Stalwart data store = SQLite configured at bootstrap. Docs 01 §2/§8, 07, 09-Q7 updated.
  2. Nix-built service containers following the ../rtw/services pattern (dockerTools.buildLayeredImage via nix/lib.nix mkImage; packages under nix/pkgs; entrypoints under nix/scripts). Stalwart is built from source @ rev 2add611e with enterprise feature OFF — nixpkgs’ stalwart-mail (0.15.5) predates the Registry architecture the whole plan depends on.
  3. devenv up orchestrates bring-up via process-compose processes (no Justfile for lifecycle).

Status: all Nix/devenv/script code written; git-dep outputHashes resolved; full Stalwart rust build running. One manual blocker for the Nix-built PDS: npm package-lock generation fails silently in this environment and npm is outside agent permissions — documented manual step in ccd3c37; devenv falls back to upstream alpha image until it lands.

agent f89287c Aug 24

Design-status note — 2026-08-24

Owner is reconsidering where data lives (on-protocol in users’ PDS repos vs. Stalwart stores) as Phase 0 progresses. Consequences for how we work until that settles:

  1. docs/00–09 are provisional. They capture the current best design but should NOT be treated as commitments; expect significant revision of the data-locality sections (esp. docs/01 §2, 03, 04).
  2. Bug reports are the durable record. Verified facts, source references (file:line @ rev 2add611e), and live-test results belong in issue comments — e.g. C1’s bootstrap findings (c7cfd93), C3’s verified auth/masquerade mechanics (comment 345b461). New discoveries go here first.
  3. Keep building the seams, not the interiors. C1/C2/C5-style integration work remains valid regardless of where mail data ultimately lives: the Registry shim, credential model, OAuth broker, and watcher pipelines are locality-independent. Defer deep investment in record-schema internals and space-projection details (C4+ of docs/03–04 candidate issues) until the data-location decision lands.
  4. When the decision is made: write it as a fresh ADR superseding affected docs rather than editing in place.
agent f89481c Aug 25

C2 (24dcaa6) complete — 2026-08-25

Child issue C2 (stalwartsync Registry-client spike) is implemented and green against the live C1 harness; see its findings comment (2644d5c) for the full record. Summary for the tracker:

  • Delivered: internal/stalwart (merged JMAP+Registry client, one constructor, no env side-effects), record packages tenant/domain/account/settings with required tenancy (ADR-0004), integration journey test proving the signup story end-to-end. All three acceptance criteria met live (lifecycle green, replay-idempotency, masked-password proof).
  • Surprises (the point of Phase 0): 8 plan assumptions from source-reading were disproven live — filter object-form, account name = local part, indexed-object list serialization, objectIsLinked, unindexed tenant name, calculateTotal gating, omission-based secret preservation, /api/discover = OIDC doc. All recorded on 24dcaa6 and corrected in code + docs/05.
  • Unblocks: C5 (c1c7e19) depends on C2’s account.IssuePassword/Update and the verified app-password auth path; C3 (3541d85) benefits from the confirmed recovery-admin + Impersonate mechanics. C2’s stalwart.Client is the shared seam C5/C6 build on.

Follow-up issues filed: pagination/O(n) tenant lookup, and the DKIM-task orphan edge (links below).

agent f69385c Aug 26

New child issue + design revision

  • b5a31bd [C7]: domain ownership verification + readiness probes (internal/domain) — implements the two-gate proof (TXT challenge + apex-handle piggyback) from ADR-0005. Depends on C2 (Stalwart lifecycle in internal/domain) and feeds C6 (signup journey gate).
  • Design revision: docs/03-provisioning.md §4 split the claim state machine into claimed → verifying → verified → active (ownership gate before any Stalwart provisioning); new §4a + docs/adr/0005-domain-ownership-verification.md. Docs/01/02 touched to match.
agent fe9586c Aug 28

Design decision landed — ADR-0006 (2026-08-28)

The data-location question tracked since 2026-08-24 (comment f89287c) is resolved. New ADR supersedes the affected docs:

  • ADR-0006 — Data placement & space topology: postmaster DID as space authority; tenant+domain space sharding (opaque skey); authorDid enforcement; in-place mutation; member-list access for v1; public service record (endpoints only, two surfaces: app view + webmail); no public claim record (signup = OAuth + XRPC + DNS gate); settings are JMAP-only.
  • Docs revised: 00 (decision log D8–D13), 01 (component model, data stores, sequences, appview/webmail split), 03 (drop claim, XRPC signup, lexicons), 04 (spaces topology rewrite), 06 (dev vs atproto-crates / prod eval vs zds), 09 (triage + deferred items).

Effect on C6

0d541aa re-scoped (comment 09d75f4): happy path no longer publishes a public claim or consumes jetstream for signup.

New child issues (Phase 1, post-C6)

ID Issue
f6192a1 Lexicons: space type declarations + record schemas (at.sovrn.*)
a472ea3 Lexicon: at.sovrn.mail.service — endpoint inventory
ee2256b Postmaster DID ceremony + KMS + hosted spaces-PDS
12ce549 spaceproj: postmaster-authored publisher + authorDid-filtered reader
f15633a Reconciler: XRPC signup → DB → Stalwart → postmaster; three-way sweep
b04aeb9 App view: admin UI + mutation/query XRPC
9185c91 v1 roadmap: feature in/out cut
agent f69d86c Sep 8

Phase 0 exit met — 2026-09-08

All Phase-0 child issues are closed and the exit gate passed:

ID Issue Status
ccd3c37 C1 deployment harness closed
24dcaa6 C2 Registry-client spike closed
3541d85 C3 credential/impersonation model closed
a589d34 C4 OAuth broker walking skeleton closed
c1c7e19 C5 OIDC issuer → Stalwart + app-password closed
0d541aa C6 E2E risk probe (exit gate) closed
b5a31bd C7 domain ownership + readiness probes closed

The risky seams (Registry integration, impersonation, OAuth, OIDC translation) are proven against a live harness. Nothing in Phase 0 blocks the next step.

Path forward

New tracker 70790ed (“PoC: minimal live deployment”) carries the thin-PoC plan. Scope is deliberately thin: control plane + Stalwart on a live server, app-password read-mail proof. The on-protocol ADR-0006/0007 work (postmaster, spaces, reconciler, webmail) is deferred until after the thin PoC deploys.

Storage/failover decision landed as ADR-0008: Stalwart store = FoundationDB in prod (SQLite in dev), app DB = SQLite everywhere, failover = active/passive + VIP + FDB lease fence (active/active coordinator deferred to Q8). docs/deployment.md updated to match.

Tracker-lag note

The tracker lags the code in two places worth recording: - f6192a1 (lexicons) is effectively done in code (lexicons/at/sovrn/* + generated api/sovrn types incl. tenant.detail/domain.detail/mail.account/mail.service); only network publication (sovrn.at _lexicon TXT) remains. - f15633a (reconciler) is ~half done: app-view XRPC, SQLite store (TID↔StalwartID), DNS gate, and background verifier exist; the mailbox_claims state machine, spaceproj publisher, and three-way sweep do not.