Diagnose an app that is missing a member

Start here. backfillOutstanding: true means the chunker has not finished and nothing is broken — wait. Also check status: a merely SUSPENDED enablement denies access with no projection fault at all. Two age signals, not one: a backfill inflates the pending COUNT and the pending AGE, so during one the only unambiguous signal is the age of the outstanding backfill itself — muting the alert during a backfill would blind the system during its riskiest operation.

Reached via act-as (scenario 37) — this is a tenant-scoped admin read. TWO OUTCOMES, AND THE SECOND IS THE COMMON CAUSE RATHER THAN AN ALL-CLEAR. granted: false → the mundane case: they were simply never granted the app (scenario 49), which is NOT projection drift. granted: true while the app still cannot see them → GO STRAIGHT TO THE RESYNC. A grant issued AFTER this tenant's backfill completed emits no projection event and is enrolled by nothing under shipped defaults, so the member is absent with IAM and the app in perfect agreement — this is the single most likely answer once onboarding is finished. A foreign-tenant userId is 404, not 403 (AUTH-OBJECT-03) — a 403 would confirm the id names a real user somewhere. Note the projection is NEVER read for an authorization decision: a member missing downstream is a data-visibility fault, never a permissions one, because grants and enablements are checked live at every /internal/validate hop. So this scenario never explains a 403.

The repair. Safe at any time and safe twice, which is what makes it a legitimate FIRST response rather than a last resort: every downstream upsert is version-guarded, so a re-emitted row at an equal or lower version is discarded — the cost is delivery work, and nothing already correct changes. It clears backfilled_at so the chunker re-emits the whole (tenant, app) pair, and because the chunker JOINS membership_app_grants it re-projects every currently granted active member. 404 when there is no live enablement for that tenant and app — silently succeeding would leave an operator believing a repair had run. SOME DRIFT REPAIRS ITSELF; ONE CLASS NEVER DOES. The reconciler compares each source row's version against the ledger of what was last sent and re-emits anything the source is ahead of — that covers a stale rename or profile edit within PROJECTION_RECONCILE_INTERVAL_MS (default 1h) and needs no operator. But it walks ONLY ledger rows that already exist, so it cannot see a pair that was never registered. The second sweep is the one that would enrol those, and it SHIPS REPORT-ONLY (PROJECTION_SCOPE_SWEEP_REPAIR_ENABLED defaults 'false'): it logs the missing pair and emits nothing. So for a member granted after the backfill, NOTHING REPAIRS IT AND WAITING DOES NOT HELP — this resync is the repair, not a fallback after the automatic ones have had their chance. A STALE ROW IS A DIFFERENT FAULT FROM A MISSING ONE, and usually needs no action: PATCH /v1/me (a user's own display_name) and the rename branch of PATCH /v1/admin/tenants/{tenantId} both change a projected field and emit nothing, but both have a live ledger row, so the reconciler re-emits them within the hour. A name up to an hour behind is EXPECTED BEHAVIOUR, not drift — there is no surface to force a reconciler pass, so the answer is to wait, and a resync does not shorten it (it re-projects membership for a pair, not a renamed tenant it was not asked about). suspended KEEPS the projection, terminated RETIRES it (scenario 41): re-activating a terminated enablement clears the backfill and re-projects the tenant in full, because the tombstone already retired the app's copy — returning is a fresh projection, not a resume. The identity app never appears in any of this: it holds no second database and is never a projection target, so a structurally empty series for it is expected rather than a drainer in perfect health.