tomdeboneandClaude Opus 5
2b695d6892
Ein Delete-Event trägt nur did + rkey, keinen Record-Body. `follows` hatte aber nur (follower_did, subject_did) und speicherte den rkey nicht — es gab also keinen Weg vom rkey zum subject_did, und der Indexer hat solche Ops geloggt und übersprungen. Unfollows hingen damit allein am Best-Effort-Push, genau der Abhängigkeit, die der Firehose beseitigen soll. Migration 0011 ergänzt die rkey-Spalte plus einen partiellen Index für den Lookup. Der Primärschlüssel bleibt (follower_did, subject_did), damit die Upserts über Push, Firehose und Replay hinweg idempotent bleiben; ein rkey im Schlüssel würde aus einem Re-Follow eine zweite Zeile machen und die Follower-Zahl verdoppeln. Der Index ist bewusst nicht unique: sonst würde ausgerechnet der Fall, für den das hier existiert — verlorener Delete, dann ein neuer Create — zu einem abgebrochenen Write. delete_follow_by_rkey löst und löscht in einem Statement (RETURNING), also ohne Rennen zwischen Auflösen und Löschen. Findet es nichts — alte Zeile ohne rkey, schon gelöscht, veralteter rkey — ist das kein Fehler. Der Push-Pfad über subject_did bleibt unverändert. Likes und Reposts haben die Lücke nicht: dort ist der rkey Teil der Zeilenidentität. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013HC9HLrUU1LNwkzp8nkDLX
80 lines
4.1 KiB
SQL
80 lines
4.1 KiB
SQL
-- AppView database schema 0011: remember which record a follow came from.
|
|
--
|
|
-- Why
|
|
--
|
|
-- A `follows` row was addressable only as `(follower_did, subject_did)`.
|
|
-- That is the right identity for the *relationship*, but it is not the
|
|
-- identity a delete event carries. A firehose / Jetstream delete op is
|
|
-- just `did` + `rkey`:
|
|
--
|
|
-- {"action": "delete", "path": "app.bsky.graph.follow/3lmnop"}
|
|
--
|
|
-- There is no record body on a delete — the record is gone, that is the
|
|
-- whole point of the event — so the subject DID is nowhere in it. With
|
|
-- no rkey stored, the indexer had no way from `3lmnop` back to
|
|
-- "did:plc:bob" and logged-and-skipped the op
|
|
-- (`indexer::apply_commit`, `app.bsky.graph.follow` arm).
|
|
--
|
|
-- The practical consequence: unfollows only ever landed through the
|
|
-- PDS's best-effort `POST /internal/ingest-commit` push, which knows the
|
|
-- subject from its own snapshot. That push has no retry and no
|
|
-- acknowledgement (see `pds_firehose`'s module docs). If it was lost —
|
|
-- AppView restarting, request timing out — the follow stayed in the
|
|
-- index forever, and the firehose, the stream that exists precisely to
|
|
-- repair such gaps, could not repair this one. Storing the rkey closes
|
|
-- that hole: the firehose replay can now apply the unfollow on its own.
|
|
--
|
|
-- The primary key deliberately stays `(follower_did, subject_did)`
|
|
-- ------------------------------------------------------------------
|
|
-- It is what makes `upsert_follow` idempotent. The same follow reaches
|
|
-- us over both transports (push *and* firehose) and again after any
|
|
-- replay, and every one of those must converge on one row. Keying on
|
|
-- the rkey instead — or adding it to the key — would make a re-follow
|
|
-- under a fresh rkey a *second* row for the same relationship, and then
|
|
-- `follower_count` would count the same follower twice.
|
|
--
|
|
-- So `rkey` is not identity here; it is a second *access path* to a row
|
|
-- the primary key already identifies.
|
|
--
|
|
-- Nullable, because history has no rkey
|
|
-- -------------------------------------
|
|
-- Every row written before this migration was inserted without one, and
|
|
-- there is nothing to backfill it from: the AppView never stored the
|
|
-- follow record itself. A NOT NULL column would need a fabricated
|
|
-- placeholder that a later delete could accidentally match. NULL says
|
|
-- exactly what is true — "we do not know which record this came from" —
|
|
-- and a delete-by-rkey simply finds nothing for those rows, which is the
|
|
-- documented no-op path in `indexer::delete_follow_by_rkey`. Those rows
|
|
-- keep working through the push path (which sends `subject_did`) and
|
|
-- heal on their own the next time the follow is re-created.
|
|
--
|
|
-- Re-follow under a new rkey
|
|
-- --------------------------
|
|
-- Follow → unfollow → follow again produces a *different* rkey each
|
|
-- time (rkeys are TIDs; the client never reuses one). The upsert
|
|
-- therefore hits the primary key and overwrites `rkey` with the newer
|
|
-- record's: the youngest record wins. That ordering is what makes a
|
|
-- late or replayed delete for the *old* rkey harmless — it matches no
|
|
-- row and is skipped, instead of tearing down a follow that is
|
|
-- currently live.
|
|
--
|
|
-- The index is NOT unique
|
|
-- -----------------------
|
|
-- `(follower_did, rkey)` is unique in practice — an rkey identifies one
|
|
-- record inside one repo's collection — but a unique index would turn
|
|
-- the one situation this migration exists for into a *write failure*:
|
|
-- if a delete was lost and a create later reused that rkey, the insert
|
|
-- would abort instead of the stale row being cleaned up. An index whose
|
|
-- only job is to serve a lookup should not be able to reject a write.
|
|
-- Partial (`WHERE rkey IS NOT NULL`) because a lookup key is never
|
|
-- NULL, so the pre-migration rows have no business bloating it.
|
|
|
|
ALTER TABLE follows ADD COLUMN IF NOT EXISTS rkey TEXT;
|
|
|
|
-- Serves `DELETE FROM follows WHERE follower_did = $1 AND rkey = $2
|
|
-- RETURNING subject_did` — the delete path for a firehose
|
|
-- unfollow, which is the only lookup this column exists for.
|
|
CREATE INDEX IF NOT EXISTS follows_follower_rkey_idx
|
|
ON follows (follower_did, rkey)
|
|
WHERE rkey IS NOT NULL;
|