-- AppView database schema 0011: remember which record a follow came from. -- -- Why -- -- A `follows` row was addressable only as `(follower_did, subject_did)`. -- That is the right identity for the *relationship*, but it is not the -- identity a delete event carries. A firehose / Jetstream delete op is -- just `did` + `rkey`: -- -- {"action": "delete", "path": "app.bsky.graph.follow/3lmnop"} -- -- There is no record body on a delete — the record is gone, that is the -- whole point of the event — so the subject DID is nowhere in it. With -- no rkey stored, the indexer had no way from `3lmnop` back to -- "did:plc:bob" and logged-and-skipped the op -- (`indexer::apply_commit`, `app.bsky.graph.follow` arm). -- -- The practical consequence: unfollows only ever landed through the -- PDS's best-effort `POST /internal/ingest-commit` push, which knows the -- subject from its own snapshot. That push has no retry and no -- acknowledgement (see `pds_firehose`'s module docs). If it was lost — -- AppView restarting, request timing out — the follow stayed in the -- index forever, and the firehose, the stream that exists precisely to -- repair such gaps, could not repair this one. Storing the rkey closes -- that hole: the firehose replay can now apply the unfollow on its own. -- -- The primary key deliberately stays `(follower_did, subject_did)` -- ------------------------------------------------------------------ -- It is what makes `upsert_follow` idempotent. The same follow reaches -- us over both transports (push *and* firehose) and again after any -- replay, and every one of those must converge on one row. Keying on -- the rkey instead — or adding it to the key — would make a re-follow -- under a fresh rkey a *second* row for the same relationship, and then -- `follower_count` would count the same follower twice. -- -- So `rkey` is not identity here; it is a second *access path* to a row -- the primary key already identifies. -- -- Nullable, because history has no rkey -- ------------------------------------- -- Every row written before this migration was inserted without one, and -- there is nothing to backfill it from: the AppView never stored the -- follow record itself. A NOT NULL column would need a fabricated -- placeholder that a later delete could accidentally match. NULL says -- exactly what is true — "we do not know which record this came from" — -- and a delete-by-rkey simply finds nothing for those rows, which is the -- documented no-op path in `indexer::delete_follow_by_rkey`. Those rows -- keep working through the push path (which sends `subject_did`) and -- heal on their own the next time the follow is re-created. -- -- Re-follow under a new rkey -- -------------------------- -- Follow → unfollow → follow again produces a *different* rkey each -- time (rkeys are TIDs; the client never reuses one). The upsert -- therefore hits the primary key and overwrites `rkey` with the newer -- record's: the youngest record wins. That ordering is what makes a -- late or replayed delete for the *old* rkey harmless — it matches no -- row and is skipped, instead of tearing down a follow that is -- currently live. -- -- The index is NOT unique -- ----------------------- -- `(follower_did, rkey)` is unique in practice — an rkey identifies one -- record inside one repo's collection — but a unique index would turn -- the one situation this migration exists for into a *write failure*: -- if a delete was lost and a create later reused that rkey, the insert -- would abort instead of the stale row being cleaned up. An index whose -- only job is to serve a lookup should not be able to reject a write. -- Partial (`WHERE rkey IS NOT NULL`) because a lookup key is never -- NULL, so the pre-migration rows have no business bloating it. ALTER TABLE follows ADD COLUMN IF NOT EXISTS rkey TEXT; -- Serves `DELETE FROM follows WHERE follower_did = $1 AND rkey = $2 -- RETURNING subject_did` — the delete path for a firehose -- unfollow, which is the only lookup this column exists for. CREATE INDEX IF NOT EXISTS follows_follower_rkey_idx ON follows (follower_did, rkey) WHERE rkey IS NOT NULL;