perf(appview): Profil, Cold-Start-Feed und Follow-Timeline entlasten
Gemessen gegen die Dev-Instanz (3,3 Mio. Posts): * GET /api/profile/<handle> 9,5 s → 0,04 s * Cold-Start-Timeline 7,4 s → 0,006 s * Timeline mit 2300 Follows 28 s → 0,02 s Drei unabhängige Ursachen, alle drei ein Seq-Scan über die posts-Tabelle: 1. resolve_profile sucht die DID über profiles.LOWER(handle) und, als Fallback, über posts.handle. Für beides gab es keinen Index. Auf profiles hatte Migration 0007 genau diesen Index entfernt, mit der Begründung, jeder Aufrufer leite ohnehin zuerst eine DID ab — das stimmt nicht mehr, seit resolve_profile den profiles-Cache zuerst befragt. 2. Der Cold-Start-Feed filtert `collection IN (…)` und sortiert nach indexed_at. Der vorhandene (collection, indexed_at, uri)-Index taugt dafür nicht: mit zwei führenden Werten liefert er keine indexed_at-Ordnung mehr. Ein partieller Index über genau das Prädikat schiebt den Filter in die Definition und lässt (indexed_at DESC, uri DESC) als Sortierschlüssel übrig. 3. Genau dieser neue Index wurde dann zur Falle für den Graph-Zweig: der Planer sah einen Index, der schon in indexed_at-Ordnung liefert, und nahm an, er treffe früh genug auf n passende Zeilen — bei dünn besetzten Followees hieß "früh" 2,87 Mio. verworfene Zeilen. Je nach Anzahl bisheriger Ausführungen des Prepared Statements kippte er zwischen diesem und dem guten Plan, was intermittierend aussah. Der Graph-Zweig formuliert die Absicht jetzt aus: pro Followee die neuesten Posts über ein LATERAL, dann mergen. Damit ist der globale Scan kein wählbarer Plan mehr, und jede Iteration ist ein begrenzter Range-Scan auf posts_did_indexed_at_uri_idx. Korrekt ist das, weil die globalen Top-N immer eine Teilmenge der Vereinigung der Top-N je Followee sind — deshalb wird pro Followee limit+1 geholt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013HC9HLrUU1LNwkzp8nkDLX
This commit is contained in:
co-authored by
Claude Opus 5
parent
a2a371b7d9
commit
73da8f0140
@@ -0,0 +1,58 @@
|
||||
-- AppView database schema 0009: indexes for handle → DID lookups.
|
||||
--
|
||||
-- `/api/profile/<handle>` took 9.5 s on a 3.3 M-row `posts` table
|
||||
-- (measured against the dev instance). Both halves of `resolve_profile`
|
||||
-- were unindexed:
|
||||
--
|
||||
-- 1. SELECT did FROM profiles WHERE LOWER(handle) = LOWER($1)
|
||||
-- 2. SELECT did FROM posts WHERE handle = $1
|
||||
-- ORDER BY indexed_at DESC LIMIT 1 -- the fallback
|
||||
--
|
||||
-- Step 2 was a parallel sequential scan over every post ever indexed
|
||||
-- (`Rows Removed by Filter: 1101310` per worker), and it runs on every
|
||||
-- profile view in the client.
|
||||
--
|
||||
-- On `profiles`: migration 0007 dropped exactly this index, reasoning
|
||||
-- that "every caller derives a DID first (via posts.handle or the
|
||||
-- handle-sync worker) and then queries profiles by PK". That stopped
|
||||
-- being true when `resolve_profile` learned to prefer the profiles
|
||||
-- cache — it now asks `profiles` by handle *first*, precisely the
|
||||
-- lookup 0007 removed the support for. Re-added, matching the
|
||||
-- expression in the query (`LOWER(handle)`) so the planner can use it.
|
||||
CREATE INDEX IF NOT EXISTS profiles_handle_lower_idx
|
||||
ON profiles (LOWER(handle));
|
||||
|
||||
-- On `posts`: `(handle, indexed_at DESC)` covers filter *and* sort, so
|
||||
-- the LIMIT 1 becomes an index scan that stops at the first row.
|
||||
--
|
||||
-- Partial on `handle <> ''`: empty handles are the un-backfilled
|
||||
-- majority on a firehose-fed instance and are never looked up by this
|
||||
-- path (the handle-sync worker queries them through its own predicate),
|
||||
-- so excluding them keeps the index small on the largest table we have.
|
||||
CREATE INDEX IF NOT EXISTS posts_handle_indexed_at_idx
|
||||
ON posts (handle, indexed_at DESC)
|
||||
WHERE handle <> '';
|
||||
|
||||
-- =====================================================
|
||||
-- posts: the cold-start global feed
|
||||
-- =====================================================
|
||||
--
|
||||
-- `/api/timeline/home` falls back to the global recent feed for users
|
||||
-- without a follow graph — every new account's first screen. It took
|
||||
-- 7.4 s (parallel seq scan + top-N sort over 3.3 M rows) and timed out
|
||||
-- the integration tests' 5 s client.
|
||||
--
|
||||
-- `posts_collection_indexed_at_uri_idx (collection, indexed_at DESC,
|
||||
-- uri DESC)` cannot serve it: the query filters
|
||||
-- `collection IN ('app.twi.post','app.bsky.feed.post')`, and with two
|
||||
-- leading values the index no longer yields rows in `indexed_at` order,
|
||||
-- so the planner falls back to scanning and sorting.
|
||||
--
|
||||
-- A partial index over exactly that predicate moves the collection
|
||||
-- filter into the index definition, which leaves `(indexed_at DESC,
|
||||
-- uri DESC)` as the sort key — the LIMIT then stops after the first
|
||||
-- page. Same shape as the existing `posts_did_indexed_at_uri_idx`,
|
||||
-- which is partial on the same two collections.
|
||||
CREATE INDEX IF NOT EXISTS posts_feed_indexed_at_uri_idx
|
||||
ON posts (indexed_at DESC, uri DESC)
|
||||
WHERE collection IN ('app.twi.post', 'app.bsky.feed.post');
|
||||
Reference in New Issue
Block a user