perf(appview): Profil, Cold-Start-Feed und Follow-Timeline entlasten
Gemessen gegen die Dev-Instanz (3,3 Mio. Posts): * GET /api/profile/<handle> 9,5 s → 0,04 s * Cold-Start-Timeline 7,4 s → 0,006 s * Timeline mit 2300 Follows 28 s → 0,02 s Drei unabhängige Ursachen, alle drei ein Seq-Scan über die posts-Tabelle: 1. resolve_profile sucht die DID über profiles.LOWER(handle) und, als Fallback, über posts.handle. Für beides gab es keinen Index. Auf profiles hatte Migration 0007 genau diesen Index entfernt, mit der Begründung, jeder Aufrufer leite ohnehin zuerst eine DID ab — das stimmt nicht mehr, seit resolve_profile den profiles-Cache zuerst befragt. 2. Der Cold-Start-Feed filtert `collection IN (…)` und sortiert nach indexed_at. Der vorhandene (collection, indexed_at, uri)-Index taugt dafür nicht: mit zwei führenden Werten liefert er keine indexed_at-Ordnung mehr. Ein partieller Index über genau das Prädikat schiebt den Filter in die Definition und lässt (indexed_at DESC, uri DESC) als Sortierschlüssel übrig. 3. Genau dieser neue Index wurde dann zur Falle für den Graph-Zweig: der Planer sah einen Index, der schon in indexed_at-Ordnung liefert, und nahm an, er treffe früh genug auf n passende Zeilen — bei dünn besetzten Followees hieß "früh" 2,87 Mio. verworfene Zeilen. Je nach Anzahl bisheriger Ausführungen des Prepared Statements kippte er zwischen diesem und dem guten Plan, was intermittierend aussah. Der Graph-Zweig formuliert die Absicht jetzt aus: pro Followee die neuesten Posts über ein LATERAL, dann mergen. Damit ist der globale Scan kein wählbarer Plan mehr, und jede Iteration ist ein begrenzter Range-Scan auf posts_did_indexed_at_uri_idx. Korrekt ist das, weil die globalen Top-N immer eine Teilmenge der Vereinigung der Top-N je Followee sind — deshalb wird pro Followee limit+1 geholt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013HC9HLrUU1LNwkzp8nkDLX
This commit is contained in:
tomdebone
co-authored by
Claude Opus 5
Claude Opus 5
parent
a2a371b7d9
commit
73da8f0140
@@ -318,20 +318,51 @@ async fn timeline_home(
|
||||
.map_err(db_err)?,
|
||||
}
|
||||
} else {
|
||||
// Graph-aware branch: filter `posts.did` to the followee set
|
||||
// plus the requesting user's own DID. `target_dids` has been
|
||||
// deduped and capped at MAX_FOLLOWED_DIDS, and the user's own
|
||||
// DID is guaranteed to be in the set.
|
||||
// Graph-aware branch: the followee set plus the requesting
|
||||
// user's own DID. `target_dids` has been deduped and capped at
|
||||
// MAX_FOLLOWED_DIDS, and the user's own DID is guaranteed to
|
||||
// be in the set.
|
||||
//
|
||||
// ## Why this is a LATERAL and not `did = ANY($2)`
|
||||
//
|
||||
// The straightforward `WHERE did = ANY($2) ORDER BY indexed_at
|
||||
// DESC LIMIT n` is a plan-stability trap once
|
||||
// `posts_feed_indexed_at_uri_idx` exists (migration 0009, added
|
||||
// for the cold-start feed). The planner sees an index that
|
||||
// already yields rows in `indexed_at DESC` order and assumes it
|
||||
// will hit `n` matching rows early — so it walks the global
|
||||
// feed and filters. When the followees are sparse (a fresh
|
||||
// account following accounts that haven't posted), "early"
|
||||
// means millions of rows: measured on the dev instance, 2.87 M
|
||||
// rows discarded and 28 s per request, while the same query
|
||||
// took 62 ms with the per-DID index. It also flipped between
|
||||
// the two plans depending on how often the prepared statement
|
||||
// had run, so it looked intermittent.
|
||||
//
|
||||
// Expressing the intent — "for each followee, their newest
|
||||
// posts, merged" — takes that plan off the table: `unnest` is a
|
||||
// relation the planner can size, and each iteration is a bounded
|
||||
// range scan on `posts_did_indexed_at_uri_idx`. Fetching `$1`
|
||||
// per followee is what makes it correct: the global top-N is
|
||||
// always a subset of the union of the per-followee top-Ns.
|
||||
match cursor_ts {
|
||||
Some(ts) => sqlx::query_as::<_, PostRowWithIndexed>(
|
||||
r#"SELECT uri, did, handle, rkey, collection, text, cid,
|
||||
parent_uri, root_uri, embed, langs, created_at,
|
||||
indexed_at
|
||||
FROM posts
|
||||
WHERE collection IN ('app.twi.post','app.bsky.feed.post')
|
||||
AND did = ANY($2::text[])
|
||||
AND (indexed_at, uri) < ($3, $4)
|
||||
ORDER BY indexed_at DESC, uri DESC
|
||||
r#"SELECT t.uri, t.did, t.handle, t.rkey, t.collection,
|
||||
t.text, t.cid, t.parent_uri, t.root_uri,
|
||||
t.embed, t.langs, t.created_at, t.indexed_at
|
||||
FROM unnest($2::text[]) AS f(did)
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT p.uri, p.did, p.handle, p.rkey, p.collection,
|
||||
p.text, p.cid, p.parent_uri, p.root_uri,
|
||||
p.embed, p.langs, p.created_at, p.indexed_at
|
||||
FROM posts p
|
||||
WHERE p.did = f.did
|
||||
AND p.collection IN ('app.twi.post','app.bsky.feed.post')
|
||||
AND (p.indexed_at, p.uri) < ($3, $4)
|
||||
ORDER BY p.indexed_at DESC, p.uri DESC
|
||||
LIMIT $1
|
||||
) t
|
||||
ORDER BY t.indexed_at DESC, t.uri DESC
|
||||
LIMIT $1"#,
|
||||
)
|
||||
.bind(fetch)
|
||||
@@ -342,13 +373,21 @@ async fn timeline_home(
|
||||
.await
|
||||
.map_err(db_err)?,
|
||||
None => sqlx::query_as::<_, PostRowWithIndexed>(
|
||||
r#"SELECT uri, did, handle, rkey, collection, text, cid,
|
||||
parent_uri, root_uri, embed, langs, created_at,
|
||||
indexed_at
|
||||
FROM posts
|
||||
WHERE collection IN ('app.twi.post','app.bsky.feed.post')
|
||||
AND did = ANY($2::text[])
|
||||
ORDER BY indexed_at DESC, uri DESC
|
||||
r#"SELECT t.uri, t.did, t.handle, t.rkey, t.collection,
|
||||
t.text, t.cid, t.parent_uri, t.root_uri,
|
||||
t.embed, t.langs, t.created_at, t.indexed_at
|
||||
FROM unnest($2::text[]) AS f(did)
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT p.uri, p.did, p.handle, p.rkey, p.collection,
|
||||
p.text, p.cid, p.parent_uri, p.root_uri,
|
||||
p.embed, p.langs, p.created_at, p.indexed_at
|
||||
FROM posts p
|
||||
WHERE p.did = f.did
|
||||
AND p.collection IN ('app.twi.post','app.bsky.feed.post')
|
||||
ORDER BY p.indexed_at DESC, p.uri DESC
|
||||
LIMIT $1
|
||||
) t
|
||||
ORDER BY t.indexed_at DESC, t.uri DESC
|
||||
LIMIT $1"#,
|
||||
)
|
||||
.bind(fetch)
|
||||
|
||||
Reference in New Issue
Block a user