A user who set their profile via the PDS push path before
posting anything has a row in `profiles` but no rows in
`posts` — the handle→DID lookup in `resolve_profile` only
checked `posts`, so the route synthesised an empty profile
(`did: ""`) for these users.
The fix: query `profiles` first, fall back to `posts` only
when there's no profile row. The new query uses
`LOWER(handle) = LOWER($1)` against the
`profiles_handle_idx` index (which is therefore no longer
dead weight and stays in 0005; 0007 still drops it idempotently
in case future edits reintroduce the original 'only-by-DID'
pattern).
Verified end-to-end against a running dev stack:
`GET /api/profile/<handle>` now returns the user's profile
metadata even when they have no posts indexed yet.
Adds the AppView-side half of the profile feature so non-local-PDS
authors also get their profile metadata indexed (the Jetstream
identity event stream only carries the handle, not display name /
bio / avatar). The PDS-push path was already wired by the previous
commit; this lands the Jetstream path.
Migration 0005:
* `profiles` table keyed by DID with display_name / description /
avatar_cid / banner_cid plus denormalised post_count /
follower_count / following_count. Backfilled from the posts
table on apply.
* `posts.avatar_cid` column — populated from the profiles cache
at `upsert_post` time so the PostCard can render an avatar
inline without a per-row PDS round trip.
Migration 0007 (clean-up): the original 0005 also created a
`LOWER(handle)` index that no query uses; this drops it
idempotently so dev DBs that already applied 0005 converge.
Indexer (`crates/appview/src/indexer.rs`):
* New `app.bsky.actor.profile` arm in `apply_commit` calls
`upsert_profile` on create, DELETEs the row on delete. Handle
is looked up from `posts` (the Jetstream commit envelope
doesn't carry it).
* `upsert_post` signature is now `&mut PostRow` so it can fill
`row.avatar_cid` from the profiles cache; the ON CONFLICT
clause uses `COALESCE(EXCLUDED, posts)` so re-indexing doesn't
overwrite an already-known avatar.
* `upsert_profile` writes display_name / description /
avatar_cid / banner_cid + the denormalised counts.
* `blob_link_of` helper accepts both `{ $type, ref.$link }`
and legacy flat `{ $link }` blob-ref shapes.
Ingest (`crates/appview/src/ingest.rs`):
* `app.bsky.actor.profile` create/delete arms in the PDS-push
path. The handle-fallback previously did `SELECT handle FROM
users WHERE did = $1` — but the AppView has no `users` table
(it's PDS-owned state). Replaced with a simple use-what-the-PDS-
sent approach; the handle_sync worker fills the column later.
Routes (`crates/appview/src/routes.rs`):
* `resolve_profile` reads the denormalised profile fields from
the cache. When no profile row exists the `post_count` fallback
uses a live `SELECT COUNT(*)` instead of `posts.len()`, so
prolific authors without a profile row report the real count
rather than the 50-post slice cap.
Tests (DB-gated, run when DATABASE_URL_APPVIEW is set):
* `blob_link_of_modern_shape` / `_legacy_flat_link` /
`_missing_field`.
* `upsert_profile_round_trip` — insert + replace semantics.
* `apply_commit_indexes_profile_create` — end-to-end Jetstream
arm + delete.
The AppView's handle_sync worker consulted the public PLC directory
and the did:web: HTTPS resolver only. DIDs hosted on the local PDS
(notably did🔑 users and any other operator-hosted method)
weren't reachable without an external round trip, and unresolvable
DIDs (did🔑 not on this PDS, did:foo: anything) blocked the
100-row batch forever because did🔑 sorts lexicographically
before did:plc: / did:web:.
This commit adds:
* `PdsHandleResolver` (at-identity) — POSTs the DID as the
`handle` field to the PDS's resolveHandle XRPC method. The PDS
now recognises a `did:` prefix and does a PK lookup on
`users.did`, returning `{did, handle}`. The resolver reads
the `handle` field, so the AppView finally gets a real local
handle for did🔑 users without ever dialing plc.directory.
* A 2 s timeout per request (was 10 s) and `DISPATCH_CONCURRENCY =
8` so the worker caps a 100-DID batch at ~2 s with parallel
dispatch instead of the ~17 min worst case the old serial + 10 s
setup allowed.
* A new `posts.handle_sync_attempted_at` column (migration 0006)
and `mark_attempted()` helper. The SELECT filter excludes rows
attempted within the last hour, so an unresolvable DID dominates
at most one batch before the worker advances. Cleared on success.
* `PDS_INTERNAL_URL` config so the AppView can reach the PDS via
a cluster-internal hostname when the public URL isn't routable
from inside the cluster.
Tests:
* `crates/at-identity/src/pds_handle.rs` — 4 unit tests against a
stub HTTP server (200/404/5xx/missing-did-field).
* Existing handle_sync integration tests updated to wire in the
new `pds_resolver` field.
The handle-sync worker picks the next BATCH_SIZE=100 distinct
DIDs with 'handle = \"\"' via 'ORDER BY did LIMIT 100'. But
'alphabetically' (which is what ORDER BY on a text column
produces) puts 'did🔑' before 'did:plc:' before 'did:web:'.
The 'dispatch' function returns 'Ok(None)' for 'did🔑'
(no resolver exists), counts that as 'skipped', and exits the
batch. Result: every pass processes the same ~3700 'did🔑'
rows first and never reaches any resolvable 'did:plc:' DID.
Fix at the SQL layer: 'WHERE did LIKE \"'did:plc:%\"' OR did
LIKE \"'did:web:%\"' \"'\") so every batch is real work. After
the first run on a fresh start, 'resolved=254 failed=0 skipped=0'
instead of 'resolved=0 failed=0 skipped=100'.
Jetstream 'identity' events are emitted every time a DID's handle
changes; the 'account' variant sometimes carries the verified
handle too. The AppView was logging both kinds as 'identity event
(logged only)' and 'account event (logged only)' — discarding the
attached handle.
Now we extract 'identity.handle' (falling back to
'account.handle'), and run it through a new
'indexer::backfill_handle(db, did, handle)'. The
'WHERE handle IS DISTINCT FROM $1' guard makes the UPDATE a
no-op when the value is already correct, so concurrent PDS
ingests and identity replays never fight.
End-of-stale-state: existing rows whose 'identity' event fired
before this code shipped will still be empty. Those get back-filled
over time as DIDs re-emit identity events, plus the 5-minute
handle-sync worker (next commit) handles the bulk for the rest.
The AppView side of the PDS→AppView ingest path. Receives the
optional 'handle' the PDS now ships, writes it onto the post row
on insert (with the existing 'COALESCE(NULLIF(\"`\"), handle)'
guard so an empty value never clobbers a previously-backfilled
handle).
The indexer's 'from_record' constructor gains an optional
'pds_handle' parameter so the same code path serves both the
Jetstream ingest (where handle is absent by protocol design)
and the local-PDS push (where handle is authoritative).
Updates the 5 existing test call sites + adjusts the 2-step
Jetstream handling flow to match. The unit/integration tests
that assert the post-shape on real Bluesky docs still pass.
resolve_profile returned 404 when the requested handle had no
posts in the local index — true for every local-PDS account
that hasn't yet had its posts ingested through the Jetstream
consumer. The UI then surfaced this as a raw 'err: appview:
profile returned 404' toast instead of an empty profile.
Return a profile row with the requested handle and zero counts
instead. The DID is left empty; the UI's 'copy did' button just
copies an empty string and the header falls back to the handle,
which is the right behaviour for a never-indexed local user.
tomdebone