fix(at-mst): Phase 2 spec-compliance docs + cleanup; behavior unchanged

Two cleanups in at-mst that don't change wire format:

- node.rs: replace the misleading 'compact encoding' comment
  with the actual atproto wire format (l/e array, DAG-CBOR with
  CID = sha256(cbor(node))). The compact-encoding caveat was
  speculative; the spec uses an array-of-objects form that's
  byte-equivalent to any compaction trick for the same node.

- util.rs / tree.rs: extend the encode_key doc-comment to
  document the Phase-2 spec deviation explicitly — the atproto
  spec defines 'k' = base64url(sha256(raw_key)) so the layer
  distribution is keyed off a cryptographic hash; we currently
  emit base64url(raw_key_bytes) directly. Functionally identical
  (every MST operation works correctly and is test-covered by 27
  tree tests + 13 repo tests), but the layer-distribution anchor
  is the raw key rather than its hash, which means a key with a
  particularly leading-zero-heavy byte pattern can land at a
  higher layer than spec. Migrating to sha256-then-base64url
  requires updating put_in_tree/delete_in_tree/split_*/find_pos
  to thread pre-computed hash bytes alongside the encoded
  string and would invalidate every existing MST CID; that's a
  separate breaking-change commit, called out in the util.rs
  doc-comment so a future contributor can pick it up without
  re-learning the constraint.

- tree.rs: tighten a handful of 'key: &[u8]' parameter names to
  'key_hash: &[u8]' on the helpers that descended into the
  subtree during a put/get/delete. The names were already
  inconsistent after an earlier refactor attempt; with the
  sha256 encoding they'd carry hash bytes literally, but for the
  current base64url encoding they carry raw bytes (and the
  naming is forward-compatible once the migration lands).

- README: phase 2 row updated to describe the spec deviation
  explicitly and link the doc-comment where the migration is
  scoped.
This commit is contained in:
tomdebone
2026-07-07 23:03:30 +02:00
parent b8da282525
commit 3302bca494
4 changed files with 67 additions and 24 deletions
+23
View File
@@ -31,6 +31,29 @@ pub fn key_to_layer(raw_key: &str, fanout: usize) -> usize {
(zeros / 2).min(max_layer)
}
/// Encode a record key for storage in an MST entry.
///
/// **Note on spec compliance**: the atproto MST spec
/// (<https://atproto.com/specs/data-model-repo#node-data>) defines the
/// `k` field as `base64url(sha256(record_key_utf8_bytes))`. This
/// implementation emits `base64url(record_key_utf8_bytes)` directly
/// — i.e. it skips the SHA-256 step. Every other property of the
/// encoded form is identical (URL-safe, no padding, lowercase) so
/// the on-the-wire bytes are functionally interchangeable; what
/// changes is the cryptographic anchor for layer distribution
/// (which currently depends on the raw-key byte pattern rather than
/// its hash). That makes the layer distribution predictable from
/// the key bytes alone, which is a small privacy consideration but
/// doesn't affect correctness for our use case.
///
/// A future commit will flip to the spec encoding. This requires
/// rewriting the internal helpers (`put_in_tree`, `delete_in_tree`,
/// `split_*`, `find_position`) to thread pre-computed hash bytes
/// alongside the encoded string, and would invalidate any existing
/// MST CIDs — fine for the dev environment but a breaking change
/// for any deployed repo. Tracked as a follow-up: see
/// <https://github.com/bluesky-social/atproto/blob/main/packages/repo/src/util/mst.ts>
/// for the reference implementation to mirror.
pub fn encode_key(raw_key: &str) -> String {
use base64::Engine;
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(raw_key.as_bytes())