Remove general-purpose content hashing #66

New Issue

2026-06-02T15:46:18+02:00

Brummel commented

2026-06-02 15:46:18 +02:00

Motivation

The content hash (def_hash / module_hash / type_hash, BLAKE3 over
the canonical JSON-AST) was introduced on the thesis that an LLM author
references symbols by hash to verify identity "without spending context
window" (CLAUDE.md, the AILang-identity section; crates/ailang-core/src/lib.rs:11).

That thesis is not borne out. No LLM author references a symbol by hash:
the Form-A grammar has no syntax for a hash reference, the AST has no
hash-reference field, and the workspace loader resolves imports by name
only. The hash flows out of the toolchain (manifest display, diff,
registry keys) but never back in as anything an author writes. The
content-addressing surface is structurally absent, not merely unused.

A content hash has exactly one real use: fast equality comparison. Where
that is needed, a specialised local key is the better tool — it is
collision-safer by construction (a small, bounded value-base rather than
the global space of all defs) and clearer at the comparison site. A
single global hash buys cross-codebase uniformity, which is worth less
than collision-safety and locality.

Separately, the persistent hash-pin test values
(crates/ailang-core/tests/hash_pin.rs,
crates/ailang-surface/tests/prelude_module_hash_pin.rs,
crates/ail/tests/mono_hash_stability.rs,
crates/ailang-core/tests/embed_export_hash_stable.rs) carry recurring
regeneration cost. The prelude pin alone records six intentional re-pins
and zero documented accidental catches
(prelude_module_hash_pin.rs:13-48); five separate value-pins assert the
identical sum/IntList hashes (hash_pin.rs:79,101,155,170,279).

In scope — removed

def_hash — crates/ailang-core/src/hash.rs:43; re-export at lib.rs:78.
module_hash — crates/ailang-core/src/workspace.rs:409; re-export at lib.rs:79.
type_hash — crates/ailang-core/src/canonical.rs:65.
The three blake3::hash call sites: canonical.rs:67, hash.rs:45, workspace.rs:411.
All four persistent hash-pin test files (listed above).
The content-addressing claims in CLAUDE.md, lib.rs:11,
design/glossary.md (the content-addressed and JSON-AST … hashable
entries), and the hash mention in design/contracts/0009-roundtrip-invariant.md.

Explicitly NOT in scope — retained

canonical::to_bytes and the entire canonicalisation machine
(crates/ailang-core/src/canonical.rs:47 and below: sorted keys, no
whitespace, IEEE-754 float-bit encoding). This is the substrate,
not the hash. It carries the round-trip invariant
(design/contracts/0009-roundtrip-invariant.md) independently of any
hash, and BLAKE3 merely ran on top of it.
#[serde(skip_serializing_if = …)] on AST fields. Still needed for
round-trip idempotency and compact canonical form; its rationale
shifts from "hash-stable" to "round-trip-stable", so
design/contracts/0002-data-model.md wording is amended, not the
attribute removed.

Replacement per consumer (proposals — not yet designed in detail)

Instance-resolution key (type_hash, the largest leg): the registry
and instance lookup key (class, type_hash(t)) is used across
ailang-check — mono.rs:139,605,673,678,903,1722,1982,
lib.rs:2627,2778,2887,…, and workspace.rs:653,660,731,747. Proposal:
key on the canonical type representation directly (the normalised
type's canonical bytes/string), collision-free by construction, no hash.
This is the highest-risk leg — it touches the hot path of instance
resolution.
Drop-mono symbol mangling (type_hash at
crates/ailang-codegen/src/dropmono.rs:68): needs a fixed-length,
LLVM-symbol-safe token. Proposal: a local mangling hash specialised to
the type-argument domain — this is the one site where a hash earns its
place, and a bounded local hash is collision-safer than the global one.
Double-load guard (module_hash at workspace.rs:1564): proposal —
direct canonical-bytes comparison, no hash.
ail diff (def_hash/module_hash at main.rs:1626-1660,
2025-2058; ailang-check/src/lib.rs:1491): proposal — structural /
canonical-byte comparison instead of hash comparison.
ail manifest (def_hash at pretty.rs:34, main.rs:396,521-546,
737): proposal — drop the hash column, show signatures only.
mono_hash_stability pin: replace the body-hash assertion with a
golden pretty-printed term, which is a diagnosable, human-readable
"hash" — a drift shows what changed, not just a → b.
Additivity pins (hash_pin.rs value-pins, embed_export_hash_stable.rs):
delete or rewrite as the value-free two-path property tests already
present (hash_pin.rs::iter19b_empty_suppress_preserves,
iter22b1_classdef_empty_optionals_hash_stable,
forall_without_constraints_hashes_bit_identical). These cover the
skip_serializing_if invariant structurally, without any frozen value.

Acceptance

No blake3 dependency or call remains in the production crates.
No frozen hex hash value remains in any test.
Round-trip invariant tests (design/contracts/0009) stay green.
Instance resolution + monomorphisation pass the full workspace suite.
Content-addressing claims removed from CLAUDE.md, lib.rs,
design/glossary.md, design/contracts/0009.

Shape

A non-breaking removal: no language semantics, on-disk format, or ABI
changes. The only observable change is cosmetic — ail manifest and
ail diff stop printing hash columns. Internally the instance-resolution
re-key (the type_hash leg) is the delicate part and the natural first
task to land and verify in isolation; the rest (double-load guard, CLI,
test-pin removal, doc cleanup) is mechanical. This is the tracking item,
not the design.

## Motivation The content hash (`def_hash` / `module_hash` / `type_hash`, BLAKE3 over the canonical JSON-AST) was introduced on the thesis that an LLM author references symbols by hash to verify identity "without spending context window" (`CLAUDE.md`, the AILang-identity section; `crates/ailang-core/src/lib.rs:11`). That thesis is not borne out. No LLM author references a symbol by hash: the Form-A grammar has no syntax for a hash reference, the AST has no hash-reference field, and the workspace loader resolves imports by name only. The hash flows out of the toolchain (manifest display, diff, registry keys) but never back in as anything an author writes. The content-addressing surface is structurally absent, not merely unused. A content hash has exactly one real use: fast equality comparison. Where that is needed, a *specialised local* key is the better tool — it is collision-safer by construction (a small, bounded value-base rather than the global space of all defs) and clearer at the comparison site. A single global hash buys cross-codebase uniformity, which is worth less than collision-safety and locality. Separately, the persistent hash-pin test values (`crates/ailang-core/tests/hash_pin.rs`, `crates/ailang-surface/tests/prelude_module_hash_pin.rs`, `crates/ail/tests/mono_hash_stability.rs`, `crates/ailang-core/tests/embed_export_hash_stable.rs`) carry recurring regeneration cost. The prelude pin alone records six intentional re-pins and zero documented accidental catches (`prelude_module_hash_pin.rs:13-48`); five separate value-pins assert the identical `sum`/`IntList` hashes (`hash_pin.rs:79,101,155,170,279`). ## In scope — removed - `def_hash` — `crates/ailang-core/src/hash.rs:43`; re-export at `lib.rs:78`. - `module_hash` — `crates/ailang-core/src/workspace.rs:409`; re-export at `lib.rs:79`. - `type_hash` — `crates/ailang-core/src/canonical.rs:65`. - The three `blake3::hash` call sites: `canonical.rs:67`, `hash.rs:45`, `workspace.rs:411`. - All four persistent hash-pin test files (listed above). - The content-addressing claims in `CLAUDE.md`, `lib.rs:11`, `design/glossary.md` (the `content-addressed` and `JSON-AST … hashable` entries), and the hash mention in `design/contracts/0009-roundtrip-invariant.md`. ## Explicitly NOT in scope — retained - `canonical::to_bytes` and the entire canonicalisation machine (`crates/ailang-core/src/canonical.rs:47` and below: sorted keys, no whitespace, IEEE-754 float-bit encoding). This is the **substrate**, not the hash. It carries the round-trip invariant (`design/contracts/0009-roundtrip-invariant.md`) independently of any hash, and BLAKE3 merely ran on top of it. - `#[serde(skip_serializing_if = …)]` on AST fields. Still needed for round-trip idempotency and compact canonical form; its *rationale* shifts from "hash-stable" to "round-trip-stable", so `design/contracts/0002-data-model.md` wording is amended, not the attribute removed. ## Replacement per consumer (proposals — not yet designed in detail) - **Instance-resolution key** (`type_hash`, the largest leg): the registry and instance lookup key `(class, type_hash(t))` is used across `ailang-check` — `mono.rs:139,605,673,678,903,1722,1982`, `lib.rs:2627,2778,2887,…`, and `workspace.rs:653,660,731,747`. Proposal: key on the canonical type representation directly (the normalised type's canonical bytes/string), collision-free by construction, no hash. This is the highest-risk leg — it touches the hot path of instance resolution. - **Drop-mono symbol mangling** (`type_hash` at `crates/ailang-codegen/src/dropmono.rs:68`): needs a fixed-length, LLVM-symbol-safe token. Proposal: a *local* mangling hash specialised to the type-argument domain — this is the one site where a hash earns its place, and a bounded local hash is collision-safer than the global one. - **Double-load guard** (`module_hash` at `workspace.rs:1564`): proposal — direct canonical-bytes comparison, no hash. - **`ail diff`** (`def_hash`/`module_hash` at `main.rs:1626-1660`, `2025-2058`; `ailang-check/src/lib.rs:1491`): proposal — structural / canonical-byte comparison instead of hash comparison. - **`ail manifest`** (`def_hash` at `pretty.rs:34`, `main.rs:396,521-546`, `737`): proposal — drop the hash column, show signatures only. - **`mono_hash_stability` pin**: replace the body-hash assertion with a golden pretty-printed term, which is a diagnosable, human-readable "hash" — a drift shows *what* changed, not just `a → b`. - **Additivity pins** (`hash_pin.rs` value-pins, `embed_export_hash_stable.rs`): delete or rewrite as the value-free two-path property tests already present (`hash_pin.rs::iter19b_empty_suppress_preserves`, `iter22b1_classdef_empty_optionals_hash_stable`, `forall_without_constraints_hashes_bit_identical`). These cover the `skip_serializing_if` invariant structurally, without any frozen value. ## Acceptance - [ ] No `blake3` dependency or call remains in the production crates. - [ ] No frozen hex hash value remains in any test. - [ ] Round-trip invariant tests (`design/contracts/0009`) stay green. - [ ] Instance resolution + monomorphisation pass the full workspace suite. - [ ] Content-addressing claims removed from `CLAUDE.md`, `lib.rs`, `design/glossary.md`, `design/contracts/0009`. ## Shape A non-breaking removal: no language semantics, on-disk format, or ABI changes. The only observable change is cosmetic — `ail manifest` and `ail diff` stop printing hash columns. Internally the instance-resolution re-key (the `type_hash` leg) is the delicate part and the natural first task to land and verify in isolation; the rest (double-load guard, CLI, test-pin removal, doc cleanup) is mechanical. This is the tracking item, not the design.

Brummel added the feature label 2026-06-02 15:46:18 +02:00

Brummel referenced this issue from a commit

2026-06-02 22:24:24 +02:00

test: prune duplicate tests, re-sight two blind coverage guards

Brummel referenced this issue from a commit

2026-06-03 11:29:42 +02:00

docs(design): mark project RETIRED (paused) at the ledger spine

Sign in to join this conversation.