Repository navigation
fix: use block root as states key in Store::from_genesis - #21
Merged
Merged
Conversation
tcoratger
pushed a commit
to tcoratger/ethlambda
that referenced
this pull request
Oct 30, 2026
* feat(types): write beacon scalars the way the Beacon API reads them
* refactor(types): serialize beacon scalars without allocating
* docs(types): correct the hex_array allocation rationale
The paragraph was adapted from quoted_or_bare's and kept its example: no
Validator field routes through hex_array, so the per-validator cost it
claimed does not exist. The fixed-width byte types it serves appear a
handful of times per state.
* feat(types): add the collection and bitfield serde adapters
* refactor(types): make a mis-annotated ssz_hex field a compile error
The adapter took any SszEncode, so annotating a container field with it
would compile and emit an opaque hex blob where a JSON object belongs.
The guard test planned for the container rollout walks for bare numbers
and cannot see that, so nothing else would catch it across 84 hand
annotations. Bound it to the bitfield and byte-list families instead,
and pin the bitfield encodings the tests only sampled by prefix.
* feat(types): serialize the beacon byte newtypes and uint256
Gives the five byte newtypes and U256 their own Serialize, so container
fields holding them need no attribute at all in the annotation pass.
U256 writes decimal, not hex, which the Beacon API requires and which no
primitive integer can hold: it goes through long division over the
big-endian bytes.
* feat(types): serialize the shared and phase0 containers as JSON
Annotates the first 28 of the 84 containers and adds the guard test the
rest of the pass is verified by: it walks a serialized block and fails on
any bare JSON number, naming the field path, since a forgotten attribute
is valid JSON and silently wrong.
Drops the SszHexEncodable marker trait added alongside ssz_hex: the
adapter takes any SszEncode again, so an ssz_hex annotation on a
non-bitfield field is a hand-checked invariant rather than a compile
error.
* test(types): make the interior-zero-byte case actually interior
The test set only byte 17, so every nonzero byte sat at one end and the
docstring's claim of nonzero bytes above and below the zero run was not
true of the value being asserted. Set the lowest byte too, so sixteen
zero bytes sit between two nonzero ones and the carry genuinely has to
cross them.
* feat(types): serialize the altair containers as JSON
Altair's additions need the three adapters told apart carefully: the
sync committee bits are a bitvector and go out as hex, the participation
lists are ParticipationFlags, a u8 alias, and go out as quoted integers,
and the committee pubkeys are a vector of a type that serializes itself.
* test(types): make the JSON guard reach what it claimed to
The guard built its block from BeaconBlockBody::default(), so every list
serialized as [] and the walker never descended into an element type, and
no test constructed a BeaconState at all. It covered 4 of 28 structs.
Populates every list with real elements, one level into nested lists, and
adds a state test per fork, taking the reach to 20/28 for phase0+shared
and 6/11 for altair; the rest are gossip-only types no block or state can
contain. Also adds a test that the walker fails on an unannotated integer,
so a green suite cannot be confused with a walker that matches nothing.
* feat(types): serialize the bellatrix containers as JSON
Bellatrix brings the execution payload, and with it the first fields the
adapter table does not answer on its own. extra_data and logs_bloom are
byte strings that must go out as one hex string each, though both would
also compile as arrays of numbers. transactions is a sequence of byte
lists, which no single adapter produces: it composes ssz_hex per element
through seq, locally, until a second fork needs the same shape.
* feat(types): serialize capella, and share the byte-string sequence adapter
Capella imports bellatrix's own Transactions type, so the composition
bellatrix kept private became a second call site. Promotes it to
serde_helpers as ssz_hex_seq and repoints bellatrix at it, which is a
net 44 lines out of that file.
The bound is Deref<Target: SszEncode> rather than SszEncode: serde hands
serialize_with a reference, and unlike Display and Serialize, SszEncode
has no blanket impl for &T.
* feat(types): serialize the deneb containers as JSON
The blob commitments are the field worth naming: a sequence of
KzgCommitment, which is a byte newtype that already writes itself as hex,
so it takes seq rather than ssz_hex_seq. Both spellings emit an array of
hex strings, so the guard test cannot tell them apart and the choice has
to come from the element type.
* feat(types): serialize the electra and fulu containers as JSON
Completes the annotation pass: all 84 containers now derive Serialize.
Fulu is where both sequence-of-hex spellings appear side by side. The
data column is a list of Cell, a foreign byte vector with no Serialize
of its own, so it takes ssz_hex_seq; the commitments and proofs beside
it are byte newtypes that write themselves, so they take seq. A single
Cell, as in MatrixEntry, is neither and takes ssz_hex.
Electra turns out to import its execution payload from deneb, and fulu
has no block types at all, so their fixtures reuse rather than
duplicate.
* feat(types): serialize lean signed blocks as JSON
Lets /lean/v0/blocks/finalized answer JSON when a caller asks for it.
Integers stay bare: that is lean's own encoding and the rpc crate's
tests assert it. The proof blob takes the 0x-prefixed convention the
roots and bitlists use, rather than the prefix-less one the fixed-width
XMSS pubkeys use, since it is opaque variable-length data and not an
identity.
* feat(types): serialize the container enums
Neither enum adds a variant tag: the Beacon API carries the fork in the
response envelope's version field and its Eth-Consensus-Version header,
never inside the data object.
The two get there differently, and the asymmetry is the point. The block
enum derives untagged, because its Lean variant really is served as JSON
on /lean/v0/blocks/finalized. The state enum cannot: a derive would
demand a JSON encoding for lean's State, which is deliberately SSZ-only,
so it is hand-written and the lean variant returns a serde error instead.
* feat(rpc): negotiate response encoding on Accept
The Beacon API serves JSON unless the caller asks for
application/octet-stream, which is the order lighthouse serves too. The
lean surface defaults the other way on its two SSZ endpoints, so the
default stays the caller's to pass rather than a constant here.
Ranks the header's entries by their q weight: a client that accepts both
states its preference that way, and ignoring it would serve SSZ to a
caller who merely tolerates it.
base.rs's private ssz_response moves here rather than being duplicated,
since both surfaces now need it.
* feat(types): report a block's body root on either chain
/eth/v1/beacon/headers/{id} answers with a SignedBeaconBlockHeader,
whose body_root is the one field the existing accessors cannot produce:
signed_beacon_block_accessors! hands back a field verbatim, and every
fork stores a different body container, so what the caller wants is the
merkle root of whichever one this is.
Dispatched including lean, since lean's Block has a body too and
/lean/v0 wants the same number.
The phase0 test computes the expected root through the raw libssz_merkle
trait rather than this crate's convenience wrapper, so a body_root that
only worked on the lean arm would fail rather than pass by
construction.
* feat(rpc): parse and resolve beacon block ids
Parsing is chain-agnostic, resolution is not. resolve_beacon reads only
accessors that answer on either chain: Store::head_state and head_slot
panic on a beacon store, so the lean slot lookup stays where it is
rather than being repointed at BlockRoots, which covers the branch
ending at the head rather than everything a lean store was bootstrapped
with.
genesis resolves to the store's anchor rather than to slot 0: a
checkpoint-synced directory has no genesis block, and its anchor is the
earliest block it can answer for at all.
* feat(types): serialize the beacon chain config as JSON
/eth/v1/config/spec echoes the chain configuration in the Beacon API's
encoding: SCREAMING_SNAKE_CASE keys, every integer a quoted decimal,
byte strings 0x-prefixed hex. Config could only be deserialized, which
is how --network <dir> loads a network.
74 fields move from deserialize_with = "m::deserialize" to with = "m",
which resolves to the same function on read, so the existing YAML
parsing tests staying green is the evidence that nothing about loading
a network moved.
Six fields keep their own arrangement: genesis_time stays skipped since
it is not a config.yaml key and /eth/v1/beacon/genesis reports it;
terminal_total_difficulty and terminal_block_hash already have
Serialize on their own types; max_blobs_per_block_deneb keeps its
rename; and blob_schedule pairs its hand-written deserializer with
seq::serialize, since its two halves do not live in one module.
* feat(rpc): add the beacon API envelope and error shape
Every fork-versioned Beacon API payload travels in
{version, execution_optimistic, finalized, data}. The fork sits in the
envelope rather than in the data because the containers serialize
untagged, so version is how a caller knows which shape it just parsed.
The error body is {code, message}, deliberately not the lean surface's
{error}. They are two different APIs and each consumer parses the shape
its own specification names, so unifying them would break one of the
two.
The submodule declarations and routes() land with the files they name,
rather than being written now and commented out.
* feat(rpc): serve beacon blocks by id in JSON and SSZ
/eth/v2/beacon/blocks/{block_id} and its /root sibling, JSON by default
and SSZ on Accept: application/octet-stream, both tagged with
Eth-Consensus-Version so an SSZ caller can tell which fork's container
it received.
genesis is now refused rather than resolved to the anchor slot.
Table::BlockRoots is written only by update_checkpoints, which computes
its delta by walking from the old head to the new one and returns early
when they are the same root. At bootstrap they always are, so the
anchor's own slot is never indexed and the previous spelling could only
ever 404. Saying so is better than a lookup that silently never works.
The fixture moves its head through update_checkpoints rather than
poking the backend, so BlockRoots is populated by its real writer and
the slot-resolution tests exercise the path a running node takes.
get_block_root loads the block rather than only resolving the id: it
needs the slot to answer finalized honestly instead of hardcoding it.
* feat(rpc): serve beacon block headers
/eth/v1/beacon/headers/{block_id}. Every field of the message comes
from SignedBeaconBlock's accessors, which dispatch including lean, plus
body_root(); only the envelope and the signature are beacon-specific,
which is what makes this the one endpoint genuinely shared with the
lean surface.
canonical is asked of BlockRoots rather than assumed, even for a block
reached by its own root, so a sibling or a block at a slot the index
does not cover answers false instead of being taken on trust. The test
pins both directions: the head is canonical, the anchor is not, since
its slot is never indexed.
The quoting comes from ethlambda_types::beacon::serde_helpers rather
than a to_string() per field, so the rule lives in one place.
* feat(rpc): serve beacon states and finality checkpoints
/eth/v2/debug/beacon/states/{state_id} is the path checkpoint_sync.rs
already fetches from other clients, so serving it makes this client
checkpoint-syncable from itself. The SSZ test decodes the body back
through slot_from_ssz, since round-tripping is the whole job.
A 0x... state_id is refused with a message naming the ids that do
work. States are keyed by block root here with no reverse index, so
the alternative to refusing is answering with the wrong state, or
being right only when the two roots happen to coincide.
finality_checkpoints reports the state's own three checkpoints rather
than the store's fork-choice view: the endpoint is defined as a read
of the state the id names, and the state is the only place a previous
justified checkpoint is kept at all.
JSON serializes through the Arc rather than cloning, since a mainnet
state runs to hundreds of megabytes.
* feat(rpc): serve beacon genesis and the spec config
/eth/v1/beacon/genesis reads genesis_validators_root off the finalized
anchor's state rather than looking for slot 0: it is a property of the
chain that every state carries, and a checkpoint-synced directory has
no genesis block to read it from.
/eth/v1/config/spec echoes the Config the store was bootstrapped with,
so a node started with --network <dir> reports that network's constants
rather than mainnet's. GENESIS_TIME is absent from it by design, since
it is not a config.yaml key and the genesis endpoint reports it.
* feat(rpc): serve the beacon node endpoints
syncing reads beacon_head rather than head_slot, which is lean-only and
panics on a beacon store, and computes the wall slot in milliseconds the
way /lean/v0/node/syncing does: slot_duration_ms is the value a loaded
network can actually change, and seconds_per_slot would truncate a
sub-second cadence to zero.
health is 206 while syncing and 200 once caught up. 503 would mean
uninitialized, and a store that answers at all is initialized.
identity reports peer_id and metadata only. enr, p2p_addresses and
discovery_addresses are empty, which is not spec-valid: the ENR is
built for discv5 and owned by the P2P actor, and BuiltSwarm hands
run_node only a local_peer_id, so serving the record means widening
the ethlambda-p2p surface. Deliberate and recorded as a deviation.
* feat(rpc): serve the beacon API from a beacon node
Which HTTP surface a node serves now follows from the store's own chain
tag rather than from the sub-command, so the two cannot disagree.
Until now a beacon node bound --api-port to the lean router off a
beacon store. Those handlers reach for lean state variants and metadata
keys a beacon directory never carries, so calling one panicked that
request. The beacon router replaces it rather than extending it, and
the integration test pins both halves: /eth/v1/node/version answers,
/lean/v0/node/syncing is a 404.
build_beacon_api_router does not merge the metrics router, for the same
reason build_api_router does not: start_http_servers serves it, and
merging a path twice makes axum panic at startup.
start_beacon_rpc_server takes no AggregatorController and no EventBus.
A follower has no aggregator duty to toggle, and the chain-events
stream is part of the lean surface.
* feat(rpc): serve the lean finalized block as JSON on request
SSZ stays the default here, which is the opposite of the beacon
surface and deliberately so: checkpoint_sync.rs reads these bytes, and
other clients' lean checkpoint sync may send no Accept header at all.
Moving the default would break every one of them, so JSON is opt-in.
Not routed through Encoding::from_accept, which defaults to JSON. The
asymmetry is the point, so the branch is spelled out here rather than
hidden behind a shared name that means the other thing.
The existing SSZ test already pinned the default; the new test pins
JSON on request, and that lean integers stay bare rather than picking
up the beacon surface's quoting.
* docs: describe the beacon API surface and its deviations
docs/rpc.md gains the endpoint table, the encoding rules (JSON by
default, SSZ on Accept, quoted integers, Eth-Consensus-Version) and the
three ids that are refused.
docs/spec_deviations.md records two: the partial node identity, and the
ids that cannot be named. The second is a property of Table::BlockRoots
rather than a choice, so it is written down where the reason lives.
CLAUDE.md said a /lean/v0 call on a beacon node panics the request and
that giving the follower its own surface was still a change of its own.
Both stopped being true with this branch.
* fix(rpc): resolve the anchor's own slot, which checkpoint sync asks for
Found by running a mainnet follower and pointing a second one at its
API: the second died with "peer served no block at the anchor slot
15265888" and never started.
checkpoint_sync.rs reads a peer's finalized state, takes the anchor
slot from it, and asks that peer for /eth/v2/beacon/blocks/{slot}.
That is exactly the one slot Table::BlockRoots never holds, because
update_checkpoints is its only writer and writes nothing when the head
does not move, which is the situation at bootstrap. So the endpoint
404'd and this node was unusable as a checkpoint-sync source for any
client, not only for itself.
On an index miss the lookup now tries the roots the store can name and
accepts one only when block_entry confirms its block really sits at
the slot asked for, so a slot the store holds nothing at still 404s
and no request can be answered with the nearest checkpoint instead.
The test that pinned the old 404 asserted a bug, so it now asserts the
anchor resolves, with a sibling holding the line on slots below it.
The docs claimed a slot at or below the anchor was refused; only
genesis and a state-root id are.
* chore: drop the beacon API spec and plans from the tree
These are working notes for building the beacon API, not documentation
of it: a design sketch and two task lists written while the endpoints
were being implemented. They went in with the last fix by accident.
What they describe is already in the tree in a form that stays true as
the code changes: docs/rpc.md for the endpoints, docs/spec_deviations.md
for where we depart from the Beacon API spec. A checklist of tasks that
are now done has nothing left to tell a reader, and would only rot.
The files stay on disk, untracked, for as long as they are still useful
to the people writing the remaining endpoints.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.