Repository navigation
Conversation
nextPartName() separated same-millisecond appends by advancing the
timestamp past the clock. The borrowed milliseconds then belong to
whoever writes next, so a second instance sharing the bucket stamps a
part with a millisecond the first instance had already taken, and its
part sorts inside the earlier burst.
From [a, b] on one instance and a third entry from another a millisecond
later, load() returned the second instance's entry in position two
instead of last. Across 300 appends from two instances, 1747 of 44460
pairs written at different wall-clock times came back inverted. One
instance stayed correct, which is why the conformance suite -- a single
store per case -- never saw it.
A per-millisecond sequence in the name separates a burst without moving
the timestamp: part-{epochMs13}-{seq6}-{rand6}.jsonl. Cross-writer
inversions go to zero at one, two and three instances, and an instance's
own parts stay in write order even if the clock steps backwards, which
the old max(now, lastMs + 1) was also protecting against.
listSessions() reads both name shapes, so a bucket written by an earlier
version keeps its timestamps and still loads in order.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
nextPartName()separated same-millisecond appends by advancing the timestamp past the clock (Math.max(now, this.lastMs + 1)). Those borrowed milliseconds then belong to whoever writes next, so a second instance sharing the bucket stamps a part with a millisecond the first instance had already taken, and its part sorts inside the earlier burst.a's burst is namedT … T+4, sob's part — stamped with the realT+1— lands in position two. The rand suffix only breaks ties inside one millisecond; it cannot undo a name that claims a millisecond that has not arrived.This matters because the transcript is what resume replays. Across 300 appends from two instances, 1747 of 44460 pairs written at different wall-clock times came back inverted:
A single instance was always correct, which is why the conformance suite did not catch it — every case builds one store, and one writer is the case that works.
The fix keeps the timestamp honest and puts the burst counter in its own field:
The millisecond orders parts across writers, the sequence orders one instance's appends inside a millisecond, and the rand suffix breaks what is left. A clock that steps backwards keeps the last millisecond and advances the sequence, so an instance's own parts stay in write order — the property
Math.max(now, lastMs + 1)was also providing.listSessions()matches both name shapes, so a bucket written by an earlier version keeps its timestamps and still loads in order. Parts of the two shapes are only unordered relative to each other inside a single shared millisecond, where two writers have no ordering to preserve anyway.Three tests added: a burst against a second writer on one bucket, a backwards clock, and
listSessions()reading a part written before the sequence field. The first fails onmain. The existing same-millisecond ordering test and the 13 conformance checks pass unchanged; 25 tests ins3/, and the Redis and Postgres adapters are untouched.