Repository navigation
fix(posting): set minTs for empty rollups, drop replayed commits belo… - #9863
shiva-istari wants to merge 1 commit into
Conversation
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes.
|
| Layer / File(s) | Summary |
|---|---|
Track empty-marker versions and guard cached replay posting/mvcc.go |
ReadPostingList records the empty marker’s version in minTs. updateItemInCache removes the cached list and stops when a commit timestamp is at or below minTs. |
Verify MVCC snapshots and cached replay posting/mvcc_test.go |
Tests cover reads around an empty rollup and replayed delete-all commits for empty and re-added posting lists. |
Verify query snapshots and backup restore systest/integration2/rollup_empty_list_test.go |
Integration tests cover earlier-start-timestamp queries across rollup and restoration after full and incremental backups. |
Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Bug fix · Severity of issue fixed: Medium
Suggested reviewers: matthewmcneely
Merge Risk: 
Minimal · previous review 59e3f
-
This change keeps deletions that empty a posting list correct across rollups. Snapshots taken before the delete still see the earlier values. Backup and restore no longer bring deleted values back. Replayed older commits no longer corrupt cached lists. No concrete defect was found, so the change looks ready to merge.
Pre-merge checks: 
4 · 
1
-
Failed checks: 1 warning Passed checks: 4 passed-
Full details: Linked Issues check
-
Explanation
The changes satisfy the coding requirements in [
#9856].ReadPostingListsetsList.minTstoitem.Version()forBitEmptyPosting, so reads before the delete can use older deltas while reads at or after the marker see an empty list.MemoryLayer.updateItemInCachedrops cached lists when a replayed commit hascommitTs <= minTs, which prevents the cache error and allows disk rebuild. The MVCC tests cover earlier snapshots and replayed commits for empty and complete rollups. The integration tests cover UID edges,count(),has()filters, and incremental backup/restore behavior.
Full details: Out of Scope Changes check
-
Explanation
All production changes and tests are connected to [
#9856]. The cache invalidation change supports replay safety for rolled-up posting lists. The unit and integration tests provide regression coverage for the corrected snapshot and backup/restore behavior. No unrelated changes are shown in the reviewed diff.
-
-
Finishing Touches
-
Generate docstrings
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
This comment has been minimized.
This comment has been minimized.
59e3f50 to
9c7d5fa
Compare
…w minTs When a delete empties a posting list, the incremental rollup rewrites the key as a BitEmptyPosting marker at commitTs+1. ReadPostingList did not set List.minTs for that marker (it does for BitCompletePosting), so: - on a cache miss MemoryLayer.ReadData, which reads the key at math.MaxUint64, served the empty list to every read ts, including transactions that started before the delete; and - ToBackupPostingList versioned the emptied key at 0, so restoring a full plus an incremental backup kept the older, non-empty version. Set minTs = item.Version() for BitEmptyPosting as well. The line was dropped in ee99a7a (#9237) and the bug first shipped in v24.1.0. With minTs set, a commit replayed from the Raft log after a restart can reach updateItemInCache with commitTs <= minTs of the cached list, because the rollup that already contains it is versioned newest commit + 1. Folding it in again puts a delta below minTs into the mutable layer, and if it is a delete-all every read the entry serves fails with "deleteBelowTs must be greater than the minTs in the list" until the entry is evicted. Drop the cache entry in that case instead; the next read rebuilds it from disk. This path was already reachable for complete rollups before this change. Tests: posting/mvcc_test.go gains the reporter's snapshot-read test and a replay test for both rollup kinds; systest/integration2/rollup_empty_list_test.go reproduces the earlier-start-ts read and the backup/restore case end to end. Fixes #9856
9c7d5fa to
85b10e5
Compare
|
Found 1 test failure on Blacksmith runners: Failure
|
mlwelles
left a comment
There was a problem hiding this comment.
[nit] Should the title say "evict cached lists on replayed commits at or below minTs" rather than "drop replayed commits"? The commit is still written; only the cache entry is evicted.
| time.Sleep(emptyRollupCooldown) | ||
| deleteAllFriends(t, gc, uids) | ||
| require.Equal(t, 0, countNodesWithFriend(t, gc)) | ||
|
|
||
| // Give the rollup time to rewrite the emptied keys. | ||
| deadline := time.Now().Add(emptyRollupWait / 2) | ||
| for time.Now().Before(deadline) { | ||
| require.Equal(t, 0, countNodesWithFriend(t, gc)) | ||
| time.Sleep(500 * time.Millisecond) | ||
| } | ||
|
|
||
| require.NoError(t, hc.Backup(c, false, dgraphtest.DefaultBackupDir)) |
There was a problem hiding this comment.
[minor] Since these end-to-end tests can still pass without the fix, should the backup case get a deterministic unit test like TestReadAtSnapshotAfterEmptyRollup? Roll the emptied key up, read it back with ReadPostingList, and assert ToBackupPostingList returns version delTs+1, which fails on main and passes here.
| // A commit at or below minTs is already in the rolled-up version this list was read from | ||
| // (rollups are versioned newest commit + 1); it gets here when the Raft log is replayed | ||
| // after a restart. Applying it again would put a delta below minTs into the list and | ||
| // break every read it serves, so drop the entry and let the next read rebuild it. | ||
| val.list.RLock() | ||
| minTs := val.list.minTs | ||
| val.list.RUnlock() | ||
| if commitTs <= minTs { | ||
| ml.del([]byte(key)) | ||
| return | ||
| } |
There was a problem hiding this comment.
[minor] Without this guard, a replayed single-edge delete that a later commit re-added makes the cached list silently drop the edge rather than error. Should the replay test cover that case too?
|
|
||
| switch item.UserMeta() { | ||
| case BitEmptyPosting: | ||
| l.minTs = item.Version() |
There was a problem hiding this comment.
[minor] Restore keeps the highest version per key, so an incremental backup taken by an affected release still has the emptied keys at version 0 and still restores the deleted edges. Should the description or release note tell users to take a fresh full backup after upgrading?
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
…w minTs
Fixes #9856
When a delete empties a posting list, the incremental rollup rewrites the key as a BitEmptyPosting marker at commitTs+1. ReadPostingList did not set List.minTs for that marker (it does for BitCompletePosting), so:
Set minTs = item.Version() for BitEmptyPosting as well. The line was dropped in ee99a7a (#9237) and the bug first shipped in v24.1.0.
With minTs set, a commit replayed from the Raft log after a restart can reach updateItemInCache with commitTs <= minTs of the cached list, because the rollup that already contains it is versioned newest commit + 1. Folding it in again puts a delta below minTs into the mutable layer, and if it is a delete-all every read the entry serves fails with "deleteBelowTs must be greater than the minTs in the list" until the entry is evicted. Drop the cache entry in that case instead; the next read rebuilds it from disk. This path was already reachable for complete rollups before this change.
Tests: posting/mvcc_test.go gains the reporter's snapshot-read test and a replay test for both rollup kinds;
systest/integration2/rollup_empty_list_test.go reproduces the earlier-start-ts read and the backup/restore case end to end.
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by CodeRabbit