Skip to content

branch-4.1: [fix](compaction): avoid decrementing uncounted cumulative compaction threads #66752 - #67662

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
Yukang-Lian:codex/pick-66752-branch-4.1
Sep 9, 2026
Merged

yiguolei merged 1 commit into
apache:branch-4.1from
Yukang-Lian:codex/pick-66752-branch-4.1

Conversation

@Yukang-Lian

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: None

Related PR: #66752

Problem Summary:

Backport the cumulative compaction thread-accounting fix from #66752 to branch-4.1.

When tablet global compaction lock acquisition fails, the worker exits before _cumu_compaction_thread_pool_used_threads is incremented. The deferred cleanup previously decremented the counter unconditionally, which could make it negative and distort compaction scheduling capacity.

This backport preserves the 4.1 compaction structure and guards cleanup so the counter is decremented only after it has actually been incremented. It also carries the focused C++ regression test and the Cloud fault-injection regression suite from the source fix.

Local validation:

  • git diff --check
  • Clang Format 16 dry-run for the changed C++ files
  • clang++ -fsyntax-only for cloud_storage_engine.cpp and cloud_compaction_test.cpp using an existing 4.1 ASAN UT compile database
  • Groovy 4.0.19 parse for test_cloud_cumu_compaction_global_lock_thread_count.groovy

The focused C++ and Docker regression tests were not executed locally; hosted CI remains required.

Release note

Fix Cloud cumulative compaction thread accounting after global lock acquisition failures.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

    Test cases:

    • CloudCompactionTest.cumulative_global_lock_failure_keeps_thread_count_balanced
    • test_cloud_cumu_compaction_global_lock_thread_count

… threads (apache#66752)

Backport the cumulative compaction thread-accounting fix and its regression coverage to branch-4.1.

(cherry picked from commit d113b43)
Signed-off-by: Yukang-Lian <lianyukang@selectdb.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Yukang-Lian

Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 74.35% (31366/42188)
Line Coverage 58.54% (348840/595862)
Region Coverage 55.20% (290617/526520)
Branch Coverage 56.07% (131044/233721)

@yiguolei
yiguolei merged commit 33c9350 into apache:branch-4.1 Sep 9, 2026
34 of 36 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

yiguolei pushed a commit that referenced this pull request Sep 10, 2026
…e compaction threads #66752 (#67662)

### What problem does this PR solve?

Issue Number: None

Related PR: #66752

Problem Summary:

Backport the cumulative compaction thread-accounting fix from #66752 to
`branch-4.1`.

When tablet global compaction lock acquisition fails, the worker exits
before `_cumu_compaction_thread_pool_used_threads` is incremented. The
deferred cleanup previously decremented the counter unconditionally,
which could make it negative and distort compaction scheduling capacity.

This backport preserves the 4.1 compaction structure and guards cleanup
so the counter is decremented only after it has actually been
incremented. It also carries the focused C++ regression test and the
Cloud fault-injection regression suite from the source fix.

Local validation:

- `git diff --check`
- Clang Format 16 dry-run for the changed C++ files
- `clang++ -fsyntax-only` for `cloud_storage_engine.cpp` and
`cloud_compaction_test.cpp` using an existing 4.1 ASAN UT compile
database
- Groovy 4.0.19 parse for
`test_cloud_cumu_compaction_global_lock_thread_count.groovy`

The focused C++ and Docker regression tests were not executed locally;
hosted CI remains required.

### Release note

Fix Cloud cumulative compaction thread accounting after global lock
acquisition failures.

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
  - [x] Regression test
  - [x] Unit Test
  - [ ] Manual test (add detailed scripts or steps below)
  - [ ] No need to test or manual test. Explain why:
    - [ ] This is a refactor/code format and no logic has been changed.
    - [ ] Previous test can cover this change.
    - [ ] No code files have been changed.
    - [ ] Other reason <!-- Add your reason? -->

  Test cases:

-
`CloudCompactionTest.cumulative_global_lock_failure_keeps_thread_count_balanced`
  - `test_cloud_cumu_compaction_global_lock_thread_count`

Signed-off-by: Yukang-Lian <lianyukang@selectdb.com>
Co-authored-by: dzr171712 <166372158+dzr171712@users.noreply.github.com>
@yiguolei yiguolei mentioned this pull request Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants