Repository navigation
Conversation
Report compatibility verification counters through host stdout so the Windows test can capture them. Verify that FC1 and FC2 both report zero missed corrections. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Tianlei Wu (tianleiwu)
previously approved these changes
Nov 13, 2026
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The diagnostic performs a host readback followed by cudaStreamSynchronize during CUDA graph capture, invalidating capture and causing inference to fail when compatibility verification is enabled.
1 open issue
What changed in this PR
This PR makes CUDA INT3 QMoE compatibility reports capturable on Windows by moving counter reporting from device-side printf to flushed host stdout.
Changes:
- Read compatibility counters back to the host and report them for FC1 and FC2.
- Strengthen the regression test to require zero missed corrections in both reports.
| File | Description |
|---|---|
onnxruntime/test/python/transformers/test_qmoe_shared_direct.py |
Require separate FC1 and FC2 missed=0 reports. |
onnxruntime/contrib_ops/cuda/moe/qmoe_int3_tc.cuh |
Replace device-side reporting with host readback, stream synchronization, and flushed output. |
Review effort: Balanced
The compatibility report performs a host readback and stream synchronization, which are illegal during graph capture. Skip only the diagnostic readback while capture is active. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
eserscor
requested review from
Tianlei Wu (tianleiwu)
and
a balanced review from Copilot
November 14, 2026 02:54
eserscor
enabled auto-merge (squash)
November 14, 2026 02:55
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The Windows CUDA stdout-capture fix still needs regression testing on a BF16 Tensor Core-capable Windows CUDA runner because static review cannot verify the platform-specific capture behavior.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Replace the CUDA device-side
printfused for INT3 QMoE compatibility counters with a host-side device-to-host counter readback and flushed stdout report. Update the test to require zero missed corrections in both FC1 and FC2 reports.Motivation and Context
The Windows CUDA CI test selects the Tensor Core path but does not capture the device-side
missed=0report, causingtest_cuda_int3_tensor_core_compatibility_shadowto fail. Host-side reporting makes the verification output capturable by the test while preserving its checks.Validation: Python syntax check and
git diff --checkpassed. The CUDA regression test was not run locally; the available T1000 does not support the BF16 Tensor Core path required by this test.