Repository navigation
fix(rocm): keep FLASH SDPA when the AOTriton images are installed - #474
Open
lyx178238820-source wants to merge 53 commits into
Open
lyx178238820-source wants to merge 53 commits into
lyx178238820-source wants to merge 53 commits into
Conversation
added 30 commits
September 28, 2026 13:12
… OSes ROCm 10.0.0 (stable, 2026-08-26) publishes torch 2.12.0 and torchvision 0.27.0 cp312 wheels for Windows and Linux on https://stable.repo.amd.com/rocm/whl-next/ with per-GPU kernel packs for gfx1100-gfx1103, gfx1150/gfx1151 and gfx1200/gfx1201, so the Radeon 780M (gfx1103) gets kernels alongside every target the ROCm 7.2.1 build had. Both OSes now share one torch/torchvision pin, the Windows setup installs the same kernel packs as the Linux image keeps, and the README names the Adrenalin 26.8.1 driver ROCm 10 needs on Windows. The protection bump moves the Linux image to ROCm 10 and bundles the ROCm payload on Linux. Refs Kruk2#408
Bundle application and dependency license texts, record exact source and model correspondence, and include the legal payload in frozen builds.\n\nDocument the private protection source as the sole reproducibility exception.
18.1.0 contains the current-CUDA-context and explicit NVENC stream changes the release builds compiled from a pinned PyAV commit. Pin it exactly because the VALI release wheel links the same pyav-ffmpeg 8.1.2-1 libraries. Drop cython/delvewheel from the dev extra; only the PyAV rebuild used them.
Patch releases within the 2.12 line that torch-tensorrt 2.12.0 accepts; the dev environment already runs them.
Also records the protection commit that fixes the release scripts' modes.
Show where supporter keys are sold in the system-check wizard and in the license dialog, and give translated messages for keys that are not accepted.
Newer versions load a new checkpoint file name, so older versions keep downloading the file they expect. The download skips the old file so users fetch only the checkpoint this version needs.
Keep downloaded models usable after upgrading.
Explain activation failures and update supporter documentation.
…el-name ltx) Inference-only restorer for the fine-tuned LTX-2.5 model, run in phased passes with one model on the GPU at a time: scan (detect, cuts, plan) -> VAE encode -> denoise -> decode + composite + encode. - jasna/ltx: track planning with one zooming follow camera per track and 121-frame windows at hop 88; video-only transformer port with prequantized INT8 ConvRot linears; lockstep sampler with overlap fusion and shared-prefix STG; block streaming from pinned RAM for low-VRAM GPUs; GPU crop/composite. - jasna/models/ltx_vae: vendored pure-PyTorch subset of the ltx-core video VAE (causal encoder, single-step diffusion decoder, tiled decode). - CLI: --restoration-model-name ltx, --ltx-large-canvas, --ltx-seed; --restoration-model-path now defaults per model. - Licensing: LTX-2.x Community License text and notices.
Use shared model loading for all LTX components and report missing access before engine setup. Clarify model access after app updates.
--ltx-fast uses an NVFP4 model on RTX 50-series GPUs. SageAttention, torch.compile and NATTEN are used when installed.
Sigma schedule and STG scale come from the file's metadata; STG 0 skips the perturbed pass.
…e warning Progress advances after every sampler step, and the Hadamard matrix is built directly so Dynamo no longer warns.
Standard and LTX cards plus seed, fast mode and sharper large mosaics options. LTX is disabled without an NVIDIA GPU or its model files.
Segments without one follow the job's model, and overlapping ranges are split by the later one. Each LTX segment is planned from its own frames and all LTX spans run in one batch.
Each LTX pass reports its share of the run and the time left; the control bar shows the pass name in place of the FPS.
New ranges take the settings panel's model and seed, the selected range is edited in place, and "Use for all ranges" copies it. LTX ranges are teal on the timeline.
restore_spans runs prepare_spans, denoise_spans and compose_spans over one LtxRender, so prepared references can be denoised with another seed.
Try seed restores the selected range in the background like a job and keeps the prepared latents, so another seed reruns only denoise and decode. Use this seed applies a tried seed.
Inductor picked kernel configs by timing them, so the output changed between runs. The block forward now compiles in deterministic mode.
Start, End and the add button share a row when there is room, and the range settings scroll on their own.
An LTX frame counts as about 30 standard frames, and time left comes from the whole job.
cv2.fillPoly with several polygons uses the even-odd rule, so overlapping regions were left out of the blend mask. Fill each polygon separately.
blending.dilate_ellipse matches cv2.dilate with MORPH_ELLIPSE pixel for pixel and replaces the LTX tracing, LTX feather and SD 1.5 copies. LTX tracing runs about 2x faster.
… models --ltx-model picks the model; each variant has its own file name. A missing model is offered for download with its size (y/N in the CLI); the download location is not set yet.
A Model menu picks distilled or undistilled. A missing model is offered for download with its size and blocks Start until done. One download worker now serves SD 1.5 and LTX.
…ense LtxModelFiles.placeholder() serves seeded random tensors with the real names, shapes and dtypes (jasna/ltx/trial_shapes.py), so every pass runs as in a real job. The output is not restored.
A "Trial run" switch runs LTX with placeholder weights to test speed. It is forced on when no LTX model is installed and offered at Start when a license is missing.
…enable Flash Attention and Memory Efficient Attention
Enable LTX restoration for ROCM
The Windows ROCm wheels split the AOTriton payload across two device packages: the exact-arch package ships torch/.kpack, a separate family package ships torch/lib/aotriton.images/<family>. With only the first one installed the images are absent, the first FLASH attention launch submits a null kernel and the process dies with hipErrorInvalidImage (ROCm/TheRock#7315). Disabling FLASH/EFFICIENT unconditionally fixes that crash but also throws the fast path away wherever the images *are* present. RF-DETR's DINOv2 backbone is attention bound: on an RX 7900 XT at 576x576 the MATH backend costs 38.3 ms/frame against 18.8 ms/frame with FLASH, which is 42.9s -> 32.8s end to end on a 20s clip. aotriton_image_dir() now decides, without needing a GPU in the usual case: * no complete image set -> MATH (previous behaviour, still safe) * exactly one set installed -> use it, no device query * several sets installed -> match gcnArchName against the exact directory, then the family; otherwise stay on MATH TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL stays on, so the fast path is available on every Radeon the wheels support, and JASNA_FORCE_MATH_SDP=1 restores the conservative behaviour explicitly.
lyx178238820-source
added a commit
to lyx178238820-source/jasna
that referenced
this pull request
Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
configure_rocm_process_env()no longer disables FLASH/EFFICIENT SDPA unconditionally. It disables them only when the AOTriton kernel images for this GPU are actually missing 鈥?which is the state that crashes.TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTALis untouched, so the fast path stays available on every Radeon the wheels support.Why the images can be missing
On Windows the ROCm wheels split the AOTriton payload across two device packages, and neither is sufficient alone:
amd-torch-device-<exact arch>(e.g.gfx1100)torch/.kpack/torch_gfx1100.kpack鈥?the torch kernelsamd-torch-device-<family>(e.g.gfx110x)torch/lib/aotriton.imagesdoes not exist, the first FLASH launch submits a null kernel (hipModuleLaunchKernel(f=<null>, blockDim={0,0,0})) and the process dies withhipErrorInvalidImage鈥?[Windows][gfx1201] SDPA FLASH/EFFICIENT fail with hipErrorInvalidValue: aotriton_v2.dll ships without the kernel loader - bundled aotriton.images can never be loaded ROCm/TheRock#7315hipErrorInvalidKernelFile, because its kpack name is not recognised by the runtimeSo "images missing" is a real state that has to be handled 鈥?but forcing MATH on every machine also removes the fast path where the images are present.
Measured on RX 7900 XT (gfx1100), torch 2.14.0+rocm10.1.0, HIP 7.16, RF-DETR v6, batch 8, fp16:
End to end on a 20 s / 602 frame clip with
--detection-model rfdetr-v6: detection 30.2 s -> 19.0 s, whole run 42.9 s -> 32.8 s. YOLO-based detection is unchanged, which is a useful control (it does not use SDPA at all).Design
aotriton_image_dir()resolves the usable image set arch-agnostically:torch/lib/aotriton.images/*/flash/attn_fwd.zip->None-> MATH fallback (previous behaviour, still safe)gcnArchName, e.g.gfx1100) against the exact directory, then the family (gfx110x); if nothing matches, stay on MATHThe GPU is only queried in case 3, so the common single-set installation does not initialise the device just to answer this question.
JASNA_FORCE_MATH_SDP=1restores the conservative behaviour explicitly.This stays correct for every architecture the wheels ship: RDNA2 (
gfx103x, no AOTriton support upstream -> MATH), RDNA3 (gfx110x), RDNA4 (gfx120x), Strix Halo (gfx115x), CDNA parts, and any single-architecture installation.Relation to #466
#466 pins the AMD build to the MATH backend to stop the
hipErrorInvalidImagecrash. That is right when the images are missing, but it also costs ~2x on attention-bound models where they are present. This change keeps the crash fix and gets the speed back, so it is meant to replace #466 rather than sit next to it.Getting the images
They ship in the architecture-family device package, e.g. for gfx1100/1101/1102/1103:
pip install amd-torch-device-gfx110x==<same version as torch>(when there is no dist-info/RECORD in an existing install, the package can also be unzipped and its
torch/lib/aotriton.imagestree copied in place.)Tests
tests/test_aotriton_sdpa_availability.py鈥?11 tests, no GPU required: image discovery (missing, incomplete, single set, family match, exact-arch precedence, uncovered arch, unknown arch) plus the resulting backend switches, the env override, and a non-ROCm build being left alone.