Skip to content

feat(gallery): add Qwen3.8 iMatrix NVFP4 variant - #12769

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
gallery/20261019-refresh
Open

localai-org-maint-bot wants to merge 1 commit into
masterfrom
gallery/20261019-refresh

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Add cdiamond's Qwen3.8-27B iMatrix NVFP4 GGUF as a variant of qwen3.8-27b-q4. It uses llama.cpp, includes the F16 vision projector, and enables the embedded MTP head with one draft token. The default context is 32,768 tokens. The publisher's repository had about 3.16 million Hugging Face downloads when checked.

Also fix qwen3.8-27b-nvfp4-mtp: its mmproj pointed to a Q8_0 filename, but its files list downloads the BF16 projector. The configuration now uses the downloaded file.

Sources: publisher's model card, original model. Both new downloads are pinned to ac343e8f44caef0896f79d372ecc07ef7ab34ec8; each SHA256 matches both HF API LFS metadata and the pinned URL's x-linked-etag. Checked current master and open gallery PR diffs for duplicates.

Validation:

  • YAML parsing, unique names, variant references and ownership passed across 2,209 entries.
  • Model and projector file references passed for both affected entries.
  • git diff --check passed.
  • go test ./core/gallery/... could not start: proxy.golang.org returned HTTP 403 for the LocalAGI module.
  • No inference smoke test was run.

Documentation updated in docs/content/features/model-gallery.md.

Human DCO sign-off remains required. The commit uses Assisted-by; repository policy prohibits AI-added Signed-off-by trailers.

Fix the existing NVFP4 variant to use its downloaded BF16 projector.

Assisted-by: Codex:GPT-6 [shell]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant