Skip to content

feat(gallery): add Ternary Bonsai 2 9B variants - #12771

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
gallery/20261019-second-pass
Open

localai-org-maint-bot wants to merge 1 commit into
masterfrom
gallery/20261019-second-pass

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

Add Ternary Bonsai 2 9B in PQ2_0, PTQ1_0, and Q2_0 packings, grouped as variants. Each entry uses the existing Bonsai backend, includes the Q8_0 vision projector, and sets the publisher's thinking-mode sampling defaults. Add installation and variant-selection documentation.

The model is absent from master and appears in Hugging Face's trending GGUF list (77,791 downloads at inspection).
Source: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf

Notes for Reviewers

  • Pin all four unique assets to revision 5b2abb02362efbed46bbe204014b02837778b2c2.
  • Double-check each SHA256 against both HF LFS metadata and the pinned download's x-linked-etag header.
  • Use Bonsai because PTQ1_0 and PQ2_0 require the PrismML fork. All three packings contain the same ternary weights.
  • Use the Q8_0 projector recommended by the publisher.
  • git diff --check passes. Standalone Go/yaml.v3 validation passes: unchanged existing entries, unique names, valid variant references, and matching model/projector file paths.
  • Attempted go test ./core/gallery -count=1; dependency resolution failed with HTTP 403 for github.com/mudler/LocalAGI@v0.0.0-20261016084905-7aa0c8fea076 from proxy.golang.org. The suite did not run.
  • Inference was not tested locally.

Signed commits

  • Yes, I signed my commits. Human DCO sign-off is required; AI attribution uses Assisted-by.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

Add three GGUF packings with the Q8_0 vision projector and document
installation. Pin asset revisions and verify hashes against HF metadata
and download headers.

Assisted-by: Codex:gpt-6

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant