Skip to content

Allow aliases to inject request parameters #12548

Open

Description

@treilhes

Allow aliases to inject request parameters

Feature request

I would like LocalAI aliases to support request parameter overrides/injection, so that different aliases can point to the same underlying loaded model while transparently modifying specific request parameters.

Use case

I want to expose the same Qwen model through two model names:

  • qwen-fast → thinking disabled
  • qwen-thinking → thinking enabled

Both aliases should resolve to the same underlying model instance / loaded weights. The only difference should be the parameters injected into the request.

For example:

name: qwen-fast
alias: qwen3
parameters:
  reasoning_effort: none

and:

name: qwen-thinking
alias: qwen3
parameters:
  reasoning_effort: high

Then a client could simply send:

{
  "model": "qwen-fast",
  "messages": [...]
}

and LocalAI would internally transform the request into something equivalent to:

{
  "model": "qwen3",
  "reasoning_effort": "none",
  "messages": [...]
}

Likewise:

{
  "model": "qwen-thinking",
  "messages": [...]
}

would internally become:

{
  "model": "qwen3",
  "reasoning_effort": "high",
  "messages": [...]
}

Why this is useful

This would allow users to expose multiple logical variants of a model without loading the model weights multiple times.

For reasoning models such as Qwen3, this is particularly useful for exposing:

qwen-fast      → same model, thinking disabled
qwen-thinking  → same model, thinking enabled

while keeping only one model/backend instance loaded in memory/VRAM.

It would also be useful for other parameters, for example:

  • reasoning_effort
  • enable_thinking
  • chat_template_kwargs
  • sampling defaults
  • backend-specific request parameters
  • other OpenAI-compatible request parameters

Possible configuration

One possible syntax could be:

name: qwen-fast
alias: qwen3

request_defaults:
  reasoning_effort: none

and:

name: qwen-thinking
alias: qwen3

request_defaults:
  reasoning_effort: high

Alternatively, parameters or another appropriate name could be used.

Ideally, the injected parameters would behave as defaults, meaning that an explicitly supplied request parameter could optionally override the alias value.

For example:

alias default
      ↓
reasoning_effort = none
      ↓
explicit request says "high"
      ↓
request value wins

Important requirement

The alias should remain a logical view of the same underlying model, rather than causing LocalAI to instantiate/load the model again.

The desired architecture is:

                 ┌── qwen-fast ──────┐
                 │                    │
one loaded Qwen ─┤                    ├── same backend/model instance
                 │                    │
                 └── qwen-thinking ──┘

This would provide a clean way to expose multiple specialized variants of a single loaded model without duplicating VRAM usage.

Expected result

With this feature, applications that only support selecting a model by name could use:

qwen-fast

or:

qwen-thinking

without needing to know about LocalAI-specific metadata, enable_thinking, or reasoning_effort request parameters.

This would make aliases significantly more useful as logical model profiles, while preserving the memory efficiency of a single loaded model.

Activity

  1. localai-org-maint-bot commented on Oct 7, 2026

    @localai-org-maint-bot
    Collaborator

    Pickup withdrawn for this run: the maintainer bot has 42 open PRs awaiting review across its repositories, exceeding its 35-PR backpressure limit. The initial check counted only LocalAI PRs. No PR was opened for this issue, and it remains available for another contributor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions