Skip to content

[Fix] Reject LoRA adapters on non-PyTorch backend - #4819

Open
zcxGGmu wants to merge 1 commit into
InternLM:mainfrom
zcxGGmu:lora-backend-check
Open

[Fix] Reject LoRA adapters on non-PyTorch backend#4819
zcxGGmu wants to merge 1 commit into
InternLM:mainfrom
zcxGGmu:lora-backend-check

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 4, 2026

Copy link
Copy Markdown

Motivation

When lmdeploy serve api_server resolves to the TurboMind backend, --adapters is currently accepted but not applied because LoRA adapters are PyTorch-backend only. This can make the server start successfully while adapter model names are unavailable, which matches the confusion reported in #3594.

Refs #3594

Modification

  • Add an early CLI validation for serve api_server: if adapters are provided after backend resolution and the selected backend is not pytorch, raise a clear ValueError asking the user to set --backend pytorch.
  • Add CLI unit coverage for both rejecting TurboMind and allowing PyTorch.

BC-breaking (Optional)

No supported behavior is removed. Invalid --adapters + non-PyTorch backend combinations now fail early with an explicit message instead of being silently ignored.

Use cases (Optional)

Users serving LoRA adapters should launch with --backend pytorch, for example:

lmdeploy serve api_server /path/to/model --backend pytorch --adapters mylora=/path/to/lora

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
    • python -m pre_commit run --files lmdeploy/cli/serve.py tests/test_lmdeploy/cli/test_serve.py
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
    • python -m pytest tests/test_lmdeploy/cli/test_serve.py tests/test_lmdeploy/serve/test_generation_config.py -q
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
    • N/A
  4. The documentation has been modified accordingly, like docstring or example tutorials.
    • N/A; this is a CLI validation/error-message fix.

Signed-off-by: zq <zhouquan1511@163.com>
@@ -0,0 +1,30 @@
# Copyright (c) OpenMMLab. All rights reserved.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These UT additions don't add much value. Please remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants