Skip to content

[Bugfix] Fix ImportError in get_chat_template for builtin chat-template names#4690

Open
waynehacking8 wants to merge 1 commit into
InternLM:mainfrom
waynehacking8:wayne/fix-4533-chat-template-import
Open

[Bugfix] Fix ImportError in get_chat_template for builtin chat-template names#4690
waynehacking8 wants to merge 1 commit into
InternLM:mainfrom
waynehacking8:wayne/fix-4533-chat-template-import

Conversation

@waynehacking8

Copy link
Copy Markdown
Contributor

Motivation

lmdeploy serve api_server ... --chat-template <name> and lmdeploy chat ... --chat-template <name> crash with ImportError whenever <name> is a builtin chat-template name (anything that is not a .json file path):

ImportError: cannot import name 'DEPRECATED_CHAT_TEMPLATE_NAMES' from 'lmdeploy.model'

Fixes #4533.

Root cause

get_chat_template (lmdeploy/cli/utils.py) imports DEPRECATED_CHAT_TEMPLATE_NAMES and REMOVED_CHAT_TEMPLATE_NAMES from lmdeploy.model. Those lists were removed in #4252 ([AsyncEngine Refactor 2/N] Remove deprecates from chat template), which cleaned up model.py, cli.py, async_engine.py and the tests but left the import — and its two guard blocks — dangling in cli/utils.py. The two names now exist nowhere in the repo, so the import raises before reaching the registry validation.

The crash only fires on the else branch (a builtin name that is not a file), which is why it went unnoticed: passing no --chat-template, or a .json file path, both return earlier. It is reachable from both lmdeploy/cli/serve.py (serve api_server) and lmdeploy/cli/chat.py (chat).

Modification

Complete the #4252 refactor in get_chat_template: drop the dead import and the removed-/deprecated-name guard blocks, keep the existing MODELS registry check, and remove the now-unused module-level logger. Unknown names are still rejected — by the registry assert, which lists the builtin templates.

BC-breaking (Optional)

No. Builtin names that previously crashed now work; unknown names still raise (an AssertionError from the registry check instead of an ImportError).

Use cases (Optional)

lmdeploy chat <model_path> --chat-template internlm2
lmdeploy serve api_server <model_path> --chat-template qwen

Checklist

  1. Added regression tests in tests/test_lmdeploy/test_utils.py (GPU-free, no model load):
    • test_get_chat_template_builtin_nameget_chat_template('llama2') returns a ChatTemplateConfig. Fails with ImportError on current main, passes with this fix.
    • test_get_chat_template_unregistered_name — an unregistered name raises AssertionError (registry check), not ImportError.
    • test_get_chat_template_noneNone in → None out.
  2. Verified locally: reverting only the cli/utils.py change makes test_get_chat_template_builtin_name fail at cli/utils.py:85: ImportError; with the fix all three pass. pre-commit (ruff-check / docformatter / copyright) is clean.

🤖 Generated with Claude Code

…te names (InternLM#4533)

`get_chat_template` (lmdeploy/cli/utils.py) imported
`DEPRECATED_CHAT_TEMPLATE_NAMES` and `REMOVED_CHAT_TEMPLATE_NAMES` from
`lmdeploy.model`, but those lists were removed in InternLM#4252 ([AsyncEngine Refactor
2/N] Remove deprecates from chat template). That refactor cleaned up model.py,
cli.py, async_engine.py and the tests, but left the import (and its two guard
blocks) dangling in cli/utils.py.

As a result, passing any builtin chat-template name that is not a file path
(e.g. `--chat-template internlm2`) to `lmdeploy serve api_server` or
`lmdeploy chat` crashed with:

    ImportError: cannot import name 'DEPRECATED_CHAT_TEMPLATE_NAMES' from 'lmdeploy.model'

before reaching the registry validation. Omitting `--chat-template`, or passing
a `.json` file path, returns early, which is why it went unnoticed.

Complete the refactor: drop the dead import and the removed-/deprecated-name
guards, keep the `MODELS` registry check, and remove the now-unused module
logger. Add regression tests in tests/test_lmdeploy/test_utils.py covering a
builtin name, an unregistered name, and the None case.

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] lmdeploy serve api_server 方式部署指定 chat-template 报错

1 participant