From 2017f2496b41b4e5bca8fe59494a823248188a6f Mon Sep 17 00:00:00 2001
From: zq <zhouquan1511@163.com>
Date: Wed, 5 Aug 2026 01:32:54 +0800
Subject: [PATCH] docs: explain prompt hidden states in pipeline

Signed-off-by: zq <zhouquan1511@163.com>
---
 docs/en/llm/pipeline.md    | 24 ++++++++++++++++++++++++
 docs/zh_cn/llm/pipeline.md | 24 ++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/docs/en/llm/pipeline.md b/docs/en/llm/pipeline.md
index 506878ced1..3aa6b1bd26 100644
--- a/docs/en/llm/pipeline.md
+++ b/docs/en/llm/pipeline.md
@@ -144,6 +144,30 @@ response = pipe(['Hi, pls intro yourself', 'Shanghai is'],
 hidden_states = [x.last_hidden_state for x in response]
 ```
 
+### Get last layer's hidden states for prompt and generated tokens
+
+Set `output_last_hidden_state='all'` to return the last layer's hidden
+states for both the prefill prompt tokens and generated tokens.
+
+```python
+from lmdeploy import pipeline, GenerationConfig
+
+pipe = pipeline('internlm/internlm2_5-7b-chat')
+
+gen_config=GenerationConfig(output_last_hidden_state='all',
+                            max_new_tokens=10)
+response = pipe(['Hi, pls intro yourself', 'Shanghai is'],
+                gen_config=gen_config)
+hidden_states = [x.last_hidden_state for x in response]
+```
+
+```{note}
+`output_last_hidden_state` is supported by the TurboMind engine. Use
+`'generation'` when you only need generated-token hidden states, and use
+`'all'` when you also need the prompt/prefill hidden states. Prefix caching
+cannot be enabled together with `output_last_hidden_state='all'`.
+```
+
 ### Calculate ppl
 
 ```python
diff --git a/docs/zh_cn/llm/pipeline.md b/docs/zh_cn/llm/pipeline.md
index ab6ae50417..e736f98e3a 100644
--- a/docs/zh_cn/llm/pipeline.md
+++ b/docs/zh_cn/llm/pipeline.md
@@ -144,6 +144,30 @@ response = pipe(['Hi, pls intro yourself', 'Shanghai is'],
 hidden_states = [x.last_hidden_state for x in response]
 ```
 
+### 获取 prompt 和生成 token 最后一层的 hidden_states
+
+设置 `output_last_hidden_state='all'` 可以同时返回 prefill 阶段的 prompt
+token 和生成 token 的最后一层 hidden states。
+
+```python
+from lmdeploy import pipeline, GenerationConfig
+
+pipe = pipeline('internlm/internlm2_5-7b-chat')
+
+gen_config=GenerationConfig(output_last_hidden_state='all',
+                            max_new_tokens=10)
+response = pipe(['Hi, pls intro yourself', 'Shanghai is'],
+                gen_config=gen_config)
+hidden_states = [x.last_hidden_state for x in response]
+```
+
+```{note}
+`output_last_hidden_state` 由 TurboMind 引擎支持。如果只需要生成 token
+的 hidden states，使用 `'generation'`；如果还需要 prompt/prefill 的 hidden
+states，使用 `'all'`。`output_last_hidden_state='all'` 不能和 prefix caching
+同时开启。
+```
+
 ### 计算 ppl
 
 ```python