[Docs] Explain prompt hidden states in pipeline

[zcxGGmu]

Motivation

The pipeline docs only showed how to fetch last-layer hidden states for generated tokens with output_last_hidden_state='generation'. Issue #3456 asks how to obtain the prompt/prefill hidden states.

Refs #3456

Modification

• Add English and Chinese pipeline examples for GenerationConfig(output_last_hidden_state='all').
• Clarify that generation returns generated-token hidden states, while all also includes prompt/prefill hidden states.
• Note the current constraints: this is supported by TurboMind, and prefix caching cannot be enabled with output_last_hidden_state='all'.

BC-breaking (Optional)

No. Documentation only.

Use cases (Optional)

Users who need prompt/prefill hidden states can use:

gen_config = GenerationConfig(output_last_hidden_state='all', max_new_tokens=10)
response = pipe(['Hi, pls intro yourself'], gen_config=gen_config)
hidden_states = [x.last_hidden_state for x in response]

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
   • PYTHONPATH= python -m pre_commit run --files docs/en/llm/pipeline.md docs/zh_cn/llm/pipeline.md
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
   • Documentation-only change; verified by grep for the new output_last_hidden_state='all' examples.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
   • N/A
4. The documentation has been modified accordingly, like docstring or example tutorials.
   • Updated docs/en/llm/pipeline.md and docs/zh_cn/llm/pipeline.md.