Skip to content

test: cover Gemma4 channel-aware close marker decoding - #15682

Open
christian-byrne wants to merge 2 commits into
masterfrom
fix/ltx2-regression-tests
Open

test: cover Gemma4 channel-aware close marker decoding#15682
christian-byrne wants to merge 2 commits into
masterfrom
fix/ltx2-regression-tests

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Adds decode-side coverage for the channel-marker handling fixed in #15611, which merged without tests.

Gemma4SDTokenizer.decode translates channel markers into <think>/</think>. Before #15611 the close was translated unconditionally, so a <channel|> that closed a primed channel became a </think> with no opener. TextGenerateLTX2Prompt primes <|channel>final\n in non-thinking mode, and since only new tokens are decoded the opener is never present — the node's parse then read the entire answer as reasoning and returned an empty string, which fed empty conditioning to CLIPTextEncode.

gemma4_template_test.py already covers the tokenize side; this is the matching decode side. The load-bearing case is test_close_of_a_non_thought_channel_is_not_reasoning.

Tests only — no source changes.

Verified against master: 22 passed with gemma4_template_test.py. Reverting decode to its pre-#15611 form fails 3 of the 6, so they hold the fix in place rather than restating it.

The LTX2 prompt node primes a 'final' channel, so only its close is generated. Decoding that close as </think> made the whole answer read as reasoning and the node returned an empty string.
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6f358ea5-2d90-4f17-83cd-ebbc4c787d9c

📥 Commits

Reviewing files that changed from the base of the PR and between 0696f61 and 7b577ea.

📒 Files selected for processing (1)
  • tests-unit/comfy_test/gemma4_decode_test.py

Included review availability: 0 reviews are currently available. Based on recent review activity, included reviews refill at 4 per hour.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (9)
  • GitHub Check: Run Pylint
  • GitHub Check: test (macos-latest)
  • GitHub Check: test (macos-latest)
  • GitHub Check: test (windows-2022)
  • GitHub Check: test (ubuntu-latest)
  • GitHub Check: test
  • GitHub Check: test (ubuntu-latest)
  • GitHub Check: test (windows-latest)
  • GitHub Check: Run Pylint
🧰 Additional context used
📓 Path-based instructions (5)
**/*

📄 CodeRabbit inference engine (AGENTS.md)

**/*: Keep changes small, direct, and limited to the narrowest necessary code path and smallest number of files.
Prefer practical fixes, minimal dependencies, and existing repository patterns; remove obsolete, dead, unreachable, or unused code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is explicitly intended.
Core ComfyUI must not add outbound internet requests, telemetry, tracking, reporting, remote configuration, or background network activity. User-authorized model downloads are limited to the requested artifact and must exclude telemetry and unrelated metadata.

Files:

  • tests-unit/comfy_test/gemma4_decode_test.py
**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Keep state and capability flags on the object that owns the behavior. Prefer explicit parent-owned attributes over probing child objects with getattr; use child checks only when the child owns the delegated behavior.
Preserve shared method signatures, argument order, return shapes, side effects, and error behavior unless every affected caller and interface is intentionally updated.
Do not add unused compatibility parameters, flags, attributes, constructor options, or model-specific options to shared helpers; keep one-off behavior at the integration boundary.
Normalize third-party return conventions at integration boundaries so core code receives the expected type and shape; avoid undocumented caller-side unwrapping.
Do not add torch.no_grad, torch.inference_mode, or inference-mode wrappers. Do not add model freeze/unfreeze toggles; only disable globally enabled inference mode when a training path requires gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; use nn.Identity when deleting a module would alter keys or ordering.
Keep imports at module scope except established optional-backend probes or imports required to avoid cycles; avoid unnecessary try/except blocks and use specific exceptions with useful fallbacks.
Do not add workarounds for unsupported library versions, especially PyTorch exception-and-float-cast retries, unless a comment names the exact versions still requiring them.
Let unsupported model formats, invalid quantization metadata, and bad states fail with clear errors instead of silently degrading output.
Match local style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM use, and offloading as correctness concerns across CPU, CUDA, ROCm, MPS, DirectML, XPU, NPU, and low-VRAM environments.
Prefer existing ComfyUI and Comfy Kitchen operations, quantization helpers, cast/offload helpe...

Files:

  • tests-unit/comfy_test/gemma4_decode_test.py
**/*.{py,json}

📄 CodeRabbit inference engine (AGENTS.md)

Treat legacy combo, io.Combo, and io.DynamicCombo values affecting filesystem access as untrusted; revalidate them at load/save boundaries with folder_paths, containment checks, or fixed allowlists.

Files:

  • tests-unit/comfy_test/gemma4_decode_test.py
**/*.{py,md,txt,json}

📄 CodeRabbit inference engine (AGENTS.md)

Keep warning and info messages short and actionable, remove noisy or misleading logging, and make documentation edits concise, factual, and tied to changed behavior.

Files:

  • tests-unit/comfy_test/gemma4_decode_test.py
**

⚙️ CodeRabbit configuration file

**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing a with: block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.

Files:

  • tests-unit/comfy_test/gemma4_decode_test.py
🧠 Learnings (1)
📚 Learning: 2026-02-21T14:01:41.482Z
Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.

Applied to files:

  • tests-unit/comfy_test/gemma4_decode_test.py
🔇 Additional comments (1)
tests-unit/comfy_test/gemma4_decode_test.py (1)

3-10: LGTM!

Also applies to: 13-25, 28-52


📝 Walkthrough

Walkthrough

Added a standalone Gemma4 decode test harness with CPU fallback and a canned tokenizer. Added regression tests for thought-channel conversion, empty and unclosed thought channels, non-thought channel closures, multiple channels, and removal of turn and EOS markers.

Merge Risk: ⚪ Minimal · up to 7b577

This is a localized tests-only change with no production behavior changes; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the Gemma4 decoding tests for channel-aware close marker handling.
Description check ✅ Passed The description directly explains the Gemma4 decode tests, the regression case, and the testing scope.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants