Render chat markdown in the ds4 CLI - #696
Open
kk1987 wants to merge 5 commits into
Open
Conversation
Model replies are markdown, but the CLI printed them as raw text: bold markers, backticks, and fences all reached the terminal verbatim. Add ds4_render.c, a streaming renderer that turns the common inline constructs into ANSI attributes and highlights fenced code blocks, and route every CLI output path through it. The inline state machine, the UTF-8 accumulator that keeps escapes out of the middle of a codepoint, and the fenced-block highlighter follow the agent renderer in ds4_agent.c. Code lines are buffered until their newline instead of being streamed and repainted, so the terminal never sees a carriage return and fence content stays byte exact apart from color. A fence without an info string, or one tagged text/plain/ascii, is not highlighted at all: those carry diagrams and command output. token_printer keeps its call-site contract and only delegates now. Rendering happens when stdout is a terminal; redirected output stays a byte for byte copy of the model text minus the <think> markers, which keep their existing greying and cross-token marker buffering. --plain (alias --no-markdown) forces the raw form on a terminal too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy
The inline pass alone still leaves the structural markers on screen: a reply that opens with "## Summary" printed the hashes, and every bullet kept its "- ". Classify the start of each logical line before the inline machine sees the bytes. Headings drop their markers and turn bold, with an underline for the top two levels; bullet markers become a dim mid dot; ordered markers stay but dim; blockquotes get one dim bar per level; and a lone marker run becomes a horizontal rule as wide as the terminal, capped at 80 columns. The classifier holds back only the bytes it cannot decide on yet, which is at most the indentation plus a marker run, and replays them through the ordinary path as soon as the line turns out to be plain prose. So "**bold** at the start of a line" and "- - -" still reach the reader as what they are. Code fences bypass the whole layer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy
A pipe table is the one construct that is unreadable as source: the columns only line up once every cell is known, and CJK text makes the byte length useless for that. Buffer consecutive table rows, then lay them out with box drawing characters once the block ends. Column widths come from an embedded wcwidth that is locale independent, so wide and fullwidth ranges count as two columns and combining marks as zero. Cells keep their inline markdown, the header stays bold, and the alignment row decides how each column is padded. Wide tables first lose their cell padding, then have their widest columns cut back with an ellipsis. Anything that is not a table, or does not fit even at minimum width, is printed exactly as it arrived: a missing alignment row, a degenerate row, or a block past the 64 KiB / 512 line buffer cap all fall back to the source text. Interrupting a reply mid-table flushes what arrived. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy
Thinking is where most of the reading time goes on a slow local model, and it was the one part of the reply left as raw markdown under a flat grey wash. Give it the same structure as the answer without letting it compete for attention. Run <think> content through the same pipeline as the answer, then map every style onto grey: base 90, bold 1;90, italic 3;90, inline code 2;90, headings 1;90 plus underline for the top two levels, and 2;90 for bullets, quote bars, rules, and table frames. No hue is ever emitted inside a think block, and fenced code is shown without syntax colors, so a highlighted block always means the answer. Both boundaries flush the renderer state before the other side starts: an in-progress table is laid out (or printed raw when it never had an alignment row), an open fence is closed, held-back markers are printed, and every attribute is reset. A "**" left dangling in the reasoning can no longer bold the answer, and an open answer attribute cannot darken into the reasoning. Reply end and interrupts take the same path, which is now shared with ds4r_finish(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy
Rendering everything is right for most replies, but a table makes the renderer buffer rows until the block ends, which shows up as a pause on a slow local model. Offer the middle ground instead of an all-or- nothing switch. --markdown takes off, basic, or full and defaults to full. basic keeps the inline pass, the line-start constructs, and fenced code, but lets table rows stream through as ordinary text, so nothing is ever held back; off prints raw model text. --plain stays as the intuitive alias for --markdown off, and --think-plain drops thinking back to flat grey while the answer still renders. Unknown modes are rejected the way an unknown backend is, listing the valid values. The mode lives in cli_config, so /markdown switches it for the next turns and prints the current mode when called without an argument. The renderer grew two matching knobs, ds4r_set_tables() and ds4r_set_think_markdown(). Redirected output stays raw in every mode. --no-markdown is gone: it was a second spelling of --plain that never shipped in a release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The interactive
ds4chat prints model output raw, but DeepSeek V4 Flash answers arrive full of markdown — tables, bold, fenced code. This PR renders that markdown in the terminal, in the style of modern coding-agent TUIs, while keeping every non-interactive path byte-identical.What it does
inline code;a * bstays literal (space-guarded, same rule as the ds4-agent renderer).text/ascii-style fences stay uncolored so ASCII diagrams pass through verbatim.:---:), cell contents rendered, and widths computed with an embedded locale-independent wcwidth so CJK/emoji cells align correctly. Tables are buffered until complete and degrade gracefully: padding shrink, then…truncation, then raw passthrough (also on missing separator row, buffer cap, or interruption).<think>) content is rendered through the same pipeline but mapped to a muted grey-only palette (no hue colors, no syntax highlighting) so reasoning stays visually de-emphasized while long thinking streams remain readable.Modes and compatibility
--markdown off|basic|full(defaultfull).basicdisables only table boxing — the one construct that buffers output — everything else still renders.--plainis an alias foroff;--think-plainkeeps thinking as flat grey regardless of mode./markdownswitches the mode at runtime in the REPL.ds4_agent.cis untouched (git diffagainst main is empty for it).Implementation
New self-contained
ds4_render.c/.h(linked into theds4binary only). The inline/fence/syntax/UTF-8 machinery is ported from the static renderer insideds4_agent.c; tables, headings/lists, wcwidth, the muted thinking palette, and the mode knobs are new. Porting rather than extracting keeps this diff zero-risk for the working agent binary; unifyingds4_agent.conto the shared TU would be a natural follow-up PR if you want it. Tables are buffered-then-drawn instead of repainted in place because cursor-up repaint is unreliable in cooked mode with autowrap across wrapped lines and scrollback.Tests
tests/ds4_render_test.c(wired intomake test, no model weights needed): every corpus is fed whole-string, byte-at-a-time, and in 1..7-byte chunks with identical output required; markers and UTF-8 sequences split across token boundaries; ANSI never lands mid-codepoint; CJK table alignment asserted by per-row visible width; think/answer boundary state isolation (an unclosed**in thinking cannot leak bold into the answer); non-TTY passthrough identity for the full corpus. Metal and cpu builds are warning-free for the touched files.🤖 Generated with Claude Code
https://claude.ai/code/session_01VwY5tNVR2wBxBBQrxRy4cy