docs(copilot): reattach an orphaned docstring and cut comment noise

One real defect. Adding hasSpecialTagMarker put it BETWEEN
unclosedTagCannotResolve and the docstring written for it, so two doc blocks sat
stacked and the function they described ended up with none. A reader scanning the
file would have read the first block as preamble for the wrong function. Moved
back, and while there: "14 substring scans" is SPECIAL_TAG_NAMES.length * 2, so
adding an eighth tag would have quietly made it wrong — it now says a pass per
tag name.

The rest is noise removal:

- Three comments narrating what the parser used to do. A comment is read by
  someone looking at the current code, not the diff, and this branch already
  writes that history at length in its commit messages.
- Three millisecond measurements. Each one already stated the durable claim —
  quadratic, or a copy thrown away per opener per chunk — and then appended a
  number that will rot. The one measurement kept is the blind-spot paragraph on
  MAX_UNCLOSED_BODY_SCAN, where the number is what justifies the constant.
- Two of the four restatements of "the renderer concatenates adjacent text
  segments". Kept where it is load-bearing, on pushText and on the test helper.
- One claim gone stale in the last commit: the read-budget doc still described
  two helpers agreeing with each other, after they became one function.

Left alone deliberately: the rules that look arbitrary and are not — marker
blanking, the differing resume offsets, the accepted trades. Those are why this
file reads as heavily commented, and they are the ones worth having.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: j15z

apps/sim/app/workspace/[workspaceId]/home/components/message-content/components/chat-content/chat-sanitize.ts

@@ -19,8 +19,7 @@ const HIDDEN_INLINE_REFERENCE_PATTERN =
  * - **No `<workspace_resource>` between opener and closer**, via the negative
  *   lookahead. This is a cost bound, not a correctness rule: a lazy scan that may
  *   cross an opener restarts from every opener, so a message repeating the tag
- *   name is quadratic. Measured at 168KB of such prose: 154ms per call without
- *   it, 0.34ms with — and this runs on the main thread for every streamed chunk.
+ *   name is quadratic — on the main thread, for every streamed chunk.
  *
  * Backticks must be DIRECTLY adjacent. Allowing whitespace between let the
  * pattern reach past the tag and take the delimiter off a neighbouring code

apps/sim/app/workspace/[workspaceId]/home/components/message-content/components/special-tags/special-tags.test.ts

@@ -223,8 +223,6 @@ describe('parseSpecialTags with <question>', () => {
   })
 
   it('still parses a valid tag that follows a rejected one', () => {
-    // Before the rewrite, rejecting an unclosed tag abandoned the rest of the
-    // message, so this <options> tag was never parsed at all.
     const { segments } = parseSpecialTags(
       'I use <thinking> loosely here. Anyway: <options>[{"title":"A","description":"d"}]</options> done.',
       false
@@ -437,8 +435,6 @@ describe('parseSpecialTags with <question>', () => {
   })
 
   it('shows prose immediately mid-stream instead of blanking the rest', () => {
-    // The failure this replaces: everything after the marker stayed invisible
-    // for the remainder of the stream, then reappeared when it ended.
     const content = 'The `<workspace_resource>` chip only renders for a real file.'
     const { segments, hasPendingTag } = parseSpecialTags(content, true)
     expect(hasPendingTag).toBe(false)
@@ -627,9 +623,6 @@ describe('parseSpecialTags with <question>', () => {
       'The `<workspace_resource>` file chip only renders when its path points to a real file.'
     const { segments, hasPendingTag } = parseSpecialTags(content, false)
     expect(hasPendingTag).toBe(false)
-    // Asserted on the joined text, not segment boundaries: the renderer
-    // concatenates adjacent text segments, so how the span is split is not
-    // observable to a reader.
     expect(segments.every((segment) => segment.type === 'text')).toBe(true)
     expect(renderedText(segments)).toBe(content)
   })

apps/sim/app/workspace/[workspaceId]/home/components/message-content/components/special-tags/special-tags.tsx

@@ -516,10 +516,9 @@ const JSON_BODY_TAG_NAMES: ReadonlySet<(typeof SPECIAL_TAG_NAMES)[number]> = new
  * first character that breaks JSON viability — so a bounded window reaches the
  * same verdict as the full remainder for any payload a tag actually carries.
  * Unbounded, the check is O(body length) and runs once per opener inside a parse
- * that re-runs for every streamed chunk: a long reply repeatedly mentioning a tag
- * name cost seconds of main-thread time, and one 58KB reply whose early close was
- * misspelled — so a single body stretched most of the message — cost 242ms per
- * parse against 35ms bounded.
+ * that re-runs for every streamed chunk. A long reply repeatedly mentioning a
+ * tag name, or one whose misspelled early close stretches a single body across
+ * most of the message, is then quadratic in the length of the reply.
  *
  * The window's one blind spot, and why it is accepted: a JSON body whose
  * top-level value closes BEYOND the window, followed by prose and no closing tag,
@@ -618,14 +617,24 @@ function isViableJsonPrefixOf(scannable: string): boolean {
   return true
 }
 
+/**
+ * Whether `text` contains a marker for one of the tags this parser knows.
+ *
+ * Deliberately the tag NAMES rather than anything tag-shaped. A prose body may
+ * legitimately contain `<div>` or `Promise<void>`; only a marker the parser
+ * would itself act on proves the enclosing opener was text. Shared so the
+ * streaming and matched-pair paths cannot answer the same question differently
+ * — them disagreeing is what let a late close swallow content already on screen.
+ */
+function hasSpecialTagMarker(text: string): boolean {
+  return SPECIAL_TAG_NAMES.some((name) => text.includes(`</${name}>`) || text.includes(`<${name}>`))
+}
+
 /**
  * True when an opening tag with no close yet can NEVER resolve, so the text
  * after it should be shown immediately instead of held back until the stream
- * ends.
- *
- * Without this, a message that merely mentions a tag in prose goes blank from
- * that point on for the rest of the stream — the text is only restored once
- * streaming stops.
+ * ends. Without it, a message that merely mentions a tag in prose goes blank
+ * from that point on until streaming stops.
  *
  * One rule decides it, chosen by body kind:
  *
@@ -635,32 +644,18 @@ function isViableJsonPrefixOf(scannable: string): boolean {
  *   prose (no `{` at all), a misspelled close like `</workflow_resource>`, a
  *   truncated `</workspac`, or no close whatsoever.
  * - **The prose-bodied tag** has no JSON to test, so the only evidence available
- *   is that tags never nest: another special-tag marker in the body — opening or
- *   closing, foreign or its own name — means this opener was literal text.
+ *   is that tags never nest: a marker for another special tag in the body means
+ *   this opener was literal text.
  *
- * Scanning for nested markers on a JSON body too was redundant: any marker
- * outside a string literal is itself content the JSON rule already rejects, and
- * markers inside one are legitimate quoted syntax that must NOT count as
- * evidence. It cost 14 substring scans per opener per streamed chunk to catch
- * nothing the viability rule misses.
+ * Nested markers are NOT scanned for on a JSON body. A marker outside a string
+ * literal is content the viability rule already rejects, and one inside is
+ * legitimate quoted syntax that must not count as evidence — so the scan cost a
+ * pass per tag name, open and close, to catch nothing.
  *
- * Both are conservative: they only fire on content that could not have parsed.
- * A false positive would merely show text early that a later chunk resolves
- * into a tag — the end-of-stream parse still produces the correct final render.
+ * Both rules are conservative: they fire only on content that could not have
+ * parsed. A false positive merely shows text early that a later chunk resolves
+ * into a tag, and the end-of-stream parse still renders correctly.
  */
-/**
- * Whether `text` contains a marker for one of the tags this parser knows.
- *
- * Deliberately the tag NAMES rather than anything tag-shaped. A prose body may
- * legitimately contain `<div>` or `Promise<void>`; only a marker the parser
- * would itself act on proves the enclosing opener was text. Shared so the
- * streaming and matched-pair paths cannot answer the same question differently
- * — them disagreeing is what let a late close swallow content already on screen.
- */
-function hasSpecialTagMarker(text: string): boolean {
-  return SPECIAL_TAG_NAMES.some((name) => text.includes(`</${name}>`) || text.includes(`<${name}>`))
-}
-
 function unclosedTagCannotResolve(
   tagName: (typeof SPECIAL_TAG_NAMES)[number],
   body: string
@@ -672,8 +667,7 @@ function unclosedTagCannotResolve(
   // Cheap rejection before the expensive one. isViableJsonPrefixOf decides on
   // the first non-whitespace character when it is not `{` or `[` — which is the
   // common case, a tag name mentioned in prose — so testing it here avoids
-  // blanking up to a full window of text only to throw the copy away. Measured
-  // at 86KB of such prose: 43ms per streaming parse before, 2.4ms after.
+  // blanking up to a full window of text only to throw the copy away.
   const firstChar = pending.trimStart().charAt(0)
   if (firstChar !== '' && firstChar !== '{' && firstChar !== '[') return true
 
@@ -804,9 +798,8 @@ export function memoizedIndexOf(
  * How much of a body may be inspected, and whether that is all of it.
  *
  * The read budget, isolated from what the body turns out to BE. Both the
- * unclosed and matched-pair paths spend it, and getting them to agree is the
- * point: they previously applied the same constant in two shapes, and the
- * difference between them was a bug.
+ * unclosed and matched-pair paths spend it through this one function, so they
+ * cannot drift out of agreement about how much of a body may be read.
  */
 interface InspectedBody {
   /** The prefix actually examined. */
@@ -988,9 +981,6 @@ function resolveTagAt(
  * tags are suppressed and flagged via `hasPendingTag` so the caller can show a
  * loading indicator, and a trailing partial opening marker (`<opt`, `<usage_`)
  * is stripped during streaming so it never flashes as raw markup.
- *
- * Adjacent text segments are concatenated by the renderer, so emitting a span
- * as several pieces is display-neutral.
  */
 export function parseSpecialTags(content: string, isStreaming: boolean): ParsedSpecialContent {
   const segments: ContentSegment[] = []