Skip to content

feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235

Open
blackwell-systems wants to merge 1 commit into
ChromeDevTools:mainfrom
blackwell-systems:experimental-gcf-format
Open

feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235
blackwell-systems wants to merge 1 commit into
ChromeDevTools:mainfrom
blackwell-systems:experimental-gcf-format

Conversation

@blackwell-systems

Copy link
Copy Markdown

Summary

Add --experimentalGcfFormat as a hidden experimental flag alongside the existing --experimentalToonFormat. When enabled, tool responses encode structured data using GCF (Graph Compact Format) instead of formatted JSON. No changes to existing behavior or default output.

Why

Benchmarked on Chrome DevTools data shapes

Data shapes verified from source code and test snapshots (ConsoleMessageConcise, NetworkRequestConcise, HeapSnapshotFormatter.toJSON(), SnapshotFormatter.toJSON()):

Dataset JSON TOON GCF TOON vs JSON GCF vs JSON GCF vs TOON
Console (100) 2,534 1,716 1,663 32.3% 34.4% +3.1%
Console (500) 12,879 8,761 8,475 32.0% 34.2% +3.3%
Network (100) 3,287 1,776 1,682 46.0% 48.8% +5.3%
Network (500) 16,547 8,936 8,442 46.0% 49.0% +5.5%
Heap (100) 2,210 797 753 63.9% 65.9% +5.5%
DOM (3x3) 282 295 244 -4.6% 13.5% +17.3%
DOM (4x3) 396 464 407 -17.2% -2.8% +12.3%

TOON increases token count on DOM snapshots (-4.6% to -17.2% vs JSON). GCF saves tokens on every data type.

Console message corruption

TOON's decoder crashes on console messages containing bracket-colon patterns, which are standard browser output:

[Error]: net::ERR_CONNECTION_REFUSED
[React DevTools]: Component rendered 3 times
[Violation]: Forced reflow while executing JavaScript took 42ms
[Performance]: Long task detected (duration: 234ms)

10 of 20 console messages fail TOON round-trip. GCF: zero failures. While Chrome DevTools MCP encodes only (no decode), the structurally ambiguous output can affect downstream consumers.

LLM comprehension

GCF scores 100% on general structured data and 90.7% on adversarial payloads across GPT-4o, GPT-5.5, Claude, and Gemini. JSON scores 53.6%. TOON scores 68.5%. (1,700+ evaluations.)

Full eval data: GCF benchmarks

Data integrity

GCF verified lossless across 43 billion+ round-trips in 5 formats and 6 languages. Zero failures. Zero runtime dependencies.

Implementation

Mirrors the existing experimentalToonFormat pattern exactly:

  • Hidden boolean CLI flag, defaults to false
  • useGcf parameter threaded through handle()format()
  • compactEncode helper selects GCF, TOON, or null (formatted JSON)
  • GCF takes precedence if both flags are set

Changes

File Change
src/bin/chrome-devtools-mcp-cli-options.ts Add experimentalGcfFormat flag
src/third_party/index.ts Export gcfEncode from @blackwell-systems/gcf
src/McpResponse.ts Add useGcf parameter, compactEncode helper
src/ToolHandler.ts Pass experimentalGcfFormat to handle()
package.json Add @blackwell-systems/gcf (pinned 2.1.2, zero deps)

Links

@zyzyzyryxy zyzyzyryxy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution and detailed benchmarking!

I left some comments to be addressed, but overall GCF idea looks good.

Comment thread src/bin/chrome-devtools-mcp-cli-options.ts Outdated
Comment thread src/third_party/index.ts Outdated
Comment thread src/McpResponse.ts Outdated
Comment thread package.json Outdated
"@rollup/plugin-json": "^6.1.0",
"@rollup/plugin-node-resolve": "^16.0.3",
"@stylistic/eslint-plugin": "^5.4.0",
"@blackwell-systems/gcf": "2.1.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to regenerate package-lock.json after this change.

@blackwell-systems

blackwell-systems commented Jun 26, 2026

Copy link
Copy Markdown
Author

Thanks for the detailed review @zyzyzyryxy! I've addressed all four points.

Rebased on top of #2260 (TOON optional dep) and reworked the approach:

  1. Single enum: Replaced both experimentalToonFormat and experimentalGcfFormat boolean flags with --experimentalDataFormat=json|toon|gcf. The legacy --experimentalToonFormat still works via fallback for backward compatibility.

  2. Optional peer dependency: @blackwell-systems/gcf is now an optional peer dep using dynamic import() via getGcfEncode(), following the exact pattern from chore: make TOON dependency optional #2260.

  3. Single encoder resolution: ToolHandler resolves the format choice into one dataFormat string, McpResponse.format() resolves it into one compactEncode function at the top. No more dual-boolean checks.

  4. package-lock.json: Regenerated.

  5. Rollup externals: Added @blackwell-systems/gcf.

Force-pushed with the reworked changes.

@blackwell-systems blackwell-systems force-pushed the experimental-gcf-format branch from 1296b19 to 1ad2d7a Compare June 26, 2026 21:01

@zyzyzyryxy zyzyzyryxy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The improvements look good, force-pushing is a good approach in this case.
I pointed out a few minor issues that remain.

One think that might be a blocker for us is there's no provenance attestations for the package on npm. Could you configure those for future builds?

Please also consider adding some unit tests - they are not strictly required for experimental feature, but we'll need them in case we do decide to make GCF the default. This might be a separate PR I think.

type: 'string',
describe:
'Format for structured data in text responses. "json" (default), "toon" (requires @toon-format/toon), or "gcf" (requires @blackwell-systems/gcf).',
choices: ['json', 'toon', 'gcf'] as const,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default choice is not json, it's custom formatters.
json is used for structuredContent, not for text representation of data returned.

Comment thread package.json Outdated
@blackwell-systems

Copy link
Copy Markdown
Author

Thanks @zyzyzyryxy. Addressed the remaining points:

  1. Default description: Updated to clarify that the default uses built-in formatters, not "json". The flag overrides structured content encoding with the specified format.

  2. Version pinning: Changed to ^2.2.2 in peerDependencies. Added to devDependencies for type checking.

  3. Provenance: Published @blackwell-systems/gcf@2.2.2 with npm provenance attestations (SLSA v1, built from GitHub Actions). Verifiable at: https://www.npmjs.com/package/@blackwell-systems/gcf#provenance

Happy to follow up with unit tests in a separate PR.

blackwell-systems added a commit to blackwell-systems/chrome-devtools-mcp that referenced this pull request Jun 30, 2026
Address review feedback from ChromeDevTools#2235:

1. Replace two boolean flags (experimentalToonFormat, experimentalGcfFormat)
   with a single --experimentalDataFormat=json|toon|gcf enum. The legacy
   --experimentalToonFormat flag still works via fallback for backward
   compatibility.

2. Make @blackwell-systems/gcf an optional peer dependency following the
   same pattern established for @toon-format/toon in ChromeDevTools#2260. Uses dynamic
   import() so the package is only loaded when the format is requested.

3. Resolve format choice into a single compactEncode function at the top
   of format(), eliminating the dual-boolean checks throughout.

4. Add @blackwell-systems/gcf to rollup externals.

5. Regenerate package-lock.json.
@zyzyzyryxy

Copy link
Copy Markdown
Contributor

I see your updated commit, but it's on attached to this PR, you need to push to the same branch to fix.
While at it, please merge current main and resolve conflicts.

  1. Default description: Updated to clarify that the default uses built-in formatters, not "json". The flag overrides structured content encoding with the specified format.

I still see 'json' | 'toon' | 'gcf' in the code, this should be 'default' | 'toon' | 'gcf'

  1. Provenance: Published @blackwell-systems/gcf@2.2.2 with npm provenance attestations (SLSA v1, built

Thank you!

@blackwell-systems blackwell-systems force-pushed the experimental-gcf-format branch from 1ad2d7a to 779491c Compare July 3, 2026 19:46
@blackwell-systems

Copy link
Copy Markdown
Author

Rebased on current main, addressed all feedback:

  • 'json''default' in DataFormat type and CLI choices
  • Merged upstream heap snapshot diff/detail features (converted to compactEncode pattern)
  • All tests passing, type checks clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants