feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235
feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235blackwell-systems wants to merge 1 commit into
Conversation
zyzyzyryxy
left a comment
There was a problem hiding this comment.
Thank you for the contribution and detailed benchmarking!
I left some comments to be addressed, but overall GCF idea looks good.
| "@rollup/plugin-json": "^6.1.0", | ||
| "@rollup/plugin-node-resolve": "^16.0.3", | ||
| "@stylistic/eslint-plugin": "^5.4.0", | ||
| "@blackwell-systems/gcf": "2.1.2", |
There was a problem hiding this comment.
You need to regenerate package-lock.json after this change.
|
Thanks for the detailed review @zyzyzyryxy! I've addressed all four points. Rebased on top of #2260 (TOON optional dep) and reworked the approach:
Force-pushed with the reworked changes. |
1296b19 to
1ad2d7a
Compare
zyzyzyryxy
left a comment
There was a problem hiding this comment.
The improvements look good, force-pushing is a good approach in this case.
I pointed out a few minor issues that remain.
One think that might be a blocker for us is there's no provenance attestations for the package on npm. Could you configure those for future builds?
Please also consider adding some unit tests - they are not strictly required for experimental feature, but we'll need them in case we do decide to make GCF the default. This might be a separate PR I think.
| type: 'string', | ||
| describe: | ||
| 'Format for structured data in text responses. "json" (default), "toon" (requires @toon-format/toon), or "gcf" (requires @blackwell-systems/gcf).', | ||
| choices: ['json', 'toon', 'gcf'] as const, |
There was a problem hiding this comment.
Default choice is not json, it's custom formatters.
json is used for structuredContent, not for text representation of data returned.
|
Thanks @zyzyzyryxy. Addressed the remaining points:
Happy to follow up with unit tests in a separate PR. |
Address review feedback from ChromeDevTools#2235: 1. Replace two boolean flags (experimentalToonFormat, experimentalGcfFormat) with a single --experimentalDataFormat=json|toon|gcf enum. The legacy --experimentalToonFormat flag still works via fallback for backward compatibility. 2. Make @blackwell-systems/gcf an optional peer dependency following the same pattern established for @toon-format/toon in ChromeDevTools#2260. Uses dynamic import() so the package is only loaded when the format is requested. 3. Resolve format choice into a single compactEncode function at the top of format(), eliminating the dual-boolean checks throughout. 4. Add @blackwell-systems/gcf to rollup externals. 5. Regenerate package-lock.json.
|
I see your updated commit, but it's on attached to this PR, you need to push to the same branch to fix.
I still see
Thank you! |
1ad2d7a to
779491c
Compare
|
Rebased on current main, addressed all feedback:
|
Summary
Add
--experimentalGcfFormatas a hidden experimental flag alongside the existing--experimentalToonFormat. When enabled, tool responses encode structured data using GCF (Graph Compact Format) instead of formatted JSON. No changes to existing behavior or default output.Why
Benchmarked on Chrome DevTools data shapes
Data shapes verified from source code and test snapshots (
ConsoleMessageConcise,NetworkRequestConcise,HeapSnapshotFormatter.toJSON(),SnapshotFormatter.toJSON()):TOON increases token count on DOM snapshots (-4.6% to -17.2% vs JSON). GCF saves tokens on every data type.
Console message corruption
TOON's decoder crashes on console messages containing bracket-colon patterns, which are standard browser output:
10 of 20 console messages fail TOON round-trip. GCF: zero failures. While Chrome DevTools MCP encodes only (no decode), the structurally ambiguous output can affect downstream consumers.
LLM comprehension
GCF scores 100% on general structured data and 90.7% on adversarial payloads across GPT-4o, GPT-5.5, Claude, and Gemini. JSON scores 53.6%. TOON scores 68.5%. (1,700+ evaluations.)
Full eval data: GCF benchmarks
Data integrity
GCF verified lossless across 43 billion+ round-trips in 5 formats and 6 languages. Zero failures. Zero runtime dependencies.
Implementation
Mirrors the existing
experimentalToonFormatpattern exactly:useGcfparameter threaded throughhandle()→format()compactEncodehelper selects GCF, TOON, or null (formatted JSON)Changes
src/bin/chrome-devtools-mcp-cli-options.tsexperimentalGcfFormatflagsrc/third_party/index.tsgcfEncodefrom@blackwell-systems/gcfsrc/McpResponse.tsuseGcfparameter,compactEncodehelpersrc/ToolHandler.tsexperimentalGcfFormattohandle()package.json@blackwell-systems/gcf(pinned 2.1.2, zero deps)Links