Toolkit docs generator silently wipes summary on LLM unavailable / failure

[jottakka]

Summary

DataMerger.maybeGenerateSummary in toolkit-docs-generator/src/merger/data-merger.ts silently drops the toolkit summary field in two paths:

1. No LLM generator: when toolkitSummaryGenerator is undefined (e.g. --skip-summary, or missing LLM env vars on a scheduled run), and the tool signature has changed since the last output (so the reuse fast path doesn't match). The function returns without copying the previous summary forward.
2. LLM call fails: when the generator throws (rate limit, transient API error, invalid JSON response), the catch block records a warning but leaves result.toolkit.summary undefined.

Because the rendered toolkit page in app/_components/toolkit-docs/components/toolkit-page.tsx only shows the prose block when data.summary is truthy, this regression manifests on production Vercel builds as the overview block disappearing entirely for affected toolkits.

Impact

Rich, hand-refined summaries for the following toolkits were wiped to null across recent [AUTO] Adding MCP Servers docs update merges:

• github (wiped in PR [AUTO] Adding MCP Servers docs update #922)
• jira (wiped in PR [AUTO] Adding MCP Servers docs update #922)
• googledocs (wiped in PR [AUTO] Adding MCP Servers docs update #915)
• linear (wiped in PR [AUTO] Adding MCP Servers docs update #915)
• salesforce (wiped in PR [AUTO] Adding MCP Servers docs update #899)
• daytona (wiped in PR [AUTO] Adding MCP Servers docs update #899)

Root cause

Reuse fast path only matches when buildToolkitSummarySignature produces the same output — any tool add/remove/parameter change invalidates it. From there, both "no generator" and "generator threw" paths fall through to a no-op instead of falling back to the previous summary.

An earlier fix for path 1 was written on the PR #907 feature branch (commit 417a51fd fix: preserve previous summary when no LLM is available to regenerate) but was dropped during the merge to main.

Fix (proposed)

In both fallback paths, carry previousToolkit?.summary forward onto result.toolkit.summary. A slightly stale summary is strictly better than losing hand-refined content on every affected run.

Follow-up PRs will restore / draft truthful summaries for the six toolkits above plus the five that never received one (posthog, granola, microsoftoutlookcalendar, microsoftoutlookmail, zohocreatorapi).

Repro

1. Run the docs generator for a toolkit that has a non-null summary in the previous output.
2. Add or remove a tool so buildToolkitSummarySignature changes.
3. Either pass --skip-summary / unset the LLM env, or stub the LLM to throw.
4. Observe: output JSON has summary: null (was populated).

Linked PR

• Fix: fix/toolkit-docs-preserve-summary (draft)
• Restoration: separate draft PRs, linked from the fix PR.

[jottakka]

PRs filed

• Code fix: #927 — fix(toolkit-docs-generator): preserve previous summary when regen fails — draft
• Restore lost summaries (6 toolkits): #928 — chore(toolkit-docs): restore lost summaries for 6 toolkits — draft
• Add missing summaries (5 toolkits that never had one): #929 — chore(toolkit-docs): add missing summaries for 5 toolkits — draft

Recommended merge order: #927 → #928 → #929. Without #927, the next auto-update run that changes a toolkit's tool signature will wipe any summary whose regeneration falls through the two fallback paths.