Skip to content

Stop reporting skill-update no-op cycles as errors#3034

Merged
joethreepwood merged 2 commits into
mainfrom
posthog-code/fix-skills-update-noise
Jun 30, 2026
Merged

Stop reporting skill-update no-op cycles as errors#3034
joethreepwood merged 2 commits into
mainfrom
posthog-code/fix-skills-update-noise

Conversation

@joethreepwood

Copy link
Copy Markdown
Contributor

Problem

PostHog Code's UpdateSkillsSaga runs on a periodic refresh (and around skill-store activity). Its two skill-download steps are intentionally non-fatal, but the validate-skills step then hard-threw "No skills found from any source" whenever the staging dir ended up empty. So a transient network blip surfaced a misleading, opaque exception via captureException — even though the user's existing skills cache and the bundled skills were fully intact and nothing was actually broken. This was the single most common skills-related error in error tracking.

Why: raised from a support thread after the error was hit while creating a skill in the skill store — it's noise that masks real failures and reads as scary to users.

Changes

  • An empty download cycle is now treated as a no-op when a skills cache already exists: log a warning, keep the existing cache, skip the swap, and retry next interval — no exception reported.
  • Only when there is genuinely no cached skills to fall back on do we throw, and now with a clear, actionable message (likely cause, that bundled skills still work, and that it retries automatically) instead of the opaque "No skills found from any source".
  • skillsUpdated is only emitted when the cache actually changed.

How did you test this?

  • pnpm exec vitest run on posthog-plugin.test.ts — 26/26 pass, including a new regression test (empty cycle with an existing cache stays silent and preserves skills) and an updated test asserting the genuine no-cache failure reports the clear message, not the old opaque one.
  • Typecheck and Biome clean on the touched files.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Created with PostHog from a Slack thread

UpdateSkillsSaga's download steps are intentionally non-fatal, but the
validate step then hard-threw "No skills found from any source" whenever
the staging dir ended up empty. A transient network blip during the
periodic refresh therefore surfaced a misleading exception even though the
user's existing skills cache and the bundled skills were fully intact.

An empty download cycle is now a no-op when a skills cache already exists:
log a warning, keep the existing cache, and skip the swap. Only when there
is genuinely nothing to fall back on do we throw — and with a clear,
actionable message explaining the likely cause, that bundled skills still
work, and that it retries automatically. Also only emit `skillsUpdated`
when the cache actually changed.

Generated-By: PostHog Code
Task-Id: bb2c77d6-b22b-4f39-89a5-69c5b1c63001
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit d391bb4.

@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "Stop reporting skill-update no-op cycles..." | Re-trigger Greptile

@joethreepwood joethreepwood marked this pull request as ready for review June 30, 2026 16:41
@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "Merge branch 'main' into posthog-code/fi..." | Re-trigger Greptile

@joethreepwood joethreepwood merged commit d8a6f73 into main Jun 30, 2026
23 checks passed
@joethreepwood joethreepwood deleted the posthog-code/fix-skills-update-noise branch June 30, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants