feat: add update-breakage observability (survivorship-free)

[ineagu]

Outcome

When a site updates Otter (or WordPress) and a block stops rendering, the editor silently shows an invalid/missing block and the user just... leaves. Today we only learn about update breakage from the people who are unhappy enough to open a support ticket — a deeply survivorship-biased signal. We never see the cohort that hits a broken page and churns quietly.

This PR adds two lightweight, opt-in telemetry surfaces that answer a concrete product question:

"Which Otter block types break after an update, and across which version transitions?"

That lets us prioritize backward-compatibility fixes and deprecation handling for the blocks that actually break in the field, on the version jumps that actually happen — instead of guessing from the loud minority. It is "survivorship-free" because the editor-side check runs for every session that opens a post containing a broken Otter block, not only for users who complain.

What changed

• inc/class-main.php — In after_update_migration(), when the stored DB version is older than the current plugin version (a real upgrade, not a fresh install — guarded by ! empty( $db_version ) && '0' !== (string) $db_version), fire two server-side events via Tracker::track(). Adds a private coarsen_version() helper that reduces any version string to MAJOR.MINOR.

  Event shapes:

  feature	featureComponent	featureValue
  lifecycle	plugin-upgrade	"<fromMajorMinor>><toMajorMinor>" e.g. 3.0>3.1
  lifecycle	wp-at-upgrade	"<wpMajorMinor>" e.g. 6.8

• src/blocks/plugins/block-health/index.js (new) — Runs once per editor session after the editor is ready. Walks all blocks (including inner blocks), and flags an Otter block as broken when either: it is isValid === false and its name is in the themeisle-blocks category, or it is a core/missing block whose originalName is an Otter block. Reports only the block-type slug — never any attribute values or content.

  Event shapes:

  feature	featureComponent	featureValue
  block-health	render-error	<block-type-slug> e.g. themeisle-blocks/advanced-heading
  block-health	page-error	"1" (emitted once if the page has any errored Otter block)

• src/blocks/plugins/registerPlugin.tsx — Registers the new block-health plugin alongside the existing data-logging import.

Compliance

• Non-PII. Nothing user-identifiable, no post content, no attribute values, no URLs in the event payload. Only: coarsened MAJOR.MINOR version numbers and Otter block-type slugs. Block-type slugs come from registered block types in the themeisle-blocks category, so they are effectively an allowlist of our own block names — a user cannot inject an arbitrary string into featureValue. Versions are bucketed to MAJOR.MINOR via coarsen_version(), which both reduces cardinality and avoids exposing a precise build fingerprint.
• Honors the existing consent gate, with no bypass. Both surfaces ride the existing otter_blocks_logger_flag opt-in:
  • PHP: Tracker::track() short-circuits unless Tracker::has_consent() (which reads otter_blocks_logger_flag) is true. We call Tracker::track( $events ) with no options argument, so we do not pass hasConsent (the documented bypass) — a user who has not opted in sends nothing.
  • JS: events are sent with window.oTrk?.set( ... ) and no { consent: true } option. Unlike the AI-content paths (which intentionally pass { consent: true }), block-health respects tiTrk's consent state, which is itself derived from otter_blocks_logger_flag (surfaced to the editor as canTrack).
• The editor-side code is additionally scoped to run only in the block editor (window.themeisleGutenberg?.isBlockEditor and a live core/editor store).

Test plan

Server-side (lifecycle):

1. Opt in to tracking: set the Otter usage-tracking toggle on (Dashboard), i.e. otter_blocks_logger_flag = yes.
2. Set an older stored version: wp option update themeisle_blocks_db_version 3.0.5.
3. Trigger the upgrade path (load wp-admin so after_update_migration() runs, or call it directly). With current OTTER_BLOCKS_VERSION you should see two outbound POSTs to https://api.themeisle.com/tracking/events: a lifecycle/plugin-upgrade event with featureValue like 3.0>3.1, and a lifecycle/wp-at-upgrade event with the coarsened WP version.
4. Negative check: set themeisle_blocks_db_version to 0 (or empty) and re-run — no lifecycle events fire (fresh-install guard).
5. Consent negative check: set otter_blocks_logger_flag off and repeat step 3 — Tracker::track() returns early, nothing is sent.

Client-side (block-health):

1. Opt in to tracking (so canTrack is true in the editor).
2. Create a post containing an Otter block, then force a render error — e.g. hand-edit the post content to corrupt an Otter block's markup so it becomes invalid, or deactivate a module so an Otter block resolves to core/missing.
3. Open the post in the editor. Watch the network tab for the tracking request: you should see a block-health/render-error event whose featureValue is the affected block slug, plus a single block-health/page-error event with featureValue: "1".
4. Negative check: open a post with only valid Otter blocks — no block-health events fire. Toggle tracking off — no events fire.

Aggregated events are visible downstream in the tracking pipeline (api.themeisle.com → Metabase), the same place the existing Otter usage events land.

Related

Part of the telemetry-expansion roadmap. Follows the established data-logging / usage-tracking pattern introduced in the block add/remove work (PR #2862), reusing the same oTrk/tiTrk accumulator on the JS side and the same Tracker + otter_blocks_logger_flag consent gate on the PHP side. Sibling to PR #2862; no schema or pipeline changes are required to ingest these events.

🤖 Generated with Claude Code

[pirate-bot]

Bundle Size Diff

Package	Old Size	New Size	Diff
Animations	178.24 KB	178.24 KB	0 B (0.00%)
Blocks	1.5 MB	1.5 MB	1.11 KB (0.07%)
CSS	7.87 KB	7.87 KB	0 B (0.00%)
Dashboard	108.48 KB	108.48 KB	0 B (0.00%)
Onboarding	68.14 KB	68.14 KB	0 B (0.00%)
Export Import	4.7 KB	4.7 KB	0 B (0.00%)
Pro	320.08 KB	320.08 KB	0 B (0.00%)

[pirate-bot]

Plugin build for d0ee079 is ready 🛎️!

• Download Otter – Page Builder Blocks & Extensions for Gutenberg - Production ~ Development
• Download Otter Pro - Production ~ Development
• Download Blocks Animation - Production ~ Development
• Download Blocks CSS - Production ~ Development
• Download Blocks Export Import - Production ~ Development

[pirate-bot]

E2E Tests

Playwright Test Status: See serial and parallel matrix jobs

Performance Results

serverResponse: {"q25":384.1,"q50":390.25,"q75":422.2,"cnt":10}, firstPaint: {"q25":469,"q50":524.3,"q75":552.7,"cnt":10}, domContentLoaded: {"q25":3252,"q50":3274.95,"q75":3316.7,"cnt":10}, loaded: {"q25":3252.5,"q50":3275.4,"q75":3317.3,"cnt":10}, firstContentfulPaint: {"q25":8778,"q50":8842.4,"q75":8857,"cnt":10}, firstBlock: {"q25":13376.1,"q50":13411.95,"q75":13515.1,"cnt":10}, type: {"q25":21.03,"q50":22.11,"q75":23.72,"cnt":10}, typeWithoutInspector: {"q25":19.12,"q50":19.55,"q75":20.72,"cnt":10}, typeWithTopToolbar: {"q25":29.13,"q50":30.33,"q75":31.96,"cnt":10}, typeContainer: {"q25":12.84,"q50":13.45,"q75":13.99,"cnt":10}, focus: {"q25":103.59,"q50":105.56,"q75":110.14,"cnt":10}, inserterOpen: {"q25":37.22,"q50":37.56,"q75":39.42,"cnt":10}, inserterSearch: {"q25":12.91,"q50":13.38,"q75":13.73,"cnt":10}, inserterHover: {"q25":5.03,"q50":5.23,"q75":5.62,"cnt":20}, loadPatterns: {"q25":1493.56,"q50":1525.5,"q75":1607.29,"cnt":10}, listViewOpen: {"q25":207.54,"q50":214.19,"q75":217.91,"cnt":10}