Skip to content

Add addon version, feature adoption and command failure metrics#3038

Merged
tastybento merged 1 commit into
developfrom
feature/bstats-telemetry
Jul 26, 2026
Merged

Add addon version, feature adoption and command failure metrics#3038
tastybento merged 1 commit into
developfrom
feature/bstats-telemetry

Conversation

@tastybento

Copy link
Copy Markdown
Member

Adds six bStats charts and removes three that were sending data nowhere. All six chart IDs already exist on bstats.org/plugin/bukkit/BentoBox/3555, so data starts landing ~30 minutes after servers update.

Closes #3022.

New charts

Chart ID Type What it answers
addonVersions Drilldown Pie Which version of each addon is actually deployed
gameModeVersions Drilldown Pie Same, for game modes
addonApiVersions Advanced Pie Which BentoBox API levels addons in the wild are built against
features Simple Bar Which settings servers change away from default
commandFailures Advanced Pie How commands fail, by kind
commandFailurePaths Simple Bar Which commands fail most

Addon versions. We knew that a server ran Level, not which Level. That was the biggest blind spot: no way to see how long the tail is after a release, or to judge when a breaking API change is safe.

Non-default settings. Reports only deviations from default, so every bar is a signal rather than a restatement of the defaults. A default-off setting appears under its own name when a server enables it (chunk-pregen); a default-on setting appears as no-<name> when disabled (no-brigadier-commands). Comparing a bar to the total server count gives that feature's adoption or rejection rate. 24 settings covered, including all four dialog toggles, both did-you-mean toggles and Brigadier — so we can finally see whether servers are switching off recently shipped work.

Command friction (#3022). CompositeCommand.call() is the single funnel where every failure mode is already distinguishable, so it now records which kind occurred and on which command. UNKNOWN_SUBCOMMAND is recorded only when did-you-mean actually produced a suggestion — the bare "leftover args and the command has subcommands" condition is too loose to be trusted, since plenty of commands legitimately have both. Everything else falls through and is classified as CANNOT_EXECUTE or BAD_ARGS naturally. That slice is therefore a lower bound, and it is gated on did-you-mean-subcommands being enabled.

The command key

Labels cannot be used to key this data. The top-level label comes from the game mode config, and getLabel() is overwritten by getCommandFromArgs() with whatever alias the player typed — so the same command would key differently across servers, and differently on the same server depending on typing.

The permission is hard-coded in setup() and carries the addon's prefix, so bskyblock.island.team.invite means the same thing everywhere. getStatsKey() uses it, prepending bentobox. for core commands (which have an empty prefix) and falling back to <addon>.<ClassName> for commands with no permission.

Privacy

Only the failure kind and that stable key are submitted — never arguments, player names or anything typed. bStats data is public, so this is the line to hold. Distinct keys are capped at 200 between submissions and only the busiest 20 are sent, so an addon with generated command names cannot grow the map without bound; failure counts by kind are never dropped.

Removals

The addons, gameModeAddons and hooks advanced pies are gone. Those chart IDs do not exist on bstats.org — verified against /api/v1/plugins/3555/charts — so every submission was discarded on arrival. addonsBar, gameModeAddonsBar and hooksBar carry identical data and do have charts. This is why the Hooks pie never showed anything.

Testing

./gradlew build green: 3366 tests, 0 failures (+11 new).

  • BStatsTest — accumulation by kind and by command, the 200-key cap, and that capped commands keep counting while new ones are dropped.
  • CommandFailureStatsTest — each failure path records the right kind and key, successful commands record nothing, and getStatsKey() for both the permission and class-name cases.

Also diffed every registration in BStats.java against the live chart config: 26 IDs, each either type-matched or a bStats built-in, zero mismatches in either direction.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ATRqygFLZ5CA3zrWmutx7H

Six new bStats charts, plus a cleanup of three that were sending data
into the void.

Addon and game mode versions (drilldown pies) and addon API versions
(advanced pie): we knew *that* a server ran Level, but not *which*
Level, so there was no way to see how long the tail is after a release
or judge when an API break is safe.

Non-default settings (bar chart): only deviations from default are
reported, so every bar is a signal. A default-off setting appears under
its own name when enabled, a default-on setting as no-<name> when
disabled. Comparing a bar to the server count gives the adoption or
rejection rate of a feature.

Command failures and top failing commands (#3022): CompositeCommand.call
is the single funnel where every failure mode is distinguishable, so it
now records the kind of failure and the command it happened on. Only the
failure kind and a stable command key are sent - never what the player
typed - because bStats data is public.

Labels cannot be used as that key: the top level label comes from the
game mode config and getLabel() is overwritten with whatever alias was
typed. The permission is hard coded in setup() and carries the addon
prefix, so bskyblock.island.team.invite means the same thing on every
server. Commands with no permission fall back to their class name.
Distinct keys are capped at 200 between submissions so an addon with
generated commands cannot grow the map without limit.

Removed the addons, gameModeAddons and hooks advanced pies. Those chart
ids do not exist on bstats.org, so the data was discarded on arrival.
The addonsBar, gameModeAddonsBar and hooksBar charts carry identical
data and do have charts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATRqygFLZ5CA3zrWmutx7H
@sonarqubecloud

Copy link
Copy Markdown

@tastybento
tastybento merged commit 313d06c into develop Jul 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: command-friction telemetry via bStats custom charts

1 participant