Skip to content

Record what the ClickBench licence actually forbids (#421) - #457

Open
ChronicallyJD wants to merge 1 commit into
commandprompt:mainfrom
ChronicallyJD:docs/clickbench-not-published
Open

Record what the ClickBench licence actually forbids (#421)#457
ChronicallyJD wants to merge 1 commit into
commandprompt:mainfrom
ChronicallyJD:docs/clickbench-not-published

Conversation

@ChronicallyJD

@ChronicallyJD ChronicallyJD commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Owner determination, 2026-08-06, after reading CC BY-NC-SA: publishing our own test results is
permitted.
We measure to compare, not to sell.

I had this wrong and this PR now says the opposite of what it opened with

The first version removed the load-and-size table and the per-query speedup ratios from
docs/benchmarks.md, on the reading that no ClickBench-derived numbers could be published. That
reading was wrong. The removal is reverted and docs/benchmarks.md is byte-identical to
main apart from one added caution.

Nothing about those measurements was ever in doubt. Only what we were entitled to do with them.

What the term actually forbids

Now written down rather than left to inference:

  • selling copies of the material, or of a remix of it
  • placing it on a website or platform monetised by ads where revenue generation is the primary goal
  • using it in promotional materials, corporate brochures, or paid advertisements for a business
  • charging a fee to access, download, or view a derivative work built on it

The third is the one worth guarding, because it is the one a well-meaning reader walks into. A
benchmark table in documentation exists so a user can decide whether this engine suits their
workload, and so we can see our own regressions. The same table in a sales deck, a conference
booth panel or a paid campaign is the prohibited use — and having produced the numbers ourselves
does not change that, because the benchmark definition they were produced with is the licensed
material.

Where each piece lives

  • docs/benchmarks.md gains a short caution beside the numbers, where someone about to lift
    them will actually see it. Everything else in that file is unchanged.
  • PROVENANCE.md carries the full list, the determination, and its reasoning. No legal
    opinion was sought, and it is recorded so it can be revisited rather than re-derived.

It also records the run-time fetch decision from the same day, which #453 implements.

On #453

Its PROVENANCE.md paragraph reached the opposite conclusion on this question before the owner
had made one, and I held it for that reason. The owner has now ruled, and the substance of that
paragraph is closer to right than my first attempt was — but it was still recording a
determination that had not been taken. Both of us should let that file follow the owner rather
than lead them.

…mpt#421)

Owner determination, 2026-08-06, after reading CC BY-NC-SA: publishing our own
test results is permitted. We measure to compare, not to sell.

An earlier version of this commit removed the ClickBench numbers from
docs/benchmarks.md on a reading that no numbers could be published. That reading
was wrong and the removal is reverted. Nothing about those measurements was in
doubt at any point; only what we were entitled to do with them.

What the term forbids, now written down rather than left to inference:

  - selling copies of the material, or of a remix of it
  - placing it on a site monetised by ads where revenue is the primary goal
  - using it in promotional materials, corporate brochures or paid advertisements
  - charging a fee to access, download or view a derivative work built on it

The third is the one worth guarding, because it is the one a well-meaning reader
walks into. A benchmark table in documentation exists so a user can decide whether
this engine suits their workload. The same table in a sales deck is the prohibited
use, and having produced the numbers ourselves does not change that, because the
definition they were produced with is the licensed material.

So the caution sits beside the table in docs/benchmarks.md, where someone about to
lift it will see it, and the full list sits in PROVENANCE.md beside the
determination and its reasoning. No legal opinion was sought, and it is recorded
so it can be revisited rather than re-derived.
@ChronicallyJD
ChronicallyJD force-pushed the docs/clickbench-not-published branch from 45c6e62 to 6ec0e96 Compare August 6, 2026 15:31
@ChronicallyJD ChronicallyJD changed the title ClickBench is an internal baseline, so stop publishing its numbers Record what the ClickBench licence actually forbids (#421) Aug 6, 2026
jdatcmd added a commit that referenced this pull request Aug 6, 2026
)

CJD is right, and the criticism lands on something I had already flagged and
then did anyway. Earlier today I noted that the NonCommercial term restricts use
"primarily intended for or directed toward commercial advantage" rather than
products as such, and that a project developed by a business is the gray part.
Then I wrote the provenance record resting on exactly the footing I had warned
against: "pgColumnar is an open-source project rather than a commercial product."
That is a stronger claim than the owner made and a riskier one.

It also omitted the four specific prohibitions, and the third is the one a
well-meaning reader walks into: a benchmark table in documentation is fine, the
same table in a sales deck is the prohibited use, and having produced the numbers
ourselves does not change that because the definition they came from is the
licensed material. A record that says "acceptable use" without saying which uses
is the half that gets quoted.

#457 records the determination with the owner's own reasoning and the full list,
and puts the caution beside the numbers where someone about to lift them will see
it. Both PRs touched this paragraph; this one drops it so #457 lands clean.

The runner change in this PR is unaffected and was approved as merge-ready.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E9W9N2tvsvK7hndJgTmqJf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant