docs: the ClickBench shape labels describe the queries they are attached to (#533) - #535
Merged
Merged
Conversation
…hed to (#533) Six of the ten query shape labels in the 2026-08-05 section described a different query. The timings were fine; only the prose beside them was wrong, which is the awkward shape of it, because the numbers look checkable and the labels did not invite checking. The worst two: the losses table put "SELECT * of every column" on q23 when it belongs to q24, and the paragraph under the table inherited it. And q28 was labelled "GROUP BY a normalised URL", which is q29. In the wins table, q20 was called "COUNT(*) with a LIKE on a short column" when it is a point lookup, SELECT UserID WHERE UserID = <constant>, with no COUNT and no LIKE. It is not an off-by-one, which was the first hypothesis, because upstream ClickBench numbers from q0 while the harness prints from q1. Two labels fit the shift exactly and three fit the 1-based numbering instead, and one fits neither. The magnitudes confirm the numbering: the doc's largest loss is q24 at 11.6x and the file's q24 is the SELECT * query, which is also the largest loss on the 2026-08-09 run. So the labels were written from memory rather than read off the definition. The shapes are now read off the definition as recorded on 2026-08-06, queries.sql a7d6673357348ee9. That run predates the digest being recorded, so the section now says the labels are matched against the recorded definition rather than proved against the file that run fetched. Claiming more would be claiming what cannot be checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #533. (Supersedes #534, which GitHub auto-closed when its base branch
was deleted on merge of #526. Same branch, same commit, now based on main.)
Six of the ten query shape labels in the 2026-08-05 ClickBench section described
a different query. The timings are fine. Only the shape column is wrong, which is
the awkward part: the numbers look checkable and the prose beside them was not
inviting a check.
Losses table
SearchPhrase LIKE,ORDER BY EventTime LIMIT 10SELECT * FROM hits WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT 10SELECT *of every columnGROUP BY SearchPhrasewithTitle LIKE '%Google%'andCOUNT(DISTINCT UserID)GROUP BYa normalised URLGROUP BY CounterIDwithAVG(length(URL)). The normalised URL is q29SearchPhrase LIKE,ORDER BY EventTime LIMIT 10GROUP BY SearchPhrasewithURL LIKE '%google%', ordered by the countThe
SELECT *label sat on q23 when it belongs to q24, and the paragraph underthe table inherited it word for word.
Wins table
GROUP BYa URL prefixURLHash, EventDate, q42 byWindowClientWidth, WindowClientHeightCOUNT(*)with aLIKEon a short columnSELECT UserID FROM hits WHERE UserID = 435090932899640449, a point lookup with noCOUNTand noLIKEIt is not an off-by-one
That was the first hypothesis, and it is a good one: upstream ClickBench numbers
from q0 and the harness prints from q1. Two labels fit the shift exactly, the
doc's q23 being the file's q24 and the doc's q28 being the file's q29. But q3, q7
and q21 fit the 1-based numbering and not the shifted one, and "GROUP BY a URL
prefix" fits neither q41/q42 nor q42/q43. So it is not a systematic shift.
The magnitudes settle which numbering the table uses. The doc's largest loss is
q24 at 11.6x, and the file's q24 is the
SELECT *query, which is also thelargest loss on the 2026-08-09 run at 6.19x. The doc's q41 and q42 are 25x wins,
and those are 17.8x and 16.8x wins on 2026-08-09. The numbering is 1-based and
only the prose was wrong: the labels were written from memory rather than read
off the definition.
What this does not claim
The 2026-08-05 run predates the definition digest being recorded, so there is no
proof it fetched this
queries.sql. The section now says the labels are matchedagainst the definition as recorded on 2026-08-06,
queries.sql a7d6673357348ee9, rather than implying the run itself was pinned.Claiming more would be claiming something that cannot be checked.
test/docs_style.shpasses 6/6.