Conversation
The corpus is a directory of YAML files versioned with the engine and
shipped to every client in the family. Java is the one client with no
runner for it, which is the gap that keeps the differential suite from
being green across all five.
This is the first piece: the reader. It is a new module rather than
sources under an existing one, and it holds main sources for the same
reason zudb-tck does, because a test source set is not something another
module can depend on without a test jar and the trouble a test jar
brings on the module path. It ends up both a command a person runs
against a corpus directory and a fixture a test can call.
It compiles to 25 rather than to the 17 the API targets. The Arrow half
of the corpus wants a result read through the C Data Interface, and
doing that without taking a dependency on arrow-java wants
java.lang.foreign.
The reader is the fifth implementation of the same small subset of YAML,
after the ones in the engine, in the C client, in zu-python and in
zu-go. Writing a fifth rather than reaching for a YAML library is
deliberate and the tests are where it shows: a library would read these
files and would read a great deal more besides, and what the corpus
needs is a reader that refuses. A case using a flow sequence, an anchor,
a tag or a block scalar has to come back as a refusal with a line number
rather than as something the author did not write, because a construct
one implementation quietly accepts is a case that passes in one
repository and fails in four.
So the refusals are checked in full rather than by substring. They are
diffed against the reference runner's output and a wording that drifted
would be a difference in the report that is not a difference in the
answer. Twenty five refused constructs and eleven accepted shapes, in
twelve tests.
Two things Go spells and Java does not needed care. Java has no escape
for a vertical tab, so the one in the whitespace set is written as its
code point. And the quoting of a value into a message follows Rust's
{:?} rather than anything on the JVM, since Java and Rust disagree about
which characters are worth escaping and the messages are compared across
languages.
The four spellings a case writes a date, a time, a datetime and a duration in, read by hand rather than handed to a formatter from the library. A corpus reader is a second opinion about the text, and a second opinion that calls the same code the client calls is not one. A formatter also takes a great deal this encoding does not: a pattern with a fractional second matches text with none, a numeric zone matches Z under some patterns and not others, and none of that is visible at the call site. Writing the spellings out says exactly what is accepted, which is the extended ISO 8601 form and nothing else. The seven types come back as dev.zudb.Value.Temporal rather than as seven java.time types, because that is the one shape the client hands back and a comparison has to be against it. The two conventions that shape carries are kept: a zoned datetime holds the instant in UTC with the offset beside it, so two texts an hour apart in zones an hour apart hold one count, and a zoned time holds the clock as written, so 12:00:00+07:00 and 05:00:00Z are two values rather than one. A duration is months or it is nanoseconds and never both. That leaves one text the fields decide and the numbers cannot, a duration of nothing, and P0M and PT0S stay two values here where the Python runner has to call them one. Nine tests, ported case for case from the Go runner's, and most of them are texts that look like temporal values and are not: a basic-form date, a leap second, a fraction of ten digits, an offset past the standard's own limit, a fraction of a year.
Every value a case can write, in both directions: the text a case wrote becomes a value to bind or compare against, and a value the engine handed back becomes the text a report prints. The shape is a sealed Cell rather than dev.zudb.Value, for two reasons that are the same reason twice. Value is sealed by the client, so a corpus that needs a shape the client does not have could not add one. And the two disagree about what a node is: the engine's node carries a table id where a case writes a table name, and its edge carries a field the corpus has no spelling for. Converting at the boundary means a comparison is a comparison and not a walk through two shapes at once. A type is written quoted or bare and never both, which is the rule that lets a reader tell an integer a case meant from an integer YAML would have handed it. Anything wider than what a double holds exactly is written quoted, so INT64 and the two 64 bit unsigned widths are quoted types and INT32 is not. DECIMAL is reserved rather than unknown, since the encoding has a spelling for it and the engine has no value, and saying so is more use than saying it is not a type. The float printer is the one place the JVM had to be talked out of its own answer. Double.toString has given the shortest run that reads back since release 19, except where a single digit would do: its specification then asks for the closest decimal of one digit or two, and for the smallest subnormal the two digit one is closer. So Java alone would print 4.9e-324 where the reference runner and every other client print 5e-324. Two digits is the only length that can be one too many, so one attempt at shortening closes it and the round trip is what decides rather than a table of special cases. Seventeen tests, ported case for case from the Go runner's, and the float ones carry the values that catch a printer written the easy way: the smallest subnormal, the largest finite, a half way case, and the three that have no numeric spelling at all.
A case may say what the export of its result looks like, which is a list of column names and Arrow format strings, or that the export is refused. This is the half that reads one. It goes through java.lang.foreign rather than arrow-java, and that is the reason this module compiles to 25 while the API targets 17. The alternative was a dependency on arrow-java for the sole purpose of reading a schema the client already hands over the C Data Interface, which would put a large library in the path of a test whose whole point is that no copy happens. The three structs are laid out here as they are in the interface, the three callbacks are called through downcall handles, and nothing on the path allocates a Java object per value. The schema is read off the stream rather than reconstructed from what the client knows about its own columns. That is what makes the answer worth comparing: the Java, Go and C runners all report the format strings the engine wrote, so a case that says a duration column is tDn is a case about the engine and not about three clients that each decided what to call it. A refusal is a type of its own. Arrow has no type for some of what the engine can return, and a case saying `arrow: refused` wants the stream to fail to open, which has to be told apart from a schema that opened and did not match. The refusal messages are the reference runner's word for word, including the one about a nested format that is not nested and the one about a nesting that is. The restricted method warnings are suppressed on the five methods that call one, which is what zudb-ffm already does in three files: release 24 put the restricted lint inside -Xlint:all, and this module builds with -Werror.
A suite is a header and a list of cases. A case is a statement, the connection to run it on, the parameters to bind, and exactly one account of what it produces: rows, or a GQLSTATUS, and never both and never neither. The reading is strict in the same way the YAML reader under it is strict, and for the same reason. A key nobody recognises is a refusal rather than a key that is ignored, because a case with a misspelled `raises` would otherwise be a case that says nothing and passes. A name is lower case words joined by dashes, checked against the file rather than trusted, because the name is what a report prints and what two runners are diffed by. Two cases in a suite may not share a name for the same reason. The schema version is checked before anything else, so that a corpus written for a later shape says so rather than failing somewhere in the middle with a message about a key. Go's [2]int for an edge is a Pair record here, so that two edges compare equal when they are equal. The rest of the model is records nested inside the Suite record, which keeps the port one file the way the original is one file while reading as Suite.read, Suite.Case and Suite.Load at the call sites. Fifteen tests, ported case for case from the Go runner's, and most of them are files that are nearly right: a case with rows and a raises, a row with the wrong number of values, a name with an underscore in it, a load whose edge points past its own row count.
The last piece: the thing that opens a database per case, puts the suite's load in, binds the parameters, runs the statement and says what came back. The report is the reference runner's line for line, down to the word order of every failure, because the value of a shared corpus is that a disagreement between two clients is a diff. An outcome is one of three things and not two: unsupported is the third, and it is what a client says about a statement the engine does not implement yet, which the corpus allows on purpose since the cases are the contract and the engine catches up to them. A load goes in through the bulk loader rather than through statements, which is the strongest form of the corpus question: the value crosses the boundary twice and by two different mechanisms, once as a column of a loader and once as a row of a result. A second connection is a duplicate of the case's own rather than a second open of the file, since the two share the write side and a case about a transaction means the first and not the second. Against the engine at d9f6b5d this reads 1399 cases and passes 1377, with 20 unsupported and 2 failed. The Go runner on the same corpus reports the same 20 unsupported, line for line and in the same order, and the same 2 failures plus 4 more that are its own pinned library being four months of reserved words behind. The two shared failures are the engine accepting a pre-reserved word where a name belongs, which is a gap in the engine and not in a client. One thing the corpus can write and this client cannot bind is a BYTES parameter, since Statement has no overload for a byte array. No case binds one today, so the runner says so by name rather than putting the value somewhere it does not belong, and the gap is filed rather than worked around.
The runner's own tests are the reference runner's, case for case: a case that says what it produces, a case that wants a condition, the eight ways one can fail, a case ahead of the engine, a named connection, bound parameters, a suite with a load, the database a failure leaves behind, what a report comes to, where compare stops, and the two shapes the report prints. Ported rather than written afresh, because a port that tests something else is a port nobody can diff. Three things the port needed that the Go tests get for free. A fresh directory per run, since a case that fails leaves its database behind on purpose and eight sub-cases all called `one` would collide where Go's t.TempDir gives each one its own. A handful of the runner's helpers made package private, because what the tests check is the status code, the unsupported rule, the column list and the case path, and a test that reaches them through the whole run checks the run instead. And a real ZuException rather than a made one, since the constructor is package private in dev.zudb and an empty statement is a better source of one anyway. The corpus test is gated on ZU_CASES and skips without it, the way zu-go and zu-python gate theirs. The cases live in the engine's repository and a client whose suite cannot run without a second repository beside it is one nobody clones to fix a typo. The module needs a provider and a grant to run at all, so it takes zudb-ffm at test scope and runs surefire off the module path with --enable-native-access=ALL-UNNAMED, which is what zudb-arrow does and what the README tells a person to pass. That puts zudb-corpus after zudb-ffm in the build order.
A job of its own rather than a step in the engine job, because the run is fourteen hundred databases and that job runs its suite three times over. The job builds the engine and reads the cases out of the same checkout. That pairing is the whole point: a corpus ahead of the library reports the engine catching up to its own cases as this client failing, and a library ahead of the corpus reports nothing at all. zu-go pins a revision because it ships an archive of the library; this client builds from source, so there is no revision to pin. The README gets the command a person runs against a corpus directory, the flags it takes, and the note that the same run happens under mvn test when ZU_CASES points at the cases and skips when it does not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The corpus is a directory of YAML cases versioned with the engine and answered by every client in the family. Until now this client was the one that did not run it, so it agreed with itself and nothing else. This adds
zudb-corpus: a reader for the case files, the value and temporal encodings the files are written in, the case model, an Arrow read over the C Data Interface, the runner, and the command that runs a directory of cases.Main sources rather than test sources, for the reason
zudb-tck's are: it is both a command a person runs and a fixture a test can call, and a test source set is not something another module depends on without a test jar. Nothing here is published. It compiles to 25 rather than the 17 the API targets, because reading a result as Arrow without a dependency on arrow-java wantsjava.lang.foreign.The report is the reference runner's word for word, so two clients disagreeing is a diff rather than a reading exercise. Run against the same corpus and the same engine build, this client and the Go client produce the same twenty unsupported lines, in the same order:
The four extra failures on the Go side are its pinned library being older, not a disagreement. The two shared failures were the same thing one layer down: both want 42001 for a pre-reserved word in a name slot, and
is_reserveddid not reach the pre-reserved list until 79b380d in the engine, which is newer than the build both runs used. The CI job here builds the engine from the checkout it reads the cases from, so it does not have that problem by construction.That pairing is the point of the job. A corpus ahead of the library reports the engine catching up to its own cases as this client failing, and a library ahead of the corpus reports nothing at all. zu-go pins a revision because it ships an archive of the library; this one builds from source, so there is no revision to pin. It is a job of its own and not a step in the engine job because the run is fourteen hundred databases while that job runs its suite three times.
Under
mvn testthe corpus run is gated onZU_CASESand skips without it, the way zu-go and zu-python gate theirs, so a checkout of this repository alone is still green. The rest of the module's tests are not gated and run everywhere.The runner's own tests are the reference runner's, case for case. Three things the port needed that the Go tests get for free: a fresh directory per run, since a case that fails leaves its database behind on purpose and eight sub-cases all called
onewould collide wheret.TempDirgives each its own; a handful of helpers made package private, because what the tests check is the status code, the unsupported rule, the column list and the case path; and a realZuExceptionfrom an empty statement rather than a made one, since the constructor is package private indev.zudb.The module takes
zudb-ffmat test scope and runs surefire off the module path with--enable-native-access=ALL-UNNAMED, which is whatzudb-arrowdoes and what the README now tells a person to pass. That puts it afterzudb-ffmin the build order.Verified on a Linux box against a release build of the engine:
and with
ZU_CASESset, the 1399 case run above.