Skip to content

ci(jit): an instrument for the crash in #726, before anyone edits the emitter - #731

Open
gHashTag wants to merge 1 commit into
mainfrom
feat/jit-crash-forensics
Open

ci(jit): an instrument for the crash in #726, before anyone edits the emitter#731
gHashTag wants to merge 1 commit into
mainfrom
feat/jit-crash-forensics

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

Produces the evidence #726 asks for. Closes nothing yet.

Why an instrument and not a fix

jit.test.JitCompiler dot product correctness does not fail an assertion — it kills the test process. zig prints the following test command failed with no error name and no stack trace, unlike the five sibling failures in the same log which all carry one. A fault inside emitted machine code never reaches zig's panic handler.

It also cannot be reproduced anywhere convenient. Each JIT test guards itself with if (builtin.cpu.arch != .x86_64) return error.SkipZigTest (src/jit.zig:571, 596, 630, 667). ubuntu-latest is x86-64, so the guard does not fire there and the emitted code genuinely runs — on the CI runner and nowhere else. This machine is arm64, where it self-skips, and the pinned Zig 0.15.2 does not link on this macOS.

Guessing at a segfault with no instrument is how a session becomes twenty hours of repairing hardware that was never broken.

What it does

  • Runs the whole suite — src/jit.zig imports the vsa module, so zig test cannot build it standalone — and recovers the core afterwards.
  • Sets kernel.core_pattern to a plain path first: the runner pipes cores to apport, which discards them for a process it does not recognise.
  • gdb -batch for bt full, rip/rsp/rbp, threads and x/8i $rip, with the exec path read from the core's own notes so the trace symbolises.
  • Uploads test.log and every core.

Its contract

It gates the instrument, not the JIT: it fails when the process dies and no core was written, because that is the instrument failing rather than the subject. When the crash does not reproduce it says so plainly instead of reporting green.

workflow_dispatch plus the two paths that can change the answer, so it never destabilises Codegen Validation — which only just reached green for the first time.

… emitter

`jit.test.JitCompiler dot product correctness` does not fail an assertion. It
kills the test process: zig prints "the following test command failed" with no
error name and no stack trace, because a fault inside emitted machine code never
reaches zig's panic handler. Five sibling failures in the same log each carry a
trace; this one carries nothing.

So there is nothing to reason from, and the tests guard themselves against being
reasoned about anyway -- each JIT test skips unless `builtin.cpu.arch == .x86_64`,
which means the emitted code runs on the CI runner and nowhere else. It cannot be
reproduced on an arm64 machine, and the pinned Zig 0.15.2 does not link there.

This workflow produces the missing observation -- signal, faulting address,
backtrace -- rather than a patch written blind. src/jit.zig imports the `vsa`
module and so cannot be built standalone with `zig test`; the suite runs whole
and the core is recovered afterwards. The runner pipes cores to apport by
default, which drops them, so core_pattern is set to a plain path first.

Its contract is deliberately about the instrument, not the subject: it fails when
the process dies and no core was written -- an instrument that failed -- and
reports plainly when the crash does not reproduce. Manual dispatch plus the two
paths that can change the answer, so it never destabilises the main gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant