Add GARI and generic detector layout support to decoder CLIs

[arshpreetmaan]

Overview

This PR makes the GARI transformed matrices introduced in #273 directly usable with the Tesseract and Simplex command-line decoders.

The main design change is that a GARI matrix DEM now preserves the original circuit detector IDs as a prefix and appends its virtual detectors as a suffix. Source-circuit shots can therefore be decoded without a GARI-specific remapping file: the physical syndrome occupies the prefix, while the virtual suffix is initialized to zero.

This PR also adds a generic --detector-orders option to the Tesseract CLI. This brings the C++ CLI closer to the Python API, where explicit detector orders can already be supplied through TesseractConfig.det_orders. The option is not specific to GARI and can be used with any compatible DEM.

Problems addressed

This PR addresses two related limitations.

1. Source circuits and GARI matrix DEMs have different detector counts

A GARI matrix DEM contains the original physical detectors followed by additional virtual detectors. Previously, the command-line decoders assumed that circuit shots and the decoding DEM used the same detector width.

That prevented safe decoding of sampled or saved source-circuit syndromes against a larger GARI matrix DEM, particularly for fixed-width formats such as b8.

2. The Tesseract CLI could not accept explicit detector orders

The Python API accepts detector-order permutations through TesseractConfig.det_orders, but the C++ CLI could only generate orders internally.

This PR adds a generic JSON input for explicit detector orders, allowing the same order lists to be used through either interface.

Source-aligned GARI matrix DEMs

By default, demutil.gari.circuit_to_gari now serializes detector rows as:

[source detector IDs][virtual detector IDs]

A syndrome produced by the source circuit can therefore be placed at the beginning of a zero-filled GARI syndrome without changing its detector IDs.

No GARI-specific layout or remapping JSON file is required.

The internal research-oriented block layout remains available through:

row_order="block"

That form uses the physical-X, physical-Z, virtual-Z, virtual-X row ordering from the matrix construction and is written with a _block.dem suffix. It is intended for matrix analysis and does not accept source syndromes directly as a prefix.

Source-width input handling

When both --circuit and --dem are supplied, Tesseract and Simplex:

• sample or read shots using the circuit detector count;
• interpret those detector IDs as the physical prefix of the supplied DEM;
• leave additional DEM detectors at zero;
• reject a circuit with more detectors than the DEM; and
• require the circuit and DEM observable counts to agree.

This applies to sampled shots and saved detection-event files, including fixed-width b8 data.

When --dem is supplied without --circuit, input records continue to use the full DEM detector count. Therefore, a source-width saved shot file should be decoded by supplying both its source circuit and its larger GARI matrix DEM.

The GARI matrix DEM is a decoding representation and must not be sampled directly. Shots must be sampled from the original circuit.

Explicit Tesseract detector orders

Tesseract now accepts:

--detector-orders FILE

The file uses the same list-of-lists representation as Python’s TesseractConfig.det_orders. Each inner list contains detector IDs in traversal order and must be a complete permutation of all detector IDs in the decoding DEM.

For example:

[
  [0, 2, 1, 3],
  [3, 1, 2, 0]
]

The implementation rejects:

• a non-array top level;
• an empty collection of orders;
• entries that are not arrays of nonnegative integers;
• incomplete orders;
• duplicate detector IDs; and
• detector IDs outside the DEM range.

--detector-orders cannot be combined with the internally generated detector-order options:

• --num-det-orders
• --det-order-seed
• --det-order-bfs
• --det-order-index
• --det-order-coordinate

Simplex does not use detector traversal orders, so it does not receive this option.

This PR deliberately leaves the existing BFS and coordinate detector-order generation behavior unchanged. Any correction to those existing generators can be handled in a separate focused PR.

Statistics

The existing statistics include the circuit and DEM paths used for the run.

Tesseract statistics now additionally record:

• detector_orders_path
• the effective num_det_orders

This records whether an explicit detector-order file was used and how many orders were decoded.

Generating a GARI matrix DEM

From a Python environment containing Stim, NumPy, SciPy, and the Tesseract Decoder Python package, generate a source-aligned GARI matrix DEM with:

python src/py/_tesseract_py_util/gari.py \
    --circuit circuit_file.stim \
    --prior xor \
    --out-dir gari_output

This writes:

gari_output/circuit_file_gari_xor.dem

The available prior policies are:

• paper
• xor
• lp-max-barred-cost

The same conversion is available through the public Python API:

import stim
from tesseract_decoder import demutil

circuit = stim.Circuit.from_file("circuit_file.stim")

gari_dem = demutil.gari.circuit_to_gari(
    circuit,
    prior_function=demutil.gari.tesseract_xor_prior_probabilities,
)

gari_dem.to_file("gari_output/circuit_file_gari_xor.dem")

Optional detector-order file

Explicit detector orders are optional. Without --detector-orders, Tesseract continues to use its existing internally generated orders.

For a source-aligned GARI matrix DEM, the Python helper can generate index-based source orders and append the virtual detector IDs:

import json
import stim
from tesseract_decoder import demutil, utils

circuit = stim.Circuit.from_file("circuit_file.stim")
gari_dem = stim.DetectorErrorModel.from_file(
    "gari_output/circuit_file_gari_xor.dem"
)

orders = demutil.gari.build_detector_orders(
    circuit,
    gari_dem,
    num_det_orders=5,
    method=utils.DetOrder.DetIndex,
    seed=1234,
)

with open("gari_orders.json", "w") as f:
    json.dump(orders, f)

The resulting file can be passed directly to --detector-orders.

Tesseract example

Build Tesseract:

bazel build --jobs=1 src:tesseract

Sample from the original circuit and decode with the GARI matrix DEM:

./bazel-bin/src/tesseract \
    --circuit circuit_file.stim \
    --dem gari_output/circuit_file_gari_xor.dem \
    --detector-orders gari_orders.json \
    --sample-num-shots 100 \
    --sample-seed 1234 \
    --threads 1 \
    --pqlimit 1000000 \
    --beam 5 \
    --beam-climbing \
    --no-revisit-dets \
    --print-stats \
    --stats-out tesseract_gari_stats.json

To use Tesseract’s existing internally generated detector orders, omit --detector-orders and use the usual detector-order options.

Simplex example

Build Simplex:

bazel build --jobs=1 src:simplex

Sample from the original circuit and decode with the same GARI matrix DEM:

./bazel-bin/src/simplex \
    --circuit circuit_file.stim \
    --dem gari_output/circuit_file_gari_xor.dem \
    --sample-num-shots 100 \
    --sample-seed 1234 \
    --threads 1 \
    --print-stats \
    --stats-out simplex_gari_stats.json

Simplex follows the same source-prefix syndrome convention but does not require a detector-order file.

Saved source-shot data

A source-width detection-event file can also be decoded against a larger GARI matrix DEM:

./bazel-bin/src/tesseract \
    --circuit circuit_file.stim \
    --dem gari_output/circuit_file_gari_xor.dem \
    --in source_shots.b8 \
    --in-format b8 \
    --threads 1 \
    --beam 5 \
    --beam-climbing \
    --no-revisit-dets \
    --print-stats

Providing the circuit is important because it defines the width of each source-shot record. With --dem alone, the reader expects full GARI-width records.

[LalehB]

can you also make sure that you update the README with examples as well please?

[arshpreetmaan]

The CI failure appears unrelated to the GARI changes. It seems to come from Bazel reusing cached binaries built on a different CPU. A possible fix is to disable only the Bazel disk cache:
In [.github/workflows/ci.yml (line 67)]

with:
  bazelisk-cache: true
  disk-cache: false
  repository-cache: true

This keeps the other caches enabled while ensuring native binaries are rebuilt on each runner.

[mhucka]

The CI failure appears unrelated to the GARI changes. It seems to come from Bazel reusing cached binaries built on a different CPU.

@arshpreetmaan Can you provide more info about the "different CPU" part? Since the GitHub workflows always run on the same type of runner, one would expect the CPU to be the same. Do you suspect a difference in the CPU feature sets?

[mhucka]

One thing I do see, though, is that the disk-cache should be further parametrized by the matrix OS. I'll do a quick PR.

[arshpreetmaan]

One thing I do see, though, is that the disk-cache should be further parametrized by the matrix OS. I'll do a quick PR.

Thanks, PR #295 might fix the issue. By “different CPU,” I meant that separate GitHub-hosted ubuntu-latest VMs may expose different CPU instruction features, even though they are all Linux x64 runners. Since the build uses -march=native, cached binaries can depend on those exact features.
The new per-OS cache name should clear the existing cache and likely fix the current failure in #277. We can rerun #277 after #295 is merged; if the Illegal instruction failure returns later, we may need to consider other options (perhaps making -march=native optional or using -march=x86-64 for CI).

[LalehB]

Thanks Arshpreet!

[noajshu]

Hi @arshpreetmaan @LalehB , WDYT about this alternative API:
just like how tesseract accepts detector orders via its python module API, we could accept one or more detector orders via the CLI. And the GARI code in demutil code can generate the gari-aware detector orders too.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[LalehB]

Hi @arshpreetmaan @LalehB , WDYT about this alternative API: just like how tesseract accepts detector orders via its python module API, we could accept one or more detector orders via the CLI. And the GARI code in demutil code can generate the gari-aware detector orders too.

@noajshu I think your proposed alternative API would be a good addition. I dunno if @arshpreetmaan addressed that.

[arshpreetmaan]

Hi @arshpreetmaan @LalehB , WDYT about this alternative API: just like how tesseract accepts detector orders via its python module API, we could accept one or more detector orders via the CLI. And the GARI code in demutil code can generate the gari-aware detector orders too.

@noajshu I think your proposed alternative API would be a good addition. I dunno if @arshpreetmaan addressed that.

Hi @LalehB @noajshu , I have applied the suggested API changes. Can you PTAL?

[noajshu]

I think this is becoming more complicated than necessary. In particular, I don't think detector remapping and detector ordering need to be coupled into a new detector_layout schema.

For GARI, can we instead have the generated DEM preserve the original detector IDs for the physical detectors and append the virtual detectors as a suffix? Then shots from the source circuit need no remapping at all: the first circuit.num_detectors entries are the physical syndrome and the remaining DEM detectors are implicitly zero. The C++ CLI would only need to support this source-prefix/augmented-DEM case when reading or sampling shots, rather than introducing a general mapping format.

Separately, if we want the C++ CLI to support explicit detector orders, I think that should just be a detector-orders file mirroring TesseractConfig.det_orders in the Python API.

Also, there is currently an inconsistency in what a “detector order” means between build_det_orders, the C++ decoder, and the Python-facing API. I think we should fix that first and use one representation everywhere (matching the current python API.)