Skip to content

Add chess.chesstb: pure-Python prober for chesstb endgame tablebases - #1194

Open
noobpwnftw wants to merge 1 commit into
niklasf:masterfrom
noobpwnftw:add-chesstb-tablebases
Open

Add chess.chesstb: pure-Python prober for chesstb endgame tablebases#1194
noobpwnftw wants to merge 1 commit into
niklasf:masterfrom
noobpwnftw:add-chesstb-tablebases

Conversation

@noobpwnftw

@noobpwnftw noobpwnftw commented Jun 8, 2026

Copy link
Copy Markdown

Add chess.chesstb: pure-Python prober for chesstb endgame tablebases

What this adds

A new module, chess.chesstb, that probes the chesstb endgame
tablebase format directly from chess.Board positions — in the same spirit as
the existing chess.syzygy and chess.gaviota modules.

chesstb ships four table types per material:

Table Extension Answer
WDL .lzw 50-move-rule-aware win/draw/loss with cursed/blessed classes
DTC .lzdtc distance-to-conversion (plies to the next zeroing move)
DTM .lzdtm unbounded distance-to-mate
DTM50 .lzdtm50 one pack giving both the unbounded DTM and the exact 50MR DTM at any halfmove clock

The DTM50 pack carries the unbounded DTM in its flat layer, which makes the
standalone DTM table redundant wherever a pack ships. Both are read, with the
pack preferred: material shipping only .lzdtm still answers probe_dtm.

Files are looked up in the wdl/, dtc/, dtm/ and dtm50/ subdirectories of
each search directory, and in the directory itself, so a flat dump of table
files is also probeable.

API

import chess, chess.chesstb

with chess.chesstb.open_tablebase("/path/to/chesstb") as tb:
    board = chess.Board("8/8/8/5k2/8/8/1Q6/K7 w - - 0 1")
    tb.probe_wdl(board)        # 2  (+2 win .. -2 loss, like syzygy)
    tb.probe_dtc(board)        # 19 (signed distance-to-conversion)
    tb.probe_dtm(board)        # 19 (signed distance-to-mate, ignoring 50MR)
    tb.probe_dtm50(board)      # (2, 19): rule-true (wdl, plies) at the board's clock

get_wdl / get_dtc / get_dtm are non-raising variants returning a default
(None) when no table is available. probe(board, rule50=0) returns the full
structured result — every metric at once, off a single walk.

The conversion metric is probe_dtc, not probe_dtz. Syzygy bases its cursed
band off 100 — n > 100 is a cursed win whose zeroing move is n or n - 100
plies away — so a DTZ magnitude carries the WDL class along with the count. A
chesstb DTC is a plain distance in every class, the class being WDL's alone.
Borrowing the dtz name for familiarity would invite callers to substitute one
for the other.

Why pure Python

chess.syzygy is pure Python; this follows suit. The module depends only on
python-chess and the standard library:

  • LZMA (DTC / DTM / DTM50 blocks) via stdlib lzma with FORMAT_RAW (the
    C++ side uses the LZMA SDK with props appended at each block tail).
  • LZ4 (WDL blocks) via a small bundled pure-Python LZ4-block decoder
    (~40 lines) supporting the optional LZ4 dictionary. No new dependency.

The position index (symmetry canonicalization, king/pawn slice managers, the
binomial piece-group ranking, the radix-composed board index and the
index-permutation layout) and the probe orchestration (dropped-frame one-ply
minimax reconstruction for shrunk files, the en-passant overlay, and the DTM50
halfmove-clock layer selection) are faithful re-implementations of the C++
src/probe library. Square numbering already matches python-chess exactly
(a1=0 … h8=63), so boards are consumed directly.

DTC and DTM are byte-for-byte twins on disk — as src/probe/dtm_file.cpp says
of its own traits — so they share one reader here, with the magic and the value
decode as the whole of what separates them.

Design notes

  • Mapped, not read. Tables are memory-mapped and decoded a block at a time,
    with one shared LRU holding decoded blocks across every open table against a
    single budget (open_tablebase(..., block_cache_bytes=...)).
  • One transport seam. _TableFile._open_source is the only place the
    transport is decided, and the four file classes are named as class attributes
    on Tablebase, so serving tables from something other than a mapping is four
    subclasses and a _find. Nothing above that seam asks for a span wider than
    one block.
  • Thread-safe. Probes register as readers so close() can wait for them
    before unmapping; table opens are double-checked under a per-kind lock, and
    block decoding is guarded per (color, block) rather than per table.

Validation

Every value is validated bit-for-bit against the reference C++ prober
(tests/probe_fen) by enumerating positions and comparing WDL, DTC, DTM and
DTM50 in lockstep:

  • All 145 shipped ≤5-man materials, ~72k positions at halfmove clock 0 —
    0 mismatches.
  • Layered DTM50 at halfmove clocks 1, 30, 40, 80, 98, 99 across all
    materials (exercising the CONST/SINGLE/DOUBLE/MULTI changepoint state machine,
    the draw-end hint, and recover_mate_at_hmc) — 0 mismatches.
  • Dropped-frame reconstruction: both the symmetric-mirror path (e.g. KRKR)
    and the asymmetric one-ply-minimax derive (109 of 145 materials ship a dropped
    frame) — 0 mismatches.
  • En-passant overlay and color-mirrored material (stronger side is
    Black) — verified against the oracle.
  • The pure-Python LZ4 decoder is cross-checked block-for-block against the
    reference lz4 C library on every shipped WDL table.

The standalone DTM reader was validated the same way, over random positions and
short playouts across the seven fixture materials (which reach en-passant
squares, mates, stalemates and every capture/promotion sub-material), against
probe_fen pointed at the same directories — 0 mismatches in each shape the
table can take on disk:

Table shape how produced positions
frames intact chesstb --builddtm 22,851
dropped frame shrink 22,851
loss-only (wins derive at probe time) transcribe --loss-only 3,453
loss-only and dropped, over shrunk wdl/+dtc/ both 14,187
all four kinds present (pack preferred) 8,640

Tests

ChesstbTestCase in test.py, against table sets for seven materials
(KBK … KRKR) committed under data/chesstb/, shrink-processed as they would
ship.

The logic is proven on the C++ side and by the sweeps above, so these tests
target what is specific to this port rather than re-proving the format: the
four-kind wiring and directory search, the shared DTC/DTM reader and its value
decodes, rejection of malformed files (and release of the mapping a failed open
had taken), the changepoint decoder at a live clock, the block-cache budget,
the mmap lifecycle, the transport seam, and probing concurrently from several
threads.

@noobpwnftw
noobpwnftw force-pushed the add-chesstb-tablebases branch 7 times, most recently from a252d4c to 228bac6 Compare July 1, 2026 01:08
@noobpwnftw
noobpwnftw force-pushed the add-chesstb-tablebases branch 22 times, most recently from 836a63f to 838e047 Compare August 15, 2026 02:34
@noobpwnftw
noobpwnftw force-pushed the add-chesstb-tablebases branch from 838e047 to 661cc2e Compare August 16, 2026 16:17
chesstb tables store WDL, DTC, DTM and DTM50 for endgame material
configurations. This adds a pure-Python reader for the format, shaped like
chess.syzygy and chess.gaviota: open_tablebase() on a directory tree, then
probe() a board for all four values at once, or probe_wdl / probe_dtc /
probe_dtm / probe_dtm50 for one at a time. Conversion is probe_dtc rather
than syzygy's probe_dtz: syzygy bases its cursed band off 100, so a DTZ
magnitude carries the WDL class along with the count, while a DTC is a
plain distance in every class.

The format, as read here: one table per material configuration, split by
side to move and compressed in blocks (LZMA, or LZ4 with an optional
dictionary prefix). Positions are addressed by a combinatorial index over
king symmetry classes and groups of identical pieces. Configurations of
equal strength share a single table, read through a rank-mirrored,
colour-swapped view of the position; one holding an opposing pawn pair may
also ship a smaller frozen-pair ('p') table, which is preferred when
present, and whose index counts only the free pieces. A table may omit one
side to move altogether, in which case its values are recovered by
minimaxing over legal children. En passant is not indexed and is applied as
a virtual-capture overlay at probe time. DTM50 layers its values by halfmove
clock and marks a drawn layer with a hint bit rather than a distance, so the
prober builds both the 256-position prefix index and the hint bitmap the
layout leaves out; its flat layer is also where DTM comes from, so the
standalone DTM table -- a DTC table but for the value decode -- is read only
for material shipping no pack.

Tables are memory-mapped and decoded a block at a time, with a shared LRU
holding decoded blocks across every open table against one budget.
_TableFile._open_source is the single seam where the transport is decided,
and the four table classes are named as class attributes, so serving
tables from something other than a mapping is four subclasses and a _find.
A Tablebase is safe to share between threads: probes register as readers so
close() can wait for them before unmapping, and table opens are
double-checked under a per-kind lock.

Positions with castling rights are rejected with MissingTableError, as
chess.syzygy and chess.gaviota do for the same input; the tables are built
without them. Positions are assumed legal, as they are on the C++ side.

Includes docs, tests, and seven small table sets (KBK..KRKR) as test data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noobpwnftw
noobpwnftw force-pushed the add-chesstb-tablebases branch from 661cc2e to ae81f3b Compare August 16, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant