-
Notifications
You must be signed in to change notification settings - Fork 0
chd format
This page documents the CHD (Compressed Hunks of Data) on-disk format as implemented by CHDSharp. It is based on the MAME 0.288 sources and the reference notes in References/CHDInfo.md.
A CHD file stores a logical disk image split into fixed-size blocks called hunks. Each hunk can be:
- stored uncompressed,
- compressed with one of several codecs,
- deduplicated (a copy of another hunk in the same file — self reference), or
- inherited from a parent CHD (delta/incremental images — parent reference).
A map (one entry per hunk) records how and where each hunk's data is stored. A linked list of metadata blobs (tagged with four-character tags) carries information such as hard-disk geometry, CD track layouts, and laserdisc A/V parameters.
All multibyte values on disk are big-endian. The file starts with the magic tag MComprHD.
| Version | Header size | Highlights |
|---|---|---|
| V1 | 76 bytes | Hard-coded CHS geometry (512-byte sectors); only zlib/raw; MD5 only; no metadata; 8-byte packed map entries. |
| V2 | 80 bytes | Same as V1 plus seclen (bytes per sector) at offset 76. |
| V3 | 120 bytes | Major redesign: logicalbytes, metaoffset, explicit hunkbytes, SHA1 hashes, ZLIB_PLUS (type 2, secondary codec), 16-byte map entries with CRC32. |
| V4 | 108 bytes | Drops MD5; sha1 becomes the combined raw+metadata hash; adds rawsha1 (raw data only); adds A/V compression type 3. |
| V5 | 124 bytes | Current MAME format: up to 4 codecs identified by 4-char tags, mapoffset (map at end of file), unitbytes (sub-hunk granularity for parent refs), CRC16 map entries, compressed (Huffman+RLE) or uncompressed maps. |
[0-7] "MComprHD" magic
[8-11] 124 header length
[12-15] 5 version
[16-19] compressors[0] 4-char codec tag, e.g. 'lzma'
[20-23] compressors[1] e.g. 'zlib'
[24-27] compressors[2] e.g. 'huff'
[28-31] compressors[3] e.g. 'flac' (all zero => uncompressed image)
[32-39] logicalbytes total uncompressed size
[40-47] mapoffset file offset of the map (0 while being written)
[48-55] metaoffset file offset of the first metadata entry (0 = none)
[56-59] hunkbytes size of each hunk
[60-63] unitbytes size of each unit (parent-reference granularity)
[64-83] rawsha1 SHA1 of the raw decompressed data
[84-103] sha1 SHA1 of raw data + checksummed metadata
[104-123] parentsha1 SHA1 of the parent (all zero => standalone)
Chd.CheckHeader() verifies the magic, that the declared header length matches the version (76/80/120/108/124), and that the version is 1–5. The version-specific parsers additionally enforce:
-
hunkbytesis non-zero and ≤ 128 MiB;hunkbytes × totalhunks ≤ 1 TiB(ValidateSizeLimits). - V3/V4: known compression type (0 = none, 1 = zlib, 2 = zlib+), and for type 2 the secondary codec (FLAC CDDA) is initialized.
- V5: each 4-char codec tag must be a known codec (
IsValidCodec), otherwiseChderrinvaliddata. - V1/V2: the obsolete CHS geometry is persisted so CHDSharp can synthesize GDDD metadata (see Metadata).
A packed 64-bit word: top 20 bits = length, low 44 bits = file offset. The hunk is uncompressed iff length == hunkbytes. V1/V2 entries carry no CRC.
[0-7] uint64 offset file offset of data
[8-11] uint32 crc32 CRC-32 of the uncompressed hunk
[12-14] uint24 length compressed length
[15] uint8 flags type (low nibble) | NO_CRC (0x10)
Entry types (flags & 0x0F):
| Value | Type | Meaning |
|---|---|---|
| 0 | invalid | — |
| 1 | compressed | decompress with codec #0 (zlib or secondary for type 6) |
| 2 | uncompressed | raw copy |
| 3 | mini | repeat the 8-byte big-endian pattern offset
|
| 4 | self | copy hunk offset in this file |
| 5 | parent | copy from parent CHD |
| 6 | 2nd compressed | compressed with the secondary codec (FLAC CDDA, V3/V4 ZLIB_PLUS) |
When the NO_CRC flag (0x10) is set, CHDSharp drops the CRC so the hunk is accepted without validation (matching MAME semantics). When present, the CRC32 is verified after decompression.
The map itself is stored compressed at mapoffset:
[0-3] uint32 length compressed byte length
[4-9] UINT48 datastart file offset of the first data block
[10-11] uint16 crc CRC-16 of the *uncompressed* map
[12] uint8 lengthbits bits per compressed-length field
[13] uint8 hunkbits bits per self-reference field
[14] uint8 parentunitbits bits per parent-unit field
[15] uint8 reserved
[16+] Huffman-encoded map entries
Each expanded entry is 12 bytes:
[0] uint8 compression type
[1-3] UINT24 compressed length
[4-9] UINT48 file offset
[10-11] uint16 crc16 CRC-16 (CCITT) of the uncompressed hunk
The map encoding applies RLE to compression types (runs of 3–18 and 19–290 become RLE_SMALL/RLE_LARGE), promotes consecutive self/parent references to SELF_0/1 and PARENT_0/1/PARENT_SELF pseudo-types, Huffman-encodes the type stream, then writes per-entry auxiliary data (lengths, CRC16s, offsets) at fixed bit widths. The whole uncompressed map's CRC16 is verified against the map header.
Encoder quirk (small maps): chdman sizes the bitstream buffer as (8*16 + (12 + max(lengthbits+16, hunkbits, parentunitbits)) * hunkcount) / 8 + 1 bytes including the 16-byte header. For small hunk counts that area is smaller than the actual tree + symbol + auxiliary bits, so MAME's bitstream_out silently drops whole trailing bytes while flush() still counts them in the map's compressed-length field; the dropped positions read back as zeroes. When a dropped byte is nonzero the stored map no longer matches its header CRC16 and the file cannot be re-opened — not even by chdman itself (reproducible with a single-hunk createraw, e.g. -hs 65536 over random data). CHDSharp's encoder replicates the allocation and clipping byte-for-byte whenever chdman's output is well-formed, and falls back to the full bitstream when clipping would corrupt the map.
Used when all four codec slots are zero: each entry is a uint32 hunk index (offset = index × hunkbytes); 0 means take the hunk from the parent (or zero-fill when there is no parent).
| Value | Name | Meaning |
|---|---|---|
| 0–3 | TYPE_0..3 |
compressed with codec #0..#3 |
| 4 | NONE |
uncompressed (length = hunkbytes) |
| 5 | SELF |
copy from hunk offset in this file |
| 6 | PARENT |
copy from parent unit offset (see below) |
| 7–13 | pseudo-types | RLE/promotion encodings used inside the compressed map stream |
A PARENT entry stores a parent-unit index, not a hunk index. With units_in_hunk = hunkbytes / unitbytes:
- aligned reference → read parent hunk
unit / units_in_hunk; - unaligned reference → straddles two parent hunks: CHDSharp stitches the tail of parent hunk N with the head of parent hunk N+1.
| FourCC | Name | Notes |
|---|---|---|
zlib |
Deflate | Raw deflate (no zlib header, -MAX_WBITS). |
lzma |
LZMA | Headerless single-call LZMA; properties fixed at lc=3, lp=0, pb=2; dictionary = hunk size. |
huff |
Huffman | Custom static 8-bit Huffman; tree itself Huffman-encoded. |
flac |
FLAC | Raw FLAC frames, 16-bit, preceded by a 1-byte endianness marker ('L'/'B'). |
zstd |
Zstandard | Single-frame zstd blocks. |
avhu |
A/V Huffman | Laserdisc A/V: FLAC or delta-Huffman audio + delta-RLE Huffman video. |
cdzl |
CD zlib | zlib for sector data + zlib for subcode; ECC/sync regenerated. |
cdlz |
CD LZMA | LZMA for sector data + zlib for subcode; ECC/sync regenerated. |
cdfl |
CD FLAC | FLAC for sector data + zlib for subcode. |
cdzs |
CD Zstd | zstd for sector data + zstd for subcode; ECC/sync regenerated. |
See Codecs for implementation details.
Metadata is a linked list of tagged binary blobs:
[0-3] uint32 metatag 4-char tag ('GDDD', 'CHT2', 'DVD ', ...)
[4] uint8 flags bit 0 (CHD_MDFLAGS_CHECKSUM): included in the combined SHA1
[5-7] UINT24 length payload length
[8-15] uint64 next file offset of the next entry (0 = end)
Standard tags:
| Tag | Meaning |
|---|---|
GDDD |
Hard-disk geometry: CYLS:%d,HEADS:%d,SECS:%d,BPS:%d
|
IDNT |
Raw 512-byte ATA IDENTIFY data |
KEY / CIS
|
Hard-disk key / PCMCIA CIS data |
CHCD |
Legacy CD-ROM metadata (binary track records) |
CHTR |
CD-ROM tracks v1: TRACK:%d TYPE:%s SUBTYPE:%s FRAMES:%d
|
CHT2 |
CD-ROM tracks v2: adds PREGAP, PGTYPE, PGSUB, POSTGAP
|
CHGT / CHGD
|
Legacy / current GD-ROM track metadata |
DVD |
DVD-ROM marker |
AVAV |
A/V metadata: FPS:%d.%06d WIDTH:%d HEIGHT:%d INTERLACED:%d CHANNELS:%d SAMPLERATE:%d
|
AVLD |
Laserdisc VBI frame data |
CHDSharp reads metadata lazily, guards against cyclic chains, caps entries at 1 MiB, and exposes flags. V1/V2 files — which have no metadata section — get a synthesized GDDD entry built from the obsolete header geometry. See Metadata.
| Hash | Scope | Used in |
|---|---|---|
| MD5 | raw data | V1–V3 |
SHA1 (rawsha1) |
raw decompressed data | V3–V5 |
SHA1 (sha1) |
rawsha1 ‖ sorted hashes of checksummed metadata entries |
V4–V5 |
| CRC32 | per-hunk (V3/V4 map) | V3–V4 |
| CRC16 | per-hunk + map (V5) | V5 |
The combined SHA1 is computed as:
sha1 = SHA1( rawsha1 ‖ sorted([ SHA1(metatag ‖ metadata) for each checksummed entry ]) )
CHDSharp's deep verification recomputes all of these; see Verification.
A child CHD stores only hunks that differ from its parent; identical hunks become PARENT references. The child header stores the parent's sha1/md5. When a PARENT entry is read and no parent was supplied, CHDSharp returns Chderrrequiresparent; if the supplied parent's hashes do not match, it returns Chderrinvalidparent.
See Parent/Child CHDs.
[0] V5 header (124 bytes)
[124] compressed hunk data block 0
[...] compressed hunk data block 1
[...] ... more data blocks (variable size)
[...] metadata linked list (if any)
[end-of-file] compressed V5 map (at mapoffset)
CHDSharp
Format & internals
API & usage
Operations
Writing CHDs
Reference