From 850f280acda5e3c4215cb468bb354e0472bec53c Mon Sep 17 00:00:00 2001 From: Chris Bieneman Date: Fri, 17 Jul 2026 11:01:45 -0500 Subject: [PATCH] [Docs] Document DXIL Container and RDAT formats I've had these docs locally for a while and have reviewed and used them to help reason about and understand aspects of the binary formats that DXC generates. These docs are generated almost entirely by LLMs during iterative chat discussions where I asked repeated clarifying quesitons. I have found them sufficiently useful to keep them around, but thought they're probably worth sharing. Assisted-by: Claude Opus 4.6-4.8 --- docs/DxilContainer_Format.md | 573 ++++++++++++++++++++++++++++++++++ docs/RDAT_Format.md | 578 +++++++++++++++++++++++++++++++++++ 2 files changed, 1151 insertions(+) create mode 100644 docs/DxilContainer_Format.md create mode 100644 docs/RDAT_Format.md diff --git a/docs/DxilContainer_Format.md b/docs/DxilContainer_Format.md new file mode 100644 index 0000000000..1b79eedba1 --- /dev/null +++ b/docs/DxilContainer_Format.md @@ -0,0 +1,573 @@ +# DXIL Container Format + +A **DXIL container** is the on-disk/blob format produced by DXC for a compiled +shader or library. It is a small archive of independently-typed **parts**, +wrapping the LLVM bitcode (the actual DXIL program) together with reflection +metadata, signatures, validation data, hashes, debug names, and more. + +The format is intentionally backward-compatible with the legacy **DXBC** +container (its header FourCC is literally `'DXBC'`) so existing tooling can walk +the part table, while the individual parts carry DXIL-specific data. + +This document describes: + +- [What the container is and how it is used](#overview) +- [The container-level structure](#container-level-structure) +- [Alignment, versioning, and hashing](#alignment-versioning-and-hashing) +- [The part catalog](#part-catalog) +- [Part encodings in detail](#part-encodings) +- [Reading a container](#reading-a-container) +- [Appendix: constants](#appendix-constants) + +Primary sources for this description: + +- [include/dxc/DxilContainer/DxilContainer.h](../include/dxc/DxilContainer/DxilContainer.h) — all container/part header structs and helpers +- [include/dxc/DxilContainer/DxilPipelineStateValidation.h](../include/dxc/DxilContainer/DxilPipelineStateValidation.h) — the PSV0 part +- [lib/DxilContainer/DxilContainerAssembler.cpp](../lib/DxilContainer/DxilContainerAssembler.cpp) — the writer that assembles all parts +- [docs/RDAT_Format.md](RDAT_Format.md) — the RDAT part, documented separately + +--- + +## Overview + +### What it is + +A DXIL container is a flat, position-independent blob: + +- A **container header** (`DxilContainerHeader`) with a FourCC, a hash, a + version, the total size, and a part count. +- A **part offset table**: one `uint32_t` per part, each an absolute offset + (from the start of the container) to that part's header. +- A sequence of **parts**, each a `DxilPartHeader` (FourCC + size) followed by + that part's payload. + +Every part is self-describing (FourCC + size), so consumers can skip parts they +do not understand, and new part types can be added without breaking old readers. + +### How it is used + +- **Runtime/driver**: reads the `DXIL` program part to get the bitcode, and + `PSV0`/`RDAT` for pipeline validation and reflection. +- **Reflection APIs** (`IDxcContainerReflection`, `ID3D12ShaderReflection`): + read `STAT`/`RDAT`/signature parts. +- **Debuggers/PIX**: use `ILDB` (debug bitcode), `ILDN` (debug name/PDB + reference), `SRCI`/`PDBI` (sources & compile args), `HASH`. +- **Validation**: `dxil.dll` validates the container's parts for consistency. +- **Tooling** (`dxa`, `dxc -dumpbin`): extract or print individual parts. + +### Which parts appear + +The set of parts depends on the shader model and compile flags. From +`SerializeDxilContainerForModule`, the high-level rules are: + +- **Feature info** (`SFI0`) is always written first. +- **Non-library** targets get signature parts (`ISG1`, `OSG1`, and `PSG1` if a + patch-constant/primitive signature exists) and `PSV0`; a `RTS0` root + signature part if one is present. +- **Library** targets get `VERS` (validator ≥ 1.8) and `RDAT` instead of + `PSV0`/signatures. +- Debug/reflection/hash parts (`ILDB`, `ILDN`, `HASH`, `STAT`, `SRCI`) are + gated by flags and validator version. +- The `DXIL` program part is always written (near the end). +- `PRIV` private data, if supplied, is written last (its size is not + guaranteed aligned). + +--- + +## Container-level structure + +``` ++---------------------------------------------------------------+ +| DxilContainerHeader | +| uint32_t HeaderFourCC = 'DXBC' | +| DxilContainerHash Hash (16 bytes) | +| DxilContainerVersion Version { uint16 Major, Minor } | +| uint32_t ContainerSizeInBytes | +| uint32_t PartCount | ++---------------------------------------------------------------+ +| uint32_t PartOffset[PartCount] (absolute, from container start) | ++---------------------------------------------------------------+ +| Part 0: DxilPartHeader { uint32 PartFourCC, uint32 PartSize }| +| uint8_t PartData[PartSize] | ++---------------------------------------------------------------+ +| Part 1: DxilPartHeader ... | +| ... | ++---------------------------------------------------------------+ +``` + +All structures in `DxilContainer.h` are declared under `#pragma pack(push, 1)`, +so there is **no implicit padding** inside these headers; any alignment is +explicit (see below). All integers are little-endian. + +### `DxilContainerHeader` + +| Offset | Type | Field | Description | +|-------:|------------------------|-----------------------|----------------------------------------------------| +| 0 | `uint32_t` | `HeaderFourCC` | `DFCC_Container` = `'DXBC'` (back-compat with DXBC).| +| 4 | `uint8_t[16]` | `Hash.Digest` | Container hash (see hashing below). | +| 20 | `uint16_t` | `Version.Major` | Container major version (currently `1`). | +| 22 | `uint16_t` | `Version.Minor` | Container minor version (currently `0`). | +| 24 | `uint32_t` | `ContainerSizeInBytes`| Total container size from start of this header. | +| 28 | `uint32_t` | `PartCount` | Number of parts. | +| 32 | `uint32_t[PartCount]` | (part offsets) | Absolute offset of each `DxilPartHeader`. | + +`DxilContainerMaxSize` = `0x80000000` bounds the total size. + +### `DxilPartHeader` + +| Offset | Type | Field | Description | +|-------:|------------|---------------|----------------------------------------------| +| 0 | `uint32_t` | `PartFourCC` | Part type FourCC (`DFCC_*`). | +| 4 | `uint32_t` | `PartSize` | Size of `PartData`, **not** including header.| +| 8 | `uint8_t[PartSize]` | (data) | Part payload. | + +The offset table is written after the container header, and each part follows +the previous one contiguously: + +``` +offset[0] = sizeof(DxilContainerHeader) + PartCount * 4 +offset[i+1] = offset[i] + sizeof(DxilPartHeader) + parts[i].PartSize +``` + +### Access helpers + +`DxilContainer.h` provides inline helpers used throughout the codebase: + +- `GetDxilContainerPart(header, index)` — part header by index (via offset table). +- `GetDxilPartData(part)` — pointer to a part's payload (`part + 1`). +- `GetDxilPartByType(header, fourCC)` — find a part by FourCC. +- `GetDxilProgramHeader(header, fourCC)` — the `DxilProgramHeader` of a program part. +- `IsDxilContainerLike(ptr, length)` / `IsValidDxilContainer(...)` — validation. +- `DxilPartIterator`, `DxilPartIsType` — iterate/filter parts. + +--- + +## Alignment, versioning, and hashing + +### Alignment + +Parts are generally sized to a 4-byte boundary using `PSVALIGN4(x) = (x+3) & ~3`. +Writers pad payloads with zero bytes to reach the aligned size. The main +exception is the trailing `PRIV` private-data part, whose size is caller-defined +and not guaranteed aligned — which is exactly why it is written **last**. + +Whether the container writer aligns parts depends on validator version: +`bUnaligned = (ValVer < 1.7)`. Newer containers are aligned. + +> **Why the writer's output must be byte-exact (validator behavior).** +> Historically, the DXIL validator verified several container parts not by +> checking their *contents* semantically, but by **regenerating the part bytes +> from the module and running a `memcmp`** against the bytes produced by the +> compiler. This is implemented by `VerifyBlobPartMatches` in +> [lib/DxilValidation/DxilContainerValidation.cpp](../lib/DxilValidation/DxilContainerValidation.cpp), +> which reserves a buffer, calls the part's `DxilPartWriter::write()` to +> re-emit the part, and compares the result to the stored blob: +> +> ```cpp +> pWriter->write(pOutputStream); +> if (memcmp(pData, pOutputStream->GetPtr(), Size)) { +> ValCtx.EmitFormatError(ValidationRule::ContainerPartMatches, {pName}); +> return; +> } +> ``` +> +> Because this is a raw byte compare, the compiler's serialization — including +> field ordering, reserved/zero fields, and any **alignment padding** — has to +> be *bit-for-bit identical* to what the validator's writer produces. This is a +> key reason the alignment/padding behavior above is tied to the validator +> version: a container built for validator version *N* must match the byte +> layout that version *N*'s writers expect. +> +> **Has this changed?** Partly. In the current validator this +> regenerate-and-`memcmp` approach is *still used* for: +> - program signatures (`ISG1`/`OSG1`/`PSG1`) via `VerifySignatureMatches`, +> - the feature-info part (`SFI0`) via `VerifyFeatureInfoMatches`, and +> - the runtime-data part (`RDAT`) via `VerifyRDATMatches`. +> +> For these parts the writer output must still be byte-exact. However, PSV0 +> validation was **changed** to a semantic, field-by-field comparison. The +> older `VerifyPSVMatches` regenerated the whole PSV0 blob and `memcmp`'d it; +> the current implementation uses `PSVContentVerifier`, which decodes both the +> stored part and the module's expected values and compares them field by field +> (emitting a human-readable `ContainerContentMatches` diff on mismatch) +> instead of requiring an identical byte image. This change landed in +> DXC in late 2024 (PRs #6859 and #6924), so validators built before then +> validate PSV0 with the strict byte-for-byte `memcmp`, while newer validators +> tolerate byte-level differences (for example padding) as long as the decoded +> content agrees. + +### Container vs. program vs. part versions + +There are several independent version numbers: + +- **Container version** — `DxilContainerHeader.Version` (major/minor, currently + 1.0). +- **Program version** — inside `DxilProgramHeader.ProgramVersion`, encodes the + shader kind and shader-model version: + + ```cpp + ProgramVersion = (ShaderKind << 16) | (Major << 4) | Minor; + // decode: + ShaderKind = (ProgramVersion >> 16) & 0xFFFF; + Major = (ProgramVersion >> 4) & 0xF; + Minor = ProgramVersion & 0xF; + ``` + +- **Validator version** — governs which parts and which part-internal versions + are emitted (e.g. `HASH` needs ≥ 1.5, `VERS` needs ≥ 1.8, unaligned parts for + < 1.7). +- **Per-part versions** — several parts version *additively by size* (PSV0, + RDAT records), so a reader infers the version from the recorded size/stride. + +### Hashing + +`DxilContainerHeader.Hash` is a 16-byte digest. Separately, the `HASH` part +(`DxilShaderHash`) stores a `Flags` field plus a 16-byte MD5 digest computed +over either the final program bitcode or the source-inclusive bitcode +(`DxilShaderHashFlags::IncludesSource` when `-Zss`/`DebugNameDependOnSource`). +A sentinel `PreviewByPassHash` (all `2`s) is used in some preview scenarios. + +--- + +## Part catalog + +| FourCC | `DxilFourCC` | Contents | +|--------|---------------------------------|----------------------------------------------------------------| +| `DXBC` | `DFCC_Container` | Container header FourCC (not a part). | +| `RDEF` | `DFCC_ResourceDef` | Legacy DXBC resource-definition reflection. | +| `ISG1` | `DFCC_InputSignature` | Input signature (`DxilProgramSignature`). | +| `OSG1` | `DFCC_OutputSignature` | Output signature. | +| `PSG1` | `DFCC_PatchConstantSignature` | Patch-constant / primitive signature. | +| `STAT` | `DFCC_ShaderStatistics` | Reflection data (program-header-wrapped reflection bitcode). | +| `ILDB` | `DFCC_ShaderDebugInfoDXIL` | DXIL bitcode **with** debug info. | +| `ILDN` | `DFCC_ShaderDebugName` | Debug name / PDB file reference (`DxilShaderDebugName`). | +| `SFI0` | `DFCC_FeatureInfo` | 64-bit shader feature flags (`DxilShaderFeatureInfo`). | +| `PRIV` | `DFCC_PrivateData` | Arbitrary caller-provided private data. | +| `RTS0` | `DFCC_RootSignature` | Serialized root signature. | +| `DXIL` | `DFCC_DXIL` | The DXIL program (LLVM bitcode via `DxilProgramHeader`). | +| `PSV0` | `DFCC_PipelineStateValidation` | Pipeline State Validation data. | +| `RDAT` | `DFCC_RuntimeData` | Runtime Data reflection (see [RDAT doc](RDAT_Format.md)). | +| `HASH` | `DFCC_ShaderHash` | Shader hash (`DxilShaderHash`). | +| `SRCI` | `DFCC_ShaderSourceInfo` | Source names, contents, and compile args. | +| `PDBI` | `DFCC_ShaderPDBInfo` | PDB info (RDAT-encoded; replaces `SRCI` in the PDB). | +| `VERS` | `DFCC_CompilerVersion` | Compiler version info (`DxilCompilerVersion`). | + +The FourCC value is computed little-endian by `DXIL_FOURCC(c0,c1,c2,c3) = +c0 | c1<<8 | c2<<16 | c3<<24`, i.e. the characters read in order in a hex dump. + +--- + +## Part encodings + +### `DXIL` / `ILDB` — the DXIL program + +Both the program part (`DXIL`) and the debug-info program part (`ILDB`) share +the same layout: a `DxilProgramHeader` followed by LLVM bitcode. + +`DxilProgramHeader`: + +| Offset | Type | Field | Description | +|-------:|--------------------|-----------------|-------------------------------------------------| +| 0 | `uint32_t` | `ProgramVersion`| Shader kind + SM version (see encoding above). | +| 4 | `uint32_t` | `SizeInUint32` | Size in `uint32_t` units, including this header. | +| 8 | `DxilBitcodeHeader`| `BitcodeHeader` | Bitcode-specific sub-header. | + +`DxilBitcodeHeader`: + +| Offset | Type | Field | Description | +|-------:|------------|-----------------|------------------------------------------------| +| 0 | `uint32_t` | `DxilMagic` | `'DXIL'` = `0x4C495844`. | +| 4 | `uint32_t` | `DxilVersion` | DXIL version. | +| 8 | `uint32_t` | `BitcodeOffset` | Offset to LLVM bitcode from start of this header.| +| 12 | `uint32_t` | `BitcodeSize` | Size of the LLVM bitcode. | + +The bitcode itself is standard LLVM bitcode encoding the DXIL module. The +program payload is zero-padded so the part size is a multiple of 4 bytes. +`ILDB` carries the same module *before* debug info is stripped, so it is only +present when debug info is requested. + +### `ISG1` / `OSG1` / `PSG1` — signatures + +Each signature part starts with a `DxilProgramSignature` header and an array of +`DxilProgramSignatureElement`, with semantic-name strings stored after the +elements and referenced by offset. + +`DxilProgramSignature`: + +| Offset | Type | Field | Description | +|-------:|------------|--------------|----------------------------------------------------| +| 0 | `uint32_t` | `ParamCount` | Number of signature elements. | +| 4 | `uint32_t` | `ParamOffset`| Offset (from start of this header) to the element array. | + +`DxilProgramSignatureElement` (exactly `0x20` = 32 bytes, static-asserted): + +| Offset | Type | Field | Description | +|-------:|-----------------------------|----------------|--------------------------------------------------------------| +| 0 | `uint32_t` | `Stream` | Stream index (non-decreasing order). | +| 4 | `uint32_t` | `SemanticName` | Offset to `LPCSTR` from start of `DxilProgramSignature`. | +| 8 | `uint32_t` | `SemanticIndex`| Semantic index. | +| 12 | `DxilProgramSigSemantic` | `SystemValue` | System-value semantic (`uint32_t` enum). | +| 16 | `DxilProgramSigCompType` | `CompType` | Component type (`uint32_t` enum). | +| 20 | `uint32_t` | `Register` | Register (row) index. | +| 24 | `uint8_t` | `Mask` | Column allocation mask. | +| 25 | `uint8_t` | `NeverWrites_Mask` / `AlwaysReads_Mask` | Union; write-never (output) or read-always (input) mask. | +| 26 | `uint16_t` | `Pad` | Padding to align. | +| 28 | `DxilProgramSigMinPrecision`| `MinPrecision` | Minimum precision (`uint32_t` enum). | + +Enums: +- `DxilProgramSigSemantic` mirrors `D3D_NAME` (Position, ClipDistance, Target, + Depth, Barycentrics, ShadingRate, …). +- `DxilProgramSigCompType` (Unknown/UInt32/SInt32/Float32/…/Float64). +- `DxilProgramSigMinPrecision` (Default/Float16/Float2_8/SInt16/UInt16/Any16/Any10). + +### `PSV0` — Pipeline State Validation + +The PSV0 part is a size-versioned, self-describing structure used by the D3D12 +runtime to validate pipeline state without parsing bitcode. It is written/read +by `DxilPipelineStateValidation::ReadOrWrite`. Every fixed-size sub-structure is +preceded by a `uint32_t` recording its size, so newer (larger) versions are +readable by older code, which just ignores trailing bytes. + +Top-level layout, in the order the blocks appear in the blob (from the header +comment in +[DxilPipelineStateValidation.h, lines 899–943](https://github.com/microsoft/DirectXShaderCompiler/blob/main/include/dxc/DxilContainer/DxilPipelineStateValidation.h#L899-L943)): + +| # | Block | Type / size | Present when | Description | +|--:|-------|-------------|--------------|-------------| +| 1 | `PSVRuntimeInfo_size` | `uint32_t` | always | Size of the following record; selects the runtime-info version *N*. | +| 2 | `PSVRuntimeInfoN` | `PSVRuntimeInfo_size` bytes | always | Fixed per-stage runtime info (see the version table below). | +| 3 | `ResourceCount` | `uint32_t` | always | Number of resource-binding records. | +| 4 | `PSVResourceBindInfo_size` | `uint32_t` | `ResourceCount > 0` | Size of each bind-info record; selects its version. | +| 5 | `PSVResourceBindInfoN` × `ResourceCount` | array | `ResourceCount > 0` | Resource-binding records. | +| 6 | `StringTableSize` | `uint32_t` (dword-aligned) | version ≥ 1 | Size of the string pool in bytes. | +| 7 | String table | `StringTableSize` bytes, null-terminated | version ≥ 1 and `StringTableSize > 0` | Semantic-name pool. | +| 8 | `SemanticIndexTableEntries` | `uint32_t` | version ≥ 1 | Number of dwords in the semantic-index table. | +| 9 | Semantic-index table | `uint32_t` × `SemanticIndexTableEntries` | `SemanticIndexTableEntries > 0` | Semantic-index values. | +| 10 | `PSVSignatureElement_size` | `uint32_t` | any of `SigInputElements` / `SigOutputElements` / `SigPatchConstOrPrimElements` | Size of each signature-element record. | +| 11 | Input signature elements | `PSVSignatureElementN` × `SigInputElements` | as row 10 | Input signature. | +| 12 | Output signature elements | `PSVSignatureElementN` × `SigOutputElements` | as row 10 | Output signature. | +| 13 | Patch-const/prim elements | `PSVSignatureElementN` × `SigPatchConstOrPrimElements` | as row 10 | Patch-constant / primitive signature. | +| 14 | ViewID→output masks | per stream *i* (0–3): `uint32_t` × `MaskDwords(SigOutputVectors[i])` | `UsesViewID` and `SigOutputVectors[i] != 0` | Outputs affected by ViewID, as a bitmask. | +| 15 | ViewID→patch-const masks | `uint32_t` × `MaskDwords(SigPatchConstOrPrimVectors)` | `UsesViewID` and HS and `SigPatchConstOrPrimVectors != 0` | Patch-const outputs affected by ViewID. | +| 16 | Input→output tables | per stream *i* (0–3): `uint32_t` × `InputOutputTableDwords(SigInputVectors, SigOutputVectors[i])` | `SigInputVectors` and `SigOutputVectors[i] != 0` | Outputs affected by inputs, as a table of bitmasks. | +| 17 | Input→patch-const table | `uint32_t` × `InputOutputTableDwords(SigInputVectors, SigPatchConstOrPrimVectors)` | HS and `SigPatchConstOrPrimVectors` and `SigInputVectors` | Patch-const outputs affected by inputs. | +| 18 | Patch-const→output table | `uint32_t` × `InputOutputTableDwords(SigPatchConstOrPrimVectors, SigOutputVectors[0])` | DS and `SigOutputVectors[0]` and `SigPatchConstOrPrimVectors` | Outputs affected by patch-constant inputs. | + +Where the dword-count helpers are: + +- `MaskDwords(vectors) = (vectors + 7) >> 3` (one bit per component, 4 + components per vector). +- `InputOutputTableDwords(in, out) = MaskDwords(out) * in * 4`. + +**`PSVRuntimeInfo0`** (base, `MAX_PSV_VERSION = 4`) begins with a union of +per-stage info (`VSInfo`/`HSInfo`/`DSInfo`/`GSInfo`/`PSInfo`/`MSInfo`/`ASInfo`) +followed by `MinimumExpectedWaveLaneCount`/`MaximumExpectedWaveLaneCount`. +Successive versions append fields: + +- **`PSVRuntimeInfo1`**: `ShaderStage` (`PSVShaderKind`), `UsesViewID`, a union + (`MaxVertexCount` for GS / `SigPatchConstOrPrimVectors` / `MSInfo1`), signature + element counts (`SigInputElements`, `SigOutputElements`, + `SigPatchConstOrPrimElements`), and packed vector counts (`SigInputVectors`, + `SigOutputVectors[4]` for GS streams). +- **`PSVRuntimeInfo2`**: `NumThreadsX/Y/Z`. +- **`PSVRuntimeInfo3`**: `EntryFunctionName` (offset into the string table). +- **`PSVRuntimeInfo4`**: `NumBytesGroupSharedMemory`. + +**`PSVResourceBindInfo0`**: `ResType` (`PSVResourceType`), `Space`, `LowerBound`, +`UpperBound`. **`PSVResourceBindInfo1`** adds `ResKind` (`PSVResourceKind`) and +`ResFlags` (`PSVResourceFlag`, e.g. `UsedByAtomic64`). + +**`PSVSignatureElement0`** (referenced via the string/semantic-index tables): + +| Offset | Type | Field | Description | +|-------:|------------|-----------------------|---------------------------------------------------------| +| 0 | `uint32_t` | `SemanticName` | Offset into PSV string table. | +| 4 | `uint32_t` | `SemanticIndexes` | Offset into semantic-index table; count == `Rows`. | +| 8 | `uint8_t` | `Rows` | Rows occupied. | +| 9 | `uint8_t` | `StartRow` | Packing start row (if allocated). | +| 10 | `uint8_t` | `ColsAndStart` | bits 0:4 `Cols`, 4:6 `StartCol`, bit 6 `Allocated`. | +| 11 | `uint8_t` | `SemanticKind` | `PSVSemanticKind`. | +| 12 | `uint8_t` | `ComponentType` | `DxilProgramSigCompType`. | +| 13 | `uint8_t` | `InterpolationMode` | `DXIL::InterpolationMode`. | +| 14 | `uint8_t` | `DynamicMaskAndStream`| bits 0:4 dynamic-index mask, 4:6 output stream. | +| 15 | `uint8_t` | `Reserved` | Reserved. | + +The PSV string table is dword-aligned with a trailing null; the semantic-index +table is an array of `uint32_t`. ViewID and input→output dependency data are +stored as packed component-mask bitfields (see helpers above). + +### `SFI0` — feature info + +`DxilShaderFeatureInfo` is a single `uint64_t FeatureFlags`. The bit meanings +are the shader feature flags (Doubles, WaveOps, Int64Ops, ViewID, Barycentrics, +Raytracing Tier 1.1, ResourceDescriptorHeapIndexing, etc.), matching the +`DxilFeatureInfo1`/`DxilFeatureInfo2` enumerations documented in the +[RDAT doc](RDAT_Format.md) (there the same 64-bit value is split into two 32-bit +halves). + +### `HASH` — shader hash + +`DxilShaderHash`: + +| Offset | Type | Field | Description | +|-------:|---------------|----------|------------------------------------------------| +| 0 | `uint32_t` | `Flags` | `DxilShaderHashFlags` (`IncludesSource` = 1). | +| 4 | `uint8_t[16]` | `Digest` | 16-byte MD5 digest. | + +Written only when validator version ≥ 1.5. + +### `ILDN` — shader debug name + +`DxilShaderDebugName` followed by the name bytes: + +| Offset | Type | Field | Description | +|-------:|------------|--------------|------------------------------------------------| +| 0 | `uint16_t` | `Flags` | Reserved, must be 0. | +| 2 | `uint16_t` | `NameLength` | Debug name length, excluding null terminator. | +| 4 | `char[NameLength]` | (name) | UTF-8 name. | +| … | `char` | (null) | Null terminator. | +| … | `char[0-3]`| (pad) | Zero padding to a 4-byte boundary. | + +`MinDxilShaderDebugNameSize = sizeof(DxilShaderDebugName) + 4`. This part +typically names the associated `.pdb` (default: `.pdb`). + +### `VERS` — compiler version + +`DxilCompilerVersion`: + +| Offset | Type | Field | Description | +|-------:|------------|----------------------------------|----------------------------------------------| +| 0 | `uint16_t` | `Major` | Compiler major version. | +| 2 | `uint16_t` | `Minor` | Compiler minor version. | +| 4 | `uint32_t` | `VersionFlags` | Flags. | +| 8 | `uint32_t` | `CommitCount` | Commit count. | +| 12 | `uint32_t` | `VersionStringListSizeInBytes` | Size of the string list that follows. | +| 16 | `char[...]`| (strings) | Up to two null-terminated strings: commit SHA, then custom version string. Zero-padded to 4 bytes. | + +Written for library targets when validator version ≥ 1.8 and version info is +available. + +### `SRCI` — shader source info + +A `DxilSourceInfo` header followed by `SectionCount` sections, each a +`DxilSourceInfoSection` header plus a type-specific, 4-byte-aligned payload. + +`DxilSourceInfo`: + +| Offset | Type | Field | Description | +|-------:|------------|----------------------|----------------------------------------------| +| 0 | `uint32_t` | `AlignedSizeInBytes` | Total size including this header. | +| 4 | `uint16_t` | `Flags` | Reserved, 0. | +| 6 | `uint16_t` | `SectionCount` | Number of sections. | + +`DxilSourceInfoSection`: + +| Offset | Type | Field | Description | +|-------:|-----------------------------|----------------------|------------------------------------------| +| 0 | `uint32_t` | `AlignedSizeInBytes` | Section size including header + padding. | +| 4 | `uint16_t` | `Flags` | Reserved, 0. | +| 6 | `DxilSourceInfoSectionType` | `Type` | `SourceContents` / `SourceNames` / `Args`.| + +Section payloads: + +- **Source names** (`DxilSourceInfo_SourceNames` + `..._SourceNamesEntry[]`): + `Count`, `EntriesSizeInBytes`, then per-entry `AlignedSizeInBytes`, `Flags`, + `NameSizeInBytes`, `ContentSizeInBytes`, followed by the UTF-8 name (null + terminated, padded to 4 bytes). +- **Source contents** (`DxilSourceInfo_SourceContents` + + `..._SourceContentsEntry[]`): may be **zlib-compressed** (`CompressType`), with + `EntriesSizeInBytes` (compressed) and `UncompressedEntriesSizeInBytes`; each + uncompressed entry is `AlignedSizeInBytes`, `Flags`, `ContentSizeInBytes`, then + the content (null terminated, padded). +- **Args** (`DxilSourceInfo_Args`): `Flags`, `SizeInBytes`, `Count`, then + `Count` argument pairs encoded as `Name\0Value\0` (e.g. `T\0ps_6_0\0`, + `D\0MyDefine=1\0`, `Zi\0\0`). + +### `PDBI` — shader PDB info + +`DxilShaderPDBInfo` header followed by (optionally zlib-compressed) data that is +itself **RDAT-encoded** (see `RDAT_PdbInfoTypes.inl` and the +[RDAT doc](RDAT_Format.md)). It supersedes `SRCI` inside PDB files. + +| Offset | Type | Field | Description | +|-------:|-----------------------------------|---------------------------|------------------------------------| +| 0 | `DxilShaderPDBInfoVersion` (`uint16_t`) | `Version` | `Version_0` currently. | +| 2 | `DxilShaderPDBInfoCompressionType` (`uint16_t`) | `CompressionType` | `Uncompressed` or `Zlib`. | +| 4 | `uint32_t` | `SizeInBytes` | Size of the (possibly compressed) data. | +| 8 | `uint32_t` | `UncompressedSizeInBytes` | Uncompressed size. | + +### `RTS0` — root signature + +A serialized root signature blob (the same format D3D12 consumes via +`D3D12SerializeRootSignature`). It can be embedded in the container and/or +emitted standalone (wrapped in its own single-part container via +`SerializeDxilContainerForRootSignature`). For non-library targets, DXC copies +the module's serialized root signature into this part. + +### `STAT` — shader statistics / reflection + +A program part (same `DxilProgramHeader` wrapper as `DXIL`) whose bitcode is a +**reflection-only** clone of the module produced by +`StripAndCreateReflectionStream`. Reflection APIs read this to expose full +reflection for both shaders and libraries. For libraries, an `RDAT` part is +appended alongside `STAT` in the separate reflection stream. + +### `PRIV` — private data + +Opaque, caller-provided bytes copied verbatim. Written **last** because its size +is not guaranteed to be 4-byte aligned, so it must not perturb the alignment of +following parts (there are none after it). + +### `RDEF` — resource definitions (legacy) + +The legacy DXBC `RDEF` reflection part is recognized by the FourCC enumeration +for back-compat with DXBC tooling. DXIL reflection is primarily carried by +`STAT`/`RDAT`; see those parts and [RDAT_Format.md](RDAT_Format.md). + +--- + +## Reading a container + +A typical consumer: + +1. Validate with `IsDxilContainerLike(ptr, length)` (checks the `'DXBC'` FourCC + and minimum length) and, for stronger guarantees, `IsValidDxilContainer`. +2. Read `PartCount` and the offset table. +3. For each index, `GetDxilContainerPart` returns the `DxilPartHeader`; switch on + `PartFourCC`, or use `GetDxilPartByType` to jump to a specific part. +4. Use `GetDxilPartData` to access a part's payload, bounded by `PartSize`. +5. For program parts, `GetDxilProgramHeader` yields the `DxilProgramHeader`; the + bitcode is at `BitcodeHeader.BitcodeOffset` for `BitcodeHeader.BitcodeSize` + bytes, and `GetVersionShaderType`/`GetVersionMajor`/`GetVersionMinor` decode + the shader model. + +Unknown FourCCs should be skipped — this is what makes the format extensible. + +--- + +## Appendix: constants + +| Name | Value | Meaning | +|------------------------------|------------------|---------------------------------------------| +| `DFCC_Container` | `'DXBC'` | Container header FourCC. | +| `DxilContainerVersionMajor` | `1` | Current container major version. | +| `DxilContainerVersionMinor` | `0` | Current container minor version. | +| `DxilContainerMaxSize` | `0x80000000` | Maximum container size. | +| `DxilContainerHashSize` | `16` | Container/hash digest length in bytes. | +| `DxilMagicValue` | `0x4C495844` | `'DXIL'` bitcode magic. | +| `MinDxilShaderDebugNameSize` | `sizeof(DxilShaderDebugName)+4` | Minimum `ILDN` part size. | +| `MAX_PSV_VERSION` | `4` | Highest PSV runtime-info version. | +| `PSVALIGN4(x)` | `(x+3)&~3` | 4-byte alignment used across parts. | +| `PreviewByPassHash` | all `2`s | Sentinel container hash for preview. | + +### Program version encoding + +``` +ProgramVersion = (ShaderKind << 16) | (Major << 4) | Minor +``` + +### FourCC encoding + +``` +DXIL_FOURCC(c0, c1, c2, c3) = c0 | (c1 << 8) | (c2 << 16) | (c3 << 24) +``` diff --git a/docs/RDAT_Format.md b/docs/RDAT_Format.md new file mode 100644 index 0000000000..b2440e7efb --- /dev/null +++ b/docs/RDAT_Format.md @@ -0,0 +1,578 @@ +# DXIL Runtime Data (RDAT) Format + +RDAT ("Runtime Data") is a compact, self-describing binary blob emitted by DXC +that carries reflection-style metadata about a compiled DXIL library or shader. +It is designed to be consumed at runtime (by a driver/runtime) without needing +to parse the LLVM bitcode, and it is engineered for forward and backward +compatibility so that newer compilers can add data that older consumers safely +ignore, and newer consumers can read older blobs. + +This document describes: + +- [What RDAT is and how it is used](#overview) +- [The high-level structure](#high-level-structure) +- [The binary encoding in detail](#binary-encoding) +- [The part types](#part-types) +- [The record/field reference model](#record-and-field-reference-model) +- [Versioning and compatibility](#versioning-and-compatibility) +- [The record schemas](#record-schemas) +- [Reading and validation](#reading-and-validation) + +The primary sources for this description are: + +- [include/dxc/DxilContainer/DxilRuntimeReflection.h](../include/dxc/DxilContainer/DxilRuntimeReflection.h) — readers, headers, context +- [include/dxc/DxilContainer/DxilRuntimeReflection.inl](../include/dxc/DxilContainer/DxilRuntimeReflection.inl) — parsing/validation implementation +- [include/dxc/DxilContainer/RDAT_Macros.inl](../include/dxc/DxilContainer/RDAT_Macros.inl) — the X-macro metaprogramming that defines all record layouts +- [include/dxc/DxilContainer/RDAT_LibraryTypes.inl](../include/dxc/DxilContainer/RDAT_LibraryTypes.inl), [RDAT_SubobjectTypes.inl](../include/dxc/DxilContainer/RDAT_SubobjectTypes.inl), [RDAT_PdbInfoTypes.inl](../include/dxc/DxilContainer/RDAT_PdbInfoTypes.inl) — the actual record definitions +- [include/dxc/DxilContainer/DxilRDATBuilder.h](../include/dxc/DxilContainer/DxilRDATBuilder.h) and [lib/DxilContainer/DxilRDATBuilder.cpp](../lib/DxilContainer/DxilRDATBuilder.cpp) — the writer +- [include/dxc/DxilContainer/DxilRDATParts.h](../include/dxc/DxilContainer/DxilRDATParts.h) — the part builders + +--- + +## Overview + +### Where RDAT lives + +RDAT is stored as a single part inside a DXIL container, identified by the +FourCC `DFCC_RuntimeData` = `'RDAT'` (see +[DxilContainer.h](../include/dxc/DxilContainer/DxilContainer.h)). The DXIL +container writer adds it via: + +```cpp +writer.AddPart(DFCC_RuntimeData, pRDATWriter->size(), + [&](AbstractMemoryStream *pStream) { pRDATWriter->write(pStream); }); +``` + +Within that container part, RDAT is *itself* a mini-container with its own +header, offset table, and sub-parts. The comment on `DxilRDATBuilder` +summarizes it well: + +> Like DXIL container, RDAT itself is a mini container that contains multiple +> RDAT parts. + +### What it contains + +RDAT primarily describes, for a library or shader: + +- **Resources** used (SRV/UAV/CBuffer/Sampler), including register binding, + space, kind and resource flags. +- **Functions / entry points**: name, unmangled name, shader kind, resources + referenced, functions called (dependencies), feature/shader-stage flags, + minimum shader target, payload/attribute sizes, and (v1.8+) stage-specific + information. +- **Subobjects** (RDAT v1.4+): state object config, root signatures, hit + groups, raytracing shader/pipeline configs, associations. +- **Node shader information** (v1.8+): launch type, node I/O, node attributes + for work graphs. +- **Per-stage signature/pipeline info** (experimental): VS/PS/HS/DS/GS/CS/MS/AS + info, signature elements, ViewID masks, thread group sizes, etc. +- **PDB info** (a separate group): embedded sources, libraries, compile + arguments, hashes. + +### How it is used + +- **Runtime/driver reflection**: A runtime can load the blob with + `DxilRuntimeData` and query typed record tables without parsing bitcode. +- **Subobject loading**: `LoadSubobjectsFromRDAT` + ([RDATDxilSubobjects.cpp](../lib/DxilContainer/RDATDxilSubobjects.cpp)) + reconstructs `DxilSubobjects` from the subobject table for RDXR/state object + creation. +- **Validation**: The validator re-checks the structure; `DxilRuntimeData::Validate()` + bounds-checks every reference. +- **Tooling/debug**: `RDATDumper` ([RDATDumper.cpp](../lib/DxilContainer/RDATDumper.cpp)) + produces a human-readable dump. + +--- + +## High-level structure + +An RDAT blob is laid out as follows: + +``` ++-------------------------------------------------------------+ +| RuntimeDataHeader | 8 bytes +| uint32_t Version | +| uint32_t PartCount | ++-------------------------------------------------------------+ +| uint32_t PartOffsets[PartCount] | 4 * PartCount bytes +| (each offset is relative to the start of the header) | ++-------------------------------------------------------------+ +| Part 0 | +| RuntimeDataPartHeader { Type, Size } | 8 bytes +| byte Data[ALIGN4(Size)] | ++-------------------------------------------------------------+ +| Part 1 ... | +| ... | ++-------------------------------------------------------------+ +``` + +Each part is one of a small set of "kinds": the **string buffer**, the **index +array buffer**, the **raw bytes buffer**, or a **record table** (many of these, +one per record type). Records inside tables never store variable-length data +inline; instead they store 4-byte *offsets/indices* into the string, index, or +raw-bytes parts. This is what keeps records fixed-size and strided, enabling +forward/backward compatibility. + +```mermaid +graph TD + Container["DXIL Container part: 'RDAT'"] --> Hdr["RuntimeDataHeader + offsets[]"] + Hdr --> P0["StringBuffer part"] + Hdr --> P1["IndexArrays part"] + Hdr --> P2["RawBytes part"] + Hdr --> P3["ResourceTable"] + Hdr --> P4["FunctionTable"] + Hdr --> P5["SubobjectTable / ... more tables"] + P4 -- "RDATString offset" --> P0 + P4 -- "RecordArrayRef / IndexArrayRef" --> P1 + P3 -- "record indices via index arrays" --> P1 + P5 -- "BytesRef offset+size" --> P2 +``` + +--- + +## Binary encoding + +All multi-byte integers are little-endian (matching the rest of the DXIL +container format and the target platforms). All sizes and offsets described as +"4-byte aligned" are padded up with `PSVALIGN4(x) = (x + 3) & ~3`. + +### `RuntimeDataHeader` + +Defined in [DxilRuntimeReflection.h](../include/dxc/DxilContainer/DxilRuntimeReflection.h). + +| Offset | Type | Field | Description | +|-------:|------------|------------|------------------------------------------------------| +| 0 | `uint32_t` | `Version` | RDAT format version. Currently `RDAT_Version_10 = 0x10`. | +| 4 | `uint32_t` | `PartCount`| Number of parts that follow. | +| 8 | `uint32_t[PartCount]` | (offsets) | Offset of each part, **relative to the start of this header**. Offsets are 4-byte aligned. | + +Size: `8 + 4 * PartCount` bytes. The value `0x10` for the version was chosen so +it "cannot be mistaken for a part count from the prerelease version" — old blobs +started with a raw part count, so `0x10` disambiguates. + +### `RuntimeDataPartHeader` + +Each part begins (at its offset) with: + +| Offset | Type | Field | Description | +|-------:|------------------------|--------|--------------------------------------------------------| +| 0 | `RuntimeDataPartType` (`uint32_t`) | `Type` | Identifies which kind of part this is. | +| 4 | `uint32_t` | `Size` | Size of the part data **not** including this header. Must be 4-byte aligned. | +| 8 | `byte[ALIGN4(Size)]` | (data) | Part payload. | + +The writer always stores `Size = PSVALIGN4(actualPartSize)`, and empty parts +(size 0) are omitted entirely (see `ComputeSize` / `FinalizeAndGetData` in +[DxilRDATBuilder.cpp](../lib/DxilContainer/DxilRDATBuilder.cpp)). + +### Overall size computation + +From `DxilRDATBuilder::ComputeSize`: + +``` +total = sizeof(RuntimeDataHeader) // 8 + + numNonEmptyParts * sizeof(uint32_t) // offset table + + Σ (sizeof(RuntimeDataPartHeader) + PSVALIGN4(part.GetPartSize())) +``` + +Only non-empty parts contribute; the header's `PartCount` equals +`numNonEmptyParts`. + +### Writing order + +The writer instantiates parts in the specific order the validator expects +(from `DxilRDATWriter`'s constructor): + +1. `StringBuffer` +2. `ResourceTable` +3. `FunctionTable` +4. `IndexArrays` +5. `RawBytes` +6. `SubobjectTable` (if allowed by validator version) +7. Remaining tables in declaration order (only those `<= maxAllowedType`) + +Ordering matters only for producing a byte-identical, validator-friendly layout; +readers locate parts by `Type` via the offset table, not by position. + +--- + +## Part types + +`RuntimeDataPartType` (a `uint32_t` enum) enumerates every part kind. The +non-table kinds are the shared buffers; the rest are record tables. + +| Value | Name | Introduced | Notes | +|-------:|-----------------------------|:----------:|------------------------------------------| +| 0 | `Invalid` | — | | +| 1 | `StringBuffer` | 1.3 | Shared UTF-8 string pool | +| 2 | `IndexArrays` | 1.3 | Shared array-of-uint32 pool | +| 3 | `ResourceTable` | 1.3 | `RuntimeDataResourceInfo` records | +| 4 | `FunctionTable` | 1.3 | `RuntimeDataFunctionInfo[2]` records; `Last_1_3` | +| 5 | `RawBytes` | 1.4 | Shared raw byte pool | +| 6 | `SubobjectTable` | 1.4 | `Last_1_4` | +| 7 | `NodeIDTable` | 1.8 | | +| 8 | `NodeShaderIOAttribTable` | 1.8 | | +| 9 | `NodeShaderFuncAttribTable` | 1.8 | | +| 10 | `IONodeTable` | 1.8 | | +| 11 | `NodeShaderInfoTable` | 1.8 | `Last_1_8` | +| 12 | `Reserved_MeshNodesPreviewInfoTable` | — | reserved | +| 13+ | `SignatureElementTable`, `VSInfoTable`, `PSInfoTable`, `HSInfoTable`, `DSInfoTable`, `GSInfoTable`, `CSInfoTable`, `MSInfoTable`, `ASInfoTable` | experimental | **experimental** | + +The PDB-info group uses a separate namespace via `RDAT_PART_ID_WITH_GROUP`, +which packs a `RuntimeDataGroup` into the high 16 bits: + +```cpp +RDAT_PART_ID_WITH_GROUP(group, id) = ((group << 16) | (id & 0xFFFF)) +``` + +- Group `Core = 0` — all the parts above. +- Group `PdbInfo = 1` — `DxilPdbInfoTable`, `DxilPdbInfoSourceTable`, + `DxilPdbInfoLibraryTable`. + +`RecordTableIndex` is a parallel dense enum used to index the reader's internal +`Tables[]` array; note that its ordering is **not** the same as the numeric +`RuntimeDataPartType` values (e.g. PDB tables are grouped in the middle of +`RecordTableIndex` but have high `RuntimeDataPartType` values). + +### StringBuffer part + +Type `1`. A flat blob of UTF-8 bytes. The first byte is always `'\0'` so that +offset `0` is a valid empty/null string: + +```cpp +StringBufferPart() { Insert(""); } // offset 0 == "" +``` + +- References into it are byte **offsets** (`RDATString`, `RDATStringArray`). +- Strings are null-terminated; the reader simply returns `table + offset`. +- Deduplicated on insert via an `unordered_map`. +- Validation requires the last byte to be `0`. + +### IndexArrays part + +Type `2`. A flat array of `uint32_t`. It stores *many* variable-length index +arrays back to back. Each array is encoded as: + +``` +[ count ][ value_0 ][ value_1 ] ... [ value_{count-1} ] +``` + +- A reference (`IndexArrayRef`, and the array-of-refs used by + `RecordArrayRef`/`RDATStringArray`) is the `uint32_t` **element index** of the + `count` word. +- `IndexTableReader::getRow(i)` returns a row starting at `table[i]` with length + `table[i]`, i.e. it reads the count then exposes the following `count` values. +- Arrays are deduplicated: `IndexArraysPart::AddIndex` appends the new array, + then uses an ordered `std::set` with a custom comparator (`CmpIndices`) that + compares count-then-elements; on a duplicate it rolls back the append and + returns the pre-existing offset. +- Part size in bytes = `4 * number_of_uint32_words`. + +This one pool is used both for numeric arrays (e.g. `NumThreads`, +`SemanticIndices`, `DispatchGrid`) and for arrays of record indices +(`RecordArrayRef`) and arrays of string offsets (`RDATStringArray`). + +### RawBytes part + +Type `5`. A flat blob of arbitrary bytes for binary payloads (root signature +blobs, ViewID masks, PDB data, hashes). + +- Referenced by `BytesRef { uint32_t Offset; uint32_t Size; }` — an 8-byte + field embedded in records. +- Deduplicated by content via `unordered_map`. +- Note: unlike the string buffer, there is no implicit leading entry and no + null terminator; `Size` fully bounds the data. + +### Record tables + +Types `3, 4, 6, 7…`. Each table part begins with a `RuntimeDataTableHeader` +followed by `RecordCount` fixed-size records: + +| Offset | Type | Field | Description | +|-------:|------------|---------------|----------------------------------------------| +| 0 | `uint32_t` | `RecordCount` | Number of records. | +| 4 | `uint32_t` | `RecordStride`| Size of each record in bytes; 4-byte aligned.| +| 8 | `byte[RecordCount * RecordStride]` | (records) | Packed records. | + +Key properties: + +- **Strided for extensibility.** `RecordStride` is stored per table so a newer + compiler can grow a record type (append fields) and older readers still walk + the table correctly. Readers only read fields that fit within both the + compiled struct size *and* the stored stride; `TableReader::Row` returns + `nullptr` if `sizeof(T) > stride`. +- **Deduplication.** When enabled (`m_bDeduplicationEnabled`), identical record + byte-images alias to the same index via an `unordered_map` + keyed on the raw record bytes. Whether dedup is allowed depends on the module + (`GetRecordDuplicationAllowed`). +- **Records contain only fixed-size fields**: scalars, enums (stored as their + underlying integer), fixed arrays, and 4-byte reference handles into the + shared pools (or the 8-byte `BytesRef`). + +`GetPartSize()` returns `0` for an empty table (so it is omitted), otherwise +`sizeof(RuntimeDataTableHeader) + RecordCount * RecordStride`. + +--- + +## Record and field reference model + +Records are **plain-old-data** structs generated by the X-macro system in +[RDAT_Macros.inl](../include/dxc/DxilContainer/RDAT_Macros.inl). The same +`.inl` definitions are expanded in multiple modes (`DEF_RDAT_TYPES_BASIC_STRUCT`, +`DEF_RDAT_TYPES_USE_HELPERS`, reader decl/impl, traits, validation, dump) so the +layout, readers, validators, and dumpers all stay in lockstep. + +Every field maps to one of these on-disk encodings: + +| Macro | On-disk size | Encoding | +|-------------------------------|-------------:|----------------------------------------------------------------| +| `RDAT_VALUE(type, name)` | `sizeof(type)` | Raw scalar (`uint8/16/32`, etc.). | +| `RDAT_VALUE_HEX` | `sizeof(type)` | Same; hint for dumping in hex. | +| `RDAT_ENUM(sTy, eTy, name)` | `sizeof(sTy)` | Enum stored in storage type `sTy` (e.g. `uint32_t`/`uint8_t`).| +| `RDAT_FLAGS(sTy, eTy, name)` | `sizeof(sTy)` | Bit flags stored in `sTy`. | +| `RDAT_STRING(name)` | 4 bytes | `RDATString` = byte offset into StringBuffer. | +| `RDAT_STRING_ARRAY_REF(name)` | 4 bytes | `RDATStringArray` = index-array offset; array of string offsets.| +| `RDAT_INDEX_ARRAY_REF(name)` | 4 bytes | `IndexArrayRef` = index-array offset; array of `uint32_t`. | +| `RDAT_RECORD_REF(type, name)` | 4 bytes | `RecordRef` = record index into that type's table. | +| `RDAT_RECORD_ARRAY_REF(type, name)` | 4 bytes | `RecordArrayRef` = index-array offset; array of record indices. | +| `RDAT_RECORD_VALUE(type, name)` | `sizeof(type)` | Nested record embedded inline (by value). | +| `RDAT_BYTES(name)` | 8 bytes | `BytesRef` = `{ uint32_t Offset; uint32_t Size; }` into RawBytes. | +| `RDAT_ARRAY_VALUE(type,count,...)` | `count*sizeof(type)` | Fixed-size inline array. | + +The sentinel `RDAT_NULL_REF = 0xFFFFFFFF` denotes a null reference for +record/index/string references. + +### Alignment guidance + +From the header comment in `RDAT_Macros.inl`: + +> Pay attention to alignment when organizing structures. `RDAT_STRING` and +> `*_REF` types are always 4 bytes. `RDAT_BYTES` is 2 × 4 bytes. + +Because `RecordStride` must be 4-byte aligned and records are packed with C +struct layout, field order in the `.inl` definitions is chosen to avoid internal +padding surprises. `uint8_t`/`uint16_t` fields are grouped so the record packs +cleanly. + +### Unions + +Records may contain a `union` (`RDAT_UNION` … `RDAT_UNION_END`) whose active +member is selected by another field (typically a `Kind`/`AttribKind` enum). The +`RDAT_UNION_IF`/`RDAT_UNION_ELIF` predicates (e.g. +`getShaderKind() == ShaderKind::Node`) drive the reader accessors, validation, +and dumping so only the active member is interpreted. On disk this is just a +fixed-size union occupying the size of its largest member. + +--- + +## Versioning and compatibility + +RDAT is explicitly built for mixed producer/consumer versions: + +- **Blob version** — `RuntimeDataHeader.Version` (`0x10`). Readers reject + anything below `RDAT_Version_10`. +- **Validator-version gating of parts** — `MaxPartTypeForValVer(Major, Minor, IsPrerelease)` + determines which part types a given validator version may emit: + - `< 1.3` → `Invalid` (no RDAT at all) + - `< 1.4` → up to `Last_1_3` (`FunctionTable`) + - `< 1.8` → up to `Last_1_4` (`SubobjectTable`) + - experimental/prerelease shader model or unbound validator → up to + `LastExperimental` + - otherwise → `LastRelease` (`Last_1_8`) + + The writer only instantiates tables whose `PartType() <= maxAllowedType`. +- **Record growth via stride** — Because each table records its `RecordStride`, + a record type can be *extended* by appending fields. Two mechanisms cooperate: + - **Derived records**: `RDAT_STRUCT_TABLE_DERIVED(type, base, table, Major, Minor)` + defines a record that inherits a base and adds fields, sharing the base's + table (e.g. `RuntimeDataFunctionInfo2` derives from + `RuntimeDataFunctionInfo` and requires validator 1.8). The table's stride is + bumped to the largest supported derived type. Readers upcast based on stride. + - A static assert enforces the core assumption: a derived record type must be + strictly larger than its base (`sizeof(derived) > sizeof(base)`), so stride + comparisons unambiguously indicate which fields are present. +- **Forward-compatible reads** — `TableReader::Row()` returns `nullptr` when + `sizeof(T) > stride`, and record readers invalidate themselves if the stored + size is smaller than the requested record type. Unrecognized part types are + simply skipped by the parser (`default: continue;`). + +--- + +## Record schemas + +The record definitions live in the `RDAT_*Types.inl` files. Below are the +most important ones with their on-disk field encodings. All `*Ref`/string fields +are 4-byte handles; `RDAT_BYTES` is 8 bytes (`Offset`+`Size`). + +### `RuntimeDataResourceInfo` (ResourceTable, v1.3+) + +| Field | Encoding | Meaning | +|--------------|-----------------|------------------------------------------------| +| `Class` | `uint32_t` enum | `hlsl::DXIL::ResourceClass` (SRV/UAV/CBuffer/Sampler) | +| `Kind` | `uint32_t` enum | `hlsl::DXIL::ResourceKind` | +| `ID` | `uint32_t` | Resource ID within its class | +| `Space` | `uint32_t` | Register space | +| `LowerBound` | `uint32_t` | First register | +| `UpperBound` | `uint32_t` | Last register | +| `Name` | `RDATString` | Global name | +| `Flags` | `uint32_t` flags| `DxilResourceFlag` (UAVCounter, ROV, Atomics64, …) | + +### `RuntimeDataFunctionInfo` (FunctionTable, v1.3+) + +| Field | Encoding | Meaning | +|-------------------------|------------------------------|------------------------------------------| +| `Name` | `RDATString` | Full (mangled) function name | +| `UnmangledName` | `RDATString` | Unmangled name | +| `Resources` | `RecordArrayRef` | Global resources used | +| `FunctionDependencies` | `RDATStringArray` | External functions called (by name) | +| `ShaderKind` | `uint32_t` enum | `hlsl::DXIL::ShaderKind` | +| `PayloadSizeInBytes` | `uint32_t` | Ray payload / callable param size | +| `AttributeSizeInBytes` | `uint32_t` | Hit attribute size | +| `FeatureInfo1` | `uint32_t` flags | Low 32 bits of feature flags | +| `FeatureInfo2` | `uint32_t` flags | High 32 bits of feature flags | +| `ShaderStageFlag` | `uint32_t` flags | Valid shader stages (`DxilShaderStageFlags`) | +| `MinShaderTarget` | `uint32_t` (hex) | Encoded minimum shader model target | + +`GetFeatureFlags()`/`SetFeatureFlags()` combine `FeatureInfo1|2` into a 64-bit value. + +### `RuntimeDataFunctionInfo2` (FunctionTable, derived, v1.8+) + +Extends `RuntimeDataFunctionInfo` with: + +| Field | Encoding | Meaning | +|------------------------------|-------------------|------------------------------------------------| +| `MinimumExpectedWaveLaneCount` | `uint8_t` | 0 = unspecified | +| `MaximumExpectedWaveLaneCount` | `uint8_t` | 0 = unspecified | +| `ShaderFlags` | `uint16_t` flags | `DxilShaderFlags` (NodeProgramEntry, UsesViewID, …) | +| *union selected by `ShaderKind`* | 4 bytes each | `RawShaderRef` (uint32) / `RecordRef` to `NodeShaderInfo`, `VSInfo`, `PSInfo`, `HSInfo`, `DSInfo`, `GSInfo`, `CSInfo`, `MSInfo`, `ASInfo` | + +The `uint8/uint8/uint16` grouping keeps the added prefix 4-byte aligned before +the 4-byte union. + +### Node/work-graph records (v1.8+) + +- `NodeID` (NodeIDTable): `Name` (`RDATString`), `Index` (`uint32_t`). +- `RecordDispatchGrid` (inline value): `ByteOffset` (`uint16_t`), + `ComponentNumAndType` (`uint16_t`, bitfields: bits 0:2 num components, + bits 2:15 `ComponentType`). +- `NodeShaderFuncAttrib` (NodeShaderFuncAttribTable): `AttribKind` + (`uint32_t` enum) + union (`RecordRef`, `IndexArrayRef` for + NumThreads/DispatchGrid/MaxDispatchGrid, or `uint32_t` values) selected by + `AttribKind`. +- `NodeShaderIOAttrib` (NodeShaderIOAttribTable): `AttribKind` + union + (record ref, inline `RecordDispatchGrid`, or `uint32_t`). +- `IONode` (IONodeTable): `IOFlagsAndKind` (`uint32_t`, packs + `NodeIOFlags`+`NodeIOKind`), `Attribs` (`RecordArrayRef`). +- `NodeShaderInfo` (NodeShaderInfoTable): `LaunchType` (`uint32_t` enum), + `GroupSharedBytesUsed` (`uint32_t`), `Attribs` + (`RecordArrayRef`), `Outputs`/`Inputs` + (`RecordArrayRef`). + +### Subobject records (SubobjectTable, v1.4+) + +`RuntimeDataSubobjectInfo`: + +| Field | Encoding | Meaning | +|---------|-----------------|--------------------------------------| +| `Kind` | `uint32_t` enum | `hlsl::DXIL::SubobjectKind` | +| `Name` | `RDATString` | Subobject name | +| *union selected by `Kind`* | varies | See below | + +Union members (inline `RDAT_RECORD_VALUE` structs): + +- `StateObjectConfig_t`: `Flags` (`uint32_t` `StateObjectFlags`). +- `RootSignature_t`: `Data` (`RDAT_BYTES`) — used for both global and local RS. +- `SubobjectToExportsAssociation_t`: `Subobject` (`RDATString`), `Exports` + (`RDATStringArray`). +- `RaytracingShaderConfig_t`: `MaxPayloadSizeInBytes`, + `MaxAttributeSizeInBytes` (`uint32_t` each). +- `RaytracingPipelineConfig_t`: `MaxTraceRecursionDepth` (`uint32_t`). +- `HitGroup_t`: `Type` (`uint32_t` `HitGroupType`), `AnyHit`, `ClosestHit`, + `Intersection` (`RDATString` each). +- `RaytracingPipelineConfig1_t`: `MaxTraceRecursionDepth` (`uint32_t`), + `Flags` (`uint32_t` `RaytracingPipelineFlags`). + +### Signature / per-stage records (experimental) + +- `SignatureElement` (SignatureElementTable): `SemanticName` (`RDATString`), + `SemanticIndices` (`IndexArrayRef`), `SemanticKind`/`ComponentType`/ + `InterpolationMode` (`uint8_t` enums), `StartRow` (`uint8_t`, `0xFF` = not + allocated), `ColsAndStream` (`uint8_t` bitfield: cols-1, start col, output + stream), `UsageAndDynIndexMasks` (`uint8_t` bitfield). +- `VSInfo`/`PSInfo`/`HSInfo`/`DSInfo`/`GSInfo`/`MSInfo`: `RecordArrayRef` + for input/output (and patch-const/prim) signatures, `RDAT_BYTES` ViewID and + input-to-output masks, plus stage-specific scalars (control-point counts, + tessellator domain/primitive, primitive topology, max vertex count, etc.). +- `CSInfo`/`ASInfo`: `NumThreads` (`IndexArrayRef`), `GroupSharedBytesUsed` + (and `PayloadSizeInBytes` for AS). + +### PDB info records (PdbInfo group) + +- `DxilPdbInfoLibrary` (DxilPdbInfoLibraryTable): `Name` (`RDATString`), + `Data` (`RDAT_BYTES`). +- `DxilPdbInfoSource` (DxilPdbInfoSourceTable): `Name`, `Content` (`RDATString`). +- `DxilPdbInfo` (DxilPdbInfoTable): `Sources` + (`RecordArrayRef`), `Libraries` + (`RecordArrayRef`), `ArgPairs` (`RDATStringArray`), + `Hash` (`RDAT_BYTES`), `PdbName` (`RDATString`), `CustomToolchainId` + (`uint32_t`), `CustomToolchainData` (`RDAT_BYTES`), `WholeDxil` (`RDAT_BYTES`). + +--- + +## Reading and validation + +### Parsing (`DxilRuntimeData::InitFromRDAT`) + +The parser uses a bounds-checked `CheckedReader` that throws on overrun/overlap: + +1. Read `RuntimeDataHeader`; reject `Version < RDAT_Version_10`. +2. Read the `PartCount` offsets. +3. For each part: `Advance(offset)`, read `RuntimeDataPartHeader`, then bound a + sub-reader to `part.Size` bytes. +4. Dispatch on `part.Type`: + - `StringBuffer` → init `StringTableReader`. + - `IndexArrays` → init `IndexTableReader` with `Size/4` words. + - `RawBytes` → init `RawBytesReader`. + - Any table type → `InitTable` reads a `RuntimeDataTableHeader` and binds a + `TableReader(data, RecordCount, RecordStride)`. + - Unknown → skipped (`default: continue;`). +5. In debug builds, `Validate()` is run automatically. + +### Access + +`RDATContext` holds the three buffer readers plus a `TableReader` per +`RecordTableIndex`. Typed access is generated per record type: + +- `RecordTableReader` iterates a table; `Row(i)` returns a reader. +- `RecordReader` validates that the stored stride/size is large enough for + the requested record type before exposing fields; otherwise it becomes a null + reader. +- Field accessors resolve handles: string offsets → `StringBuffer.Get`, index + refs → `IndexTable.getRow`, record refs → `Table().Row`, byte refs → + `RawBytes.Get` with `Size`. + +### Validation (`DxilRuntimeData::Validate`) + +- The string buffer, if present, must end in `'\0'`. +- Every table is walked; for each record, all references are bounds-checked: + - `ValidateRecordRef` — index `< table.Count()` (or `RDAT_NULL_REF`). + - `ValidateIndexArrayRef` — offset in range and the encoded array length fits + within the remaining index space. + - `ValidateRecordArrayRef` / `ValidateStringArrayRef` — the index array is + well-formed and each element is itself a valid record/string ref. + - `ValidateStringRef` — offset `< StringBuffer.Size()`. +- `RecursiveRecordValidator` re-validates each record at every version up to the + one supported by the table stride, so extended (derived) fields are checked + too. A static assert guarantees `sizeof(derived) > sizeof(base)`. + +--- + +## Appendix: Key constants + +| Name | Value | Meaning | +|----------------------|--------------|--------------------------------------------| +| `DFCC_RuntimeData` | `'RDAT'` | Container FourCC for the RDAT part | +| `RDAT_Version_10` | `0x10` | Current RDAT blob version | +| `RDAT_NULL_REF` | `0xFFFFFFFF` | Null record/index/string reference | +| `PSVALIGN4(x)` | `(x+3)&~3` | 4-byte alignment used for part/record sizes | +| `RuntimeDataGroup::Core` | `0` | Default part group | +| `RuntimeDataGroup::PdbInfo` | `1` | PDB info part group (packed into high 16 bits) |