diff --git a/docs/DxilContainer_Format.md b/docs/DxilContainer_Format.md new file mode 100644 index 0000000000..1b79eedba1 --- /dev/null +++ b/docs/DxilContainer_Format.md @@ -0,0 +1,573 @@ +# DXIL Container Format + +A **DXIL container** is the on-disk/blob format produced by DXC for a compiled +shader or library. It is a small archive of independently-typed **parts**, +wrapping the LLVM bitcode (the actual DXIL program) together with reflection +metadata, signatures, validation data, hashes, debug names, and more. + +The format is intentionally backward-compatible with the legacy **DXBC** +container (its header FourCC is literally `'DXBC'`) so existing tooling can walk +the part table, while the individual parts carry DXIL-specific data. + +This document describes: + +- [What the container is and how it is used](#overview) +- [The container-level structure](#container-level-structure) +- [Alignment, versioning, and hashing](#alignment-versioning-and-hashing) +- [The part catalog](#part-catalog) +- [Part encodings in detail](#part-encodings) +- [Reading a container](#reading-a-container) +- [Appendix: constants](#appendix-constants) + +Primary sources for this description: + +- [include/dxc/DxilContainer/DxilContainer.h](../include/dxc/DxilContainer/DxilContainer.h) — all container/part header structs and helpers +- [include/dxc/DxilContainer/DxilPipelineStateValidation.h](../include/dxc/DxilContainer/DxilPipelineStateValidation.h) — the PSV0 part +- [lib/DxilContainer/DxilContainerAssembler.cpp](../lib/DxilContainer/DxilContainerAssembler.cpp) — the writer that assembles all parts +- [docs/RDAT_Format.md](RDAT_Format.md) — the RDAT part, documented separately + +--- + +## Overview + +### What it is + +A DXIL container is a flat, position-independent blob: + +- A **container header** (`DxilContainerHeader`) with a FourCC, a hash, a + version, the total size, and a part count. +- A **part offset table**: one `uint32_t` per part, each an absolute offset + (from the start of the container) to that part's header. +- A sequence of **parts**, each a `DxilPartHeader` (FourCC + size) followed by + that part's payload. + +Every part is self-describing (FourCC + size), so consumers can skip parts they +do not understand, and new part types can be added without breaking old readers. + +### How it is used + +- **Runtime/driver**: reads the `DXIL` program part to get the bitcode, and + `PSV0`/`RDAT` for pipeline validation and reflection. +- **Reflection APIs** (`IDxcContainerReflection`, `ID3D12ShaderReflection`): + read `STAT`/`RDAT`/signature parts. +- **Debuggers/PIX**: use `ILDB` (debug bitcode), `ILDN` (debug name/PDB + reference), `SRCI`/`PDBI` (sources & compile args), `HASH`. +- **Validation**: `dxil.dll` validates the container's parts for consistency. +- **Tooling** (`dxa`, `dxc -dumpbin`): extract or print individual parts. + +### Which parts appear + +The set of parts depends on the shader model and compile flags. From +`SerializeDxilContainerForModule`, the high-level rules are: + +- **Feature info** (`SFI0`) is always written first. +- **Non-library** targets get signature parts (`ISG1`, `OSG1`, and `PSG1` if a + patch-constant/primitive signature exists) and `PSV0`; a `RTS0` root + signature part if one is present. +- **Library** targets get `VERS` (validator ≥ 1.8) and `RDAT` instead of + `PSV0`/signatures. +- Debug/reflection/hash parts (`ILDB`, `ILDN`, `HASH`, `STAT`, `SRCI`) are + gated by flags and validator version. +- The `DXIL` program part is always written (near the end). +- `PRIV` private data, if supplied, is written last (its size is not + guaranteed aligned). + +--- + +## Container-level structure + +``` ++---------------------------------------------------------------+ +| DxilContainerHeader | +| uint32_t HeaderFourCC = 'DXBC' | +| DxilContainerHash Hash (16 bytes) | +| DxilContainerVersion Version { uint16 Major, Minor } | +| uint32_t ContainerSizeInBytes | +| uint32_t PartCount | ++---------------------------------------------------------------+ +| uint32_t PartOffset[PartCount] (absolute, from container start) | ++---------------------------------------------------------------+ +| Part 0: DxilPartHeader { uint32 PartFourCC, uint32 PartSize }| +| uint8_t PartData[PartSize] | ++---------------------------------------------------------------+ +| Part 1: DxilPartHeader ... | +| ... | ++---------------------------------------------------------------+ +``` + +All structures in `DxilContainer.h` are declared under `#pragma pack(push, 1)`, +so there is **no implicit padding** inside these headers; any alignment is +explicit (see below). All integers are little-endian. + +### `DxilContainerHeader` + +| Offset | Type | Field | Description | +|-------:|------------------------|-----------------------|----------------------------------------------------| +| 0 | `uint32_t` | `HeaderFourCC` | `DFCC_Container` = `'DXBC'` (back-compat with DXBC).| +| 4 | `uint8_t[16]` | `Hash.Digest` | Container hash (see hashing below). | +| 20 | `uint16_t` | `Version.Major` | Container major version (currently `1`). | +| 22 | `uint16_t` | `Version.Minor` | Container minor version (currently `0`). | +| 24 | `uint32_t` | `ContainerSizeInBytes`| Total container size from start of this header. | +| 28 | `uint32_t` | `PartCount` | Number of parts. | +| 32 | `uint32_t[PartCount]` | (part offsets) | Absolute offset of each `DxilPartHeader`. | + +`DxilContainerMaxSize` = `0x80000000` bounds the total size. + +### `DxilPartHeader` + +| Offset | Type | Field | Description | +|-------:|------------|---------------|----------------------------------------------| +| 0 | `uint32_t` | `PartFourCC` | Part type FourCC (`DFCC_*`). | +| 4 | `uint32_t` | `PartSize` | Size of `PartData`, **not** including header.| +| 8 | `uint8_t[PartSize]` | (data) | Part payload. | + +The offset table is written after the container header, and each part follows +the previous one contiguously: + +``` +offset[0] = sizeof(DxilContainerHeader) + PartCount * 4 +offset[i+1] = offset[i] + sizeof(DxilPartHeader) + parts[i].PartSize +``` + +### Access helpers + +`DxilContainer.h` provides inline helpers used throughout the codebase: + +- `GetDxilContainerPart(header, index)` — part header by index (via offset table). +- `GetDxilPartData(part)` — pointer to a part's payload (`part + 1`). +- `GetDxilPartByType(header, fourCC)` — find a part by FourCC. +- `GetDxilProgramHeader(header, fourCC)` — the `DxilProgramHeader` of a program part. +- `IsDxilContainerLike(ptr, length)` / `IsValidDxilContainer(...)` — validation. +- `DxilPartIterator`, `DxilPartIsType` — iterate/filter parts. + +--- + +## Alignment, versioning, and hashing + +### Alignment + +Parts are generally sized to a 4-byte boundary using `PSVALIGN4(x) = (x+3) & ~3`. +Writers pad payloads with zero bytes to reach the aligned size. The main +exception is the trailing `PRIV` private-data part, whose size is caller-defined +and not guaranteed aligned — which is exactly why it is written **last**. + +Whether the container writer aligns parts depends on validator version: +`bUnaligned = (ValVer < 1.7)`. Newer containers are aligned. + +> **Why the writer's output must be byte-exact (validator behavior).** +> Historically, the DXIL validator verified several container parts not by +> checking their *contents* semantically, but by **regenerating the part bytes +> from the module and running a `memcmp`** against the bytes produced by the +> compiler. This is implemented by `VerifyBlobPartMatches` in +> [lib/DxilValidation/DxilContainerValidation.cpp](../lib/DxilValidation/DxilContainerValidation.cpp), +> which reserves a buffer, calls the part's `DxilPartWriter::write()` to +> re-emit the part, and compares the result to the stored blob: +> +> ```cpp +> pWriter->write(pOutputStream); +> if (memcmp(pData, pOutputStream->GetPtr(), Size)) { +> ValCtx.EmitFormatError(ValidationRule::ContainerPartMatches, {pName}); +> return; +> } +> ``` +> +> Because this is a raw byte compare, the compiler's serialization — including +> field ordering, reserved/zero fields, and any **alignment padding** — has to +> be *bit-for-bit identical* to what the validator's writer produces. This is a +> key reason the alignment/padding behavior above is tied to the validator +> version: a container built for validator version *N* must match the byte +> layout that version *N*'s writers expect. +> +> **Has this changed?** Partly. In the current validator this +> regenerate-and-`memcmp` approach is *still used* for: +> - program signatures (`ISG1`/`OSG1`/`PSG1`) via `VerifySignatureMatches`, +> - the feature-info part (`SFI0`) via `VerifyFeatureInfoMatches`, and +> - the runtime-data part (`RDAT`) via `VerifyRDATMatches`. +> +> For these parts the writer output must still be byte-exact. However, PSV0 +> validation was **changed** to a semantic, field-by-field comparison. The +> older `VerifyPSVMatches` regenerated the whole PSV0 blob and `memcmp`'d it; +> the current implementation uses `PSVContentVerifier`, which decodes both the +> stored part and the module's expected values and compares them field by field +> (emitting a human-readable `ContainerContentMatches` diff on mismatch) +> instead of requiring an identical byte image. This change landed in +> DXC in late 2024 (PRs #6859 and #6924), so validators built before then +> validate PSV0 with the strict byte-for-byte `memcmp`, while newer validators +> tolerate byte-level differences (for example padding) as long as the decoded +> content agrees. + +### Container vs. program vs. part versions + +There are several independent version numbers: + +- **Container version** — `DxilContainerHeader.Version` (major/minor, currently + 1.0). +- **Program version** — inside `DxilProgramHeader.ProgramVersion`, encodes the + shader kind and shader-model version: + + ```cpp + ProgramVersion = (ShaderKind << 16) | (Major << 4) | Minor; + // decode: + ShaderKind = (ProgramVersion >> 16) & 0xFFFF; + Major = (ProgramVersion >> 4) & 0xF; + Minor = ProgramVersion & 0xF; + ``` + +- **Validator version** — governs which parts and which part-internal versions + are emitted (e.g. `HASH` needs ≥ 1.5, `VERS` needs ≥ 1.8, unaligned parts for + < 1.7). +- **Per-part versions** — several parts version *additively by size* (PSV0, + RDAT records), so a reader infers the version from the recorded size/stride. + +### Hashing + +`DxilContainerHeader.Hash` is a 16-byte digest. Separately, the `HASH` part +(`DxilShaderHash`) stores a `Flags` field plus a 16-byte MD5 digest computed +over either the final program bitcode or the source-inclusive bitcode +(`DxilShaderHashFlags::IncludesSource` when `-Zss`/`DebugNameDependOnSource`). +A sentinel `PreviewByPassHash` (all `2`s) is used in some preview scenarios. + +--- + +## Part catalog + +| FourCC | `DxilFourCC` | Contents | +|--------|---------------------------------|----------------------------------------------------------------| +| `DXBC` | `DFCC_Container` | Container header FourCC (not a part). | +| `RDEF` | `DFCC_ResourceDef` | Legacy DXBC resource-definition reflection. | +| `ISG1` | `DFCC_InputSignature` | Input signature (`DxilProgramSignature`). | +| `OSG1` | `DFCC_OutputSignature` | Output signature. | +| `PSG1` | `DFCC_PatchConstantSignature` | Patch-constant / primitive signature. | +| `STAT` | `DFCC_ShaderStatistics` | Reflection data (program-header-wrapped reflection bitcode). | +| `ILDB` | `DFCC_ShaderDebugInfoDXIL` | DXIL bitcode **with** debug info. | +| `ILDN` | `DFCC_ShaderDebugName` | Debug name / PDB file reference (`DxilShaderDebugName`). | +| `SFI0` | `DFCC_FeatureInfo` | 64-bit shader feature flags (`DxilShaderFeatureInfo`). | +| `PRIV` | `DFCC_PrivateData` | Arbitrary caller-provided private data. | +| `RTS0` | `DFCC_RootSignature` | Serialized root signature. | +| `DXIL` | `DFCC_DXIL` | The DXIL program (LLVM bitcode via `DxilProgramHeader`). | +| `PSV0` | `DFCC_PipelineStateValidation` | Pipeline State Validation data. | +| `RDAT` | `DFCC_RuntimeData` | Runtime Data reflection (see [RDAT doc](RDAT_Format.md)). | +| `HASH` | `DFCC_ShaderHash` | Shader hash (`DxilShaderHash`). | +| `SRCI` | `DFCC_ShaderSourceInfo` | Source names, contents, and compile args. | +| `PDBI` | `DFCC_ShaderPDBInfo` | PDB info (RDAT-encoded; replaces `SRCI` in the PDB). | +| `VERS` | `DFCC_CompilerVersion` | Compiler version info (`DxilCompilerVersion`). | + +The FourCC value is computed little-endian by `DXIL_FOURCC(c0,c1,c2,c3) = +c0 | c1<<8 | c2<<16 | c3<<24`, i.e. the characters read in order in a hex dump. + +--- + +## Part encodings + +### `DXIL` / `ILDB` — the DXIL program + +Both the program part (`DXIL`) and the debug-info program part (`ILDB`) share +the same layout: a `DxilProgramHeader` followed by LLVM bitcode. + +`DxilProgramHeader`: + +| Offset | Type | Field | Description | +|-------:|--------------------|-----------------|-------------------------------------------------| +| 0 | `uint32_t` | `ProgramVersion`| Shader kind + SM version (see encoding above). | +| 4 | `uint32_t` | `SizeInUint32` | Size in `uint32_t` units, including this header. | +| 8 | `DxilBitcodeHeader`| `BitcodeHeader` | Bitcode-specific sub-header. | + +`DxilBitcodeHeader`: + +| Offset | Type | Field | Description | +|-------:|------------|-----------------|------------------------------------------------| +| 0 | `uint32_t` | `DxilMagic` | `'DXIL'` = `0x4C495844`. | +| 4 | `uint32_t` | `DxilVersion` | DXIL version. | +| 8 | `uint32_t` | `BitcodeOffset` | Offset to LLVM bitcode from start of this header.| +| 12 | `uint32_t` | `BitcodeSize` | Size of the LLVM bitcode. | + +The bitcode itself is standard LLVM bitcode encoding the DXIL module. The +program payload is zero-padded so the part size is a multiple of 4 bytes. +`ILDB` carries the same module *before* debug info is stripped, so it is only +present when debug info is requested. + +### `ISG1` / `OSG1` / `PSG1` — signatures + +Each signature part starts with a `DxilProgramSignature` header and an array of +`DxilProgramSignatureElement`, with semantic-name strings stored after the +elements and referenced by offset. + +`DxilProgramSignature`: + +| Offset | Type | Field | Description | +|-------:|------------|--------------|----------------------------------------------------| +| 0 | `uint32_t` | `ParamCount` | Number of signature elements. | +| 4 | `uint32_t` | `ParamOffset`| Offset (from start of this header) to the element array. | + +`DxilProgramSignatureElement` (exactly `0x20` = 32 bytes, static-asserted): + +| Offset | Type | Field | Description | +|-------:|-----------------------------|----------------|--------------------------------------------------------------| +| 0 | `uint32_t` | `Stream` | Stream index (non-decreasing order). | +| 4 | `uint32_t` | `SemanticName` | Offset to `LPCSTR` from start of `DxilProgramSignature`. | +| 8 | `uint32_t` | `SemanticIndex`| Semantic index. | +| 12 | `DxilProgramSigSemantic` | `SystemValue` | System-value semantic (`uint32_t` enum). | +| 16 | `DxilProgramSigCompType` | `CompType` | Component type (`uint32_t` enum). | +| 20 | `uint32_t` | `Register` | Register (row) index. | +| 24 | `uint8_t` | `Mask` | Column allocation mask. | +| 25 | `uint8_t` | `NeverWrites_Mask` / `AlwaysReads_Mask` | Union; write-never (output) or read-always (input) mask. | +| 26 | `uint16_t` | `Pad` | Padding to align. | +| 28 | `DxilProgramSigMinPrecision`| `MinPrecision` | Minimum precision (`uint32_t` enum). | + +Enums: +- `DxilProgramSigSemantic` mirrors `D3D_NAME` (Position, ClipDistance, Target, + Depth, Barycentrics, ShadingRate, …). +- `DxilProgramSigCompType` (Unknown/UInt32/SInt32/Float32/…/Float64). +- `DxilProgramSigMinPrecision` (Default/Float16/Float2_8/SInt16/UInt16/Any16/Any10). + +### `PSV0` — Pipeline State Validation + +The PSV0 part is a size-versioned, self-describing structure used by the D3D12 +runtime to validate pipeline state without parsing bitcode. It is written/read +by `DxilPipelineStateValidation::ReadOrWrite`. Every fixed-size sub-structure is +preceded by a `uint32_t` recording its size, so newer (larger) versions are +readable by older code, which just ignores trailing bytes. + +Top-level layout, in the order the blocks appear in the blob (from the header +comment in +[DxilPipelineStateValidation.h, lines 899–943](https://github.com/microsoft/DirectXShaderCompiler/blob/main/include/dxc/DxilContainer/DxilPipelineStateValidation.h#L899-L943)): + +| # | Block | Type / size | Present when | Description | +|--:|-------|-------------|--------------|-------------| +| 1 | `PSVRuntimeInfo_size` | `uint32_t` | always | Size of the following record; selects the runtime-info version *N*. | +| 2 | `PSVRuntimeInfoN` | `PSVRuntimeInfo_size` bytes | always | Fixed per-stage runtime info (see the version table below). | +| 3 | `ResourceCount` | `uint32_t` | always | Number of resource-binding records. | +| 4 | `PSVResourceBindInfo_size` | `uint32_t` | `ResourceCount > 0` | Size of each bind-info record; selects its version. | +| 5 | `PSVResourceBindInfoN` × `ResourceCount` | array | `ResourceCount > 0` | Resource-binding records. | +| 6 | `StringTableSize` | `uint32_t` (dword-aligned) | version ≥ 1 | Size of the string pool in bytes. | +| 7 | String table | `StringTableSize` bytes, null-terminated | version ≥ 1 and `StringTableSize > 0` | Semantic-name pool. | +| 8 | `SemanticIndexTableEntries` | `uint32_t` | version ≥ 1 | Number of dwords in the semantic-index table. | +| 9 | Semantic-index table | `uint32_t` × `SemanticIndexTableEntries` | `SemanticIndexTableEntries > 0` | Semantic-index values. | +| 10 | `PSVSignatureElement_size` | `uint32_t` | any of `SigInputElements` / `SigOutputElements` / `SigPatchConstOrPrimElements` | Size of each signature-element record. | +| 11 | Input signature elements | `PSVSignatureElementN` × `SigInputElements` | as row 10 | Input signature. | +| 12 | Output signature elements | `PSVSignatureElementN` × `SigOutputElements` | as row 10 | Output signature. | +| 13 | Patch-const/prim elements | `PSVSignatureElementN` × `SigPatchConstOrPrimElements` | as row 10 | Patch-constant / primitive signature. | +| 14 | ViewID→output masks | per stream *i* (0–3): `uint32_t` × `MaskDwords(SigOutputVectors[i])` | `UsesViewID` and `SigOutputVectors[i] != 0` | Outputs affected by ViewID, as a bitmask. | +| 15 | ViewID→patch-const masks | `uint32_t` × `MaskDwords(SigPatchConstOrPrimVectors)` | `UsesViewID` and HS and `SigPatchConstOrPrimVectors != 0` | Patch-const outputs affected by ViewID. | +| 16 | Input→output tables | per stream *i* (0–3): `uint32_t` × `InputOutputTableDwords(SigInputVectors, SigOutputVectors[i])` | `SigInputVectors` and `SigOutputVectors[i] != 0` | Outputs affected by inputs, as a table of bitmasks. | +| 17 | Input→patch-const table | `uint32_t` × `InputOutputTableDwords(SigInputVectors, SigPatchConstOrPrimVectors)` | HS and `SigPatchConstOrPrimVectors` and `SigInputVectors` | Patch-const outputs affected by inputs. | +| 18 | Patch-const→output table | `uint32_t` × `InputOutputTableDwords(SigPatchConstOrPrimVectors, SigOutputVectors[0])` | DS and `SigOutputVectors[0]` and `SigPatchConstOrPrimVectors` | Outputs affected by patch-constant inputs. | + +Where the dword-count helpers are: + +- `MaskDwords(vectors) = (vectors + 7) >> 3` (one bit per component, 4 + components per vector). +- `InputOutputTableDwords(in, out) = MaskDwords(out) * in * 4`. + +**`PSVRuntimeInfo0`** (base, `MAX_PSV_VERSION = 4`) begins with a union of +per-stage info (`VSInfo`/`HSInfo`/`DSInfo`/`GSInfo`/`PSInfo`/`MSInfo`/`ASInfo`) +followed by `MinimumExpectedWaveLaneCount`/`MaximumExpectedWaveLaneCount`. +Successive versions append fields: + +- **`PSVRuntimeInfo1`**: `ShaderStage` (`PSVShaderKind`), `UsesViewID`, a union + (`MaxVertexCount` for GS / `SigPatchConstOrPrimVectors` / `MSInfo1`), signature + element counts (`SigInputElements`, `SigOutputElements`, + `SigPatchConstOrPrimElements`), and packed vector counts (`SigInputVectors`, + `SigOutputVectors[4]` for GS streams). +- **`PSVRuntimeInfo2`**: `NumThreadsX/Y/Z`. +- **`PSVRuntimeInfo3`**: `EntryFunctionName` (offset into the string table). +- **`PSVRuntimeInfo4`**: `NumBytesGroupSharedMemory`. + +**`PSVResourceBindInfo0`**: `ResType` (`PSVResourceType`), `Space`, `LowerBound`, +`UpperBound`. **`PSVResourceBindInfo1`** adds `ResKind` (`PSVResourceKind`) and +`ResFlags` (`PSVResourceFlag`, e.g. `UsedByAtomic64`). + +**`PSVSignatureElement0`** (referenced via the string/semantic-index tables): + +| Offset | Type | Field | Description | +|-------:|------------|-----------------------|---------------------------------------------------------| +| 0 | `uint32_t` | `SemanticName` | Offset into PSV string table. | +| 4 | `uint32_t` | `SemanticIndexes` | Offset into semantic-index table; count == `Rows`. | +| 8 | `uint8_t` | `Rows` | Rows occupied. | +| 9 | `uint8_t` | `StartRow` | Packing start row (if allocated). | +| 10 | `uint8_t` | `ColsAndStart` | bits 0:4 `Cols`, 4:6 `StartCol`, bit 6 `Allocated`. | +| 11 | `uint8_t` | `SemanticKind` | `PSVSemanticKind`. | +| 12 | `uint8_t` | `ComponentType` | `DxilProgramSigCompType`. | +| 13 | `uint8_t` | `InterpolationMode` | `DXIL::InterpolationMode`. | +| 14 | `uint8_t` | `DynamicMaskAndStream`| bits 0:4 dynamic-index mask, 4:6 output stream. | +| 15 | `uint8_t` | `Reserved` | Reserved. | + +The PSV string table is dword-aligned with a trailing null; the semantic-index +table is an array of `uint32_t`. ViewID and input→output dependency data are +stored as packed component-mask bitfields (see helpers above). + +### `SFI0` — feature info + +`DxilShaderFeatureInfo` is a single `uint64_t FeatureFlags`. The bit meanings +are the shader feature flags (Doubles, WaveOps, Int64Ops, ViewID, Barycentrics, +Raytracing Tier 1.1, ResourceDescriptorHeapIndexing, etc.), matching the +`DxilFeatureInfo1`/`DxilFeatureInfo2` enumerations documented in the +[RDAT doc](RDAT_Format.md) (there the same 64-bit value is split into two 32-bit +halves). + +### `HASH` — shader hash + +`DxilShaderHash`: + +| Offset | Type | Field | Description | +|-------:|---------------|----------|------------------------------------------------| +| 0 | `uint32_t` | `Flags` | `DxilShaderHashFlags` (`IncludesSource` = 1). | +| 4 | `uint8_t[16]` | `Digest` | 16-byte MD5 digest. | + +Written only when validator version ≥ 1.5. + +### `ILDN` — shader debug name + +`DxilShaderDebugName` followed by the name bytes: + +| Offset | Type | Field | Description | +|-------:|------------|--------------|------------------------------------------------| +| 0 | `uint16_t` | `Flags` | Reserved, must be 0. | +| 2 | `uint16_t` | `NameLength` | Debug name length, excluding null terminator. | +| 4 | `char[NameLength]` | (name) | UTF-8 name. | +| … | `char` | (null) | Null terminator. | +| … | `char[0-3]`| (pad) | Zero padding to a 4-byte boundary. | + +`MinDxilShaderDebugNameSize = sizeof(DxilShaderDebugName) + 4`. This part +typically names the associated `.pdb` (default: `.pdb`). + +### `VERS` — compiler version + +`DxilCompilerVersion`: + +| Offset | Type | Field | Description | +|-------:|------------|----------------------------------|----------------------------------------------| +| 0 | `uint16_t` | `Major` | Compiler major version. | +| 2 | `uint16_t` | `Minor` | Compiler minor version. | +| 4 | `uint32_t` | `VersionFlags` | Flags. | +| 8 | `uint32_t` | `CommitCount` | Commit count. | +| 12 | `uint32_t` | `VersionStringListSizeInBytes` | Size of the string list that follows. | +| 16 | `char[...]`| (strings) | Up to two null-terminated strings: commit SHA, then custom version string. Zero-padded to 4 bytes. | + +Written for library targets when validator version ≥ 1.8 and version info is +available. + +### `SRCI` — shader source info + +A `DxilSourceInfo` header followed by `SectionCount` sections, each a +`DxilSourceInfoSection` header plus a type-specific, 4-byte-aligned payload. + +`DxilSourceInfo`: + +| Offset | Type | Field | Description | +|-------:|------------|----------------------|----------------------------------------------| +| 0 | `uint32_t` | `AlignedSizeInBytes` | Total size including this header. | +| 4 | `uint16_t` | `Flags` | Reserved, 0. | +| 6 | `uint16_t` | `SectionCount` | Number of sections. | + +`DxilSourceInfoSection`: + +| Offset | Type | Field | Description | +|-------:|-----------------------------|----------------------|------------------------------------------| +| 0 | `uint32_t` | `AlignedSizeInBytes` | Section size including header + padding. | +| 4 | `uint16_t` | `Flags` | Reserved, 0. | +| 6 | `DxilSourceInfoSectionType` | `Type` | `SourceContents` / `SourceNames` / `Args`.| + +Section payloads: + +- **Source names** (`DxilSourceInfo_SourceNames` + `..._SourceNamesEntry[]`): + `Count`, `EntriesSizeInBytes`, then per-entry `AlignedSizeInBytes`, `Flags`, + `NameSizeInBytes`, `ContentSizeInBytes`, followed by the UTF-8 name (null + terminated, padded to 4 bytes). +- **Source contents** (`DxilSourceInfo_SourceContents` + + `..._SourceContentsEntry[]`): may be **zlib-compressed** (`CompressType`), with + `EntriesSizeInBytes` (compressed) and `UncompressedEntriesSizeInBytes`; each + uncompressed entry is `AlignedSizeInBytes`, `Flags`, `ContentSizeInBytes`, then + the content (null terminated, padded). +- **Args** (`DxilSourceInfo_Args`): `Flags`, `SizeInBytes`, `Count`, then + `Count` argument pairs encoded as `Name\0Value\0` (e.g. `T\0ps_6_0\0`, + `D\0MyDefine=1\0`, `Zi\0\0`). + +### `PDBI` — shader PDB info + +`DxilShaderPDBInfo` header followed by (optionally zlib-compressed) data that is +itself **RDAT-encoded** (see `RDAT_PdbInfoTypes.inl` and the +[RDAT doc](RDAT_Format.md)). It supersedes `SRCI` inside PDB files. + +| Offset | Type | Field | Description | +|-------:|-----------------------------------|---------------------------|------------------------------------| +| 0 | `DxilShaderPDBInfoVersion` (`uint16_t`) | `Version` | `Version_0` currently. | +| 2 | `DxilShaderPDBInfoCompressionType` (`uint16_t`) | `CompressionType` | `Uncompressed` or `Zlib`. | +| 4 | `uint32_t` | `SizeInBytes` | Size of the (possibly compressed) data. | +| 8 | `uint32_t` | `UncompressedSizeInBytes` | Uncompressed size. | + +### `RTS0` — root signature + +A serialized root signature blob (the same format D3D12 consumes via +`D3D12SerializeRootSignature`). It can be embedded in the container and/or +emitted standalone (wrapped in its own single-part container via +`SerializeDxilContainerForRootSignature`). For non-library targets, DXC copies +the module's serialized root signature into this part. + +### `STAT` — shader statistics / reflection + +A program part (same `DxilProgramHeader` wrapper as `DXIL`) whose bitcode is a +**reflection-only** clone of the module produced by +`StripAndCreateReflectionStream`. Reflection APIs read this to expose full +reflection for both shaders and libraries. For libraries, an `RDAT` part is +appended alongside `STAT` in the separate reflection stream. + +### `PRIV` — private data + +Opaque, caller-provided bytes copied verbatim. Written **last** because its size +is not guaranteed to be 4-byte aligned, so it must not perturb the alignment of +following parts (there are none after it). + +### `RDEF` — resource definitions (legacy) + +The legacy DXBC `RDEF` reflection part is recognized by the FourCC enumeration +for back-compat with DXBC tooling. DXIL reflection is primarily carried by +`STAT`/`RDAT`; see those parts and [RDAT_Format.md](RDAT_Format.md). + +--- + +## Reading a container + +A typical consumer: + +1. Validate with `IsDxilContainerLike(ptr, length)` (checks the `'DXBC'` FourCC + and minimum length) and, for stronger guarantees, `IsValidDxilContainer`. +2. Read `PartCount` and the offset table. +3. For each index, `GetDxilContainerPart` returns the `DxilPartHeader`; switch on + `PartFourCC`, or use `GetDxilPartByType` to jump to a specific part. +4. Use `GetDxilPartData` to access a part's payload, bounded by `PartSize`. +5. For program parts, `GetDxilProgramHeader` yields the `DxilProgramHeader`; the + bitcode is at `BitcodeHeader.BitcodeOffset` for `BitcodeHeader.BitcodeSize` + bytes, and `GetVersionShaderType`/`GetVersionMajor`/`GetVersionMinor` decode + the shader model. + +Unknown FourCCs should be skipped — this is what makes the format extensible. + +--- + +## Appendix: constants + +| Name | Value | Meaning | +|------------------------------|------------------|---------------------------------------------| +| `DFCC_Container` | `'DXBC'` | Container header FourCC. | +| `DxilContainerVersionMajor` | `1` | Current container major version. | +| `DxilContainerVersionMinor` | `0` | Current container minor version. | +| `DxilContainerMaxSize` | `0x80000000` | Maximum container size. | +| `DxilContainerHashSize` | `16` | Container/hash digest length in bytes. | +| `DxilMagicValue` | `0x4C495844` | `'DXIL'` bitcode magic. | +| `MinDxilShaderDebugNameSize` | `sizeof(DxilShaderDebugName)+4` | Minimum `ILDN` part size. | +| `MAX_PSV_VERSION` | `4` | Highest PSV runtime-info version. | +| `PSVALIGN4(x)` | `(x+3)&~3` | 4-byte alignment used across parts. | +| `PreviewByPassHash` | all `2`s | Sentinel container hash for preview. | + +### Program version encoding + +``` +ProgramVersion = (ShaderKind << 16) | (Major << 4) | Minor +``` + +### FourCC encoding + +``` +DXIL_FOURCC(c0, c1, c2, c3) = c0 | (c1 << 8) | (c2 << 16) | (c3 << 24) +``` diff --git a/docs/RDAT_Format.md b/docs/RDAT_Format.md new file mode 100644 index 0000000000..b2440e7efb --- /dev/null +++ b/docs/RDAT_Format.md @@ -0,0 +1,578 @@ +# DXIL Runtime Data (RDAT) Format + +RDAT ("Runtime Data") is a compact, self-describing binary blob emitted by DXC +that carries reflection-style metadata about a compiled DXIL library or shader. +It is designed to be consumed at runtime (by a driver/runtime) without needing +to parse the LLVM bitcode, and it is engineered for forward and backward +compatibility so that newer compilers can add data that older consumers safely +ignore, and newer consumers can read older blobs. + +This document describes: + +- [What RDAT is and how it is used](#overview) +- [The high-level structure](#high-level-structure) +- [The binary encoding in detail](#binary-encoding) +- [The part types](#part-types) +- [The record/field reference model](#record-and-field-reference-model) +- [Versioning and compatibility](#versioning-and-compatibility) +- [The record schemas](#record-schemas) +- [Reading and validation](#reading-and-validation) + +The primary sources for this description are: + +- [include/dxc/DxilContainer/DxilRuntimeReflection.h](../include/dxc/DxilContainer/DxilRuntimeReflection.h) — readers, headers, context +- [include/dxc/DxilContainer/DxilRuntimeReflection.inl](../include/dxc/DxilContainer/DxilRuntimeReflection.inl) — parsing/validation implementation +- [include/dxc/DxilContainer/RDAT_Macros.inl](../include/dxc/DxilContainer/RDAT_Macros.inl) — the X-macro metaprogramming that defines all record layouts +- [include/dxc/DxilContainer/RDAT_LibraryTypes.inl](../include/dxc/DxilContainer/RDAT_LibraryTypes.inl), [RDAT_SubobjectTypes.inl](../include/dxc/DxilContainer/RDAT_SubobjectTypes.inl), [RDAT_PdbInfoTypes.inl](../include/dxc/DxilContainer/RDAT_PdbInfoTypes.inl) — the actual record definitions +- [include/dxc/DxilContainer/DxilRDATBuilder.h](../include/dxc/DxilContainer/DxilRDATBuilder.h) and [lib/DxilContainer/DxilRDATBuilder.cpp](../lib/DxilContainer/DxilRDATBuilder.cpp) — the writer +- [include/dxc/DxilContainer/DxilRDATParts.h](../include/dxc/DxilContainer/DxilRDATParts.h) — the part builders + +--- + +## Overview + +### Where RDAT lives + +RDAT is stored as a single part inside a DXIL container, identified by the +FourCC `DFCC_RuntimeData` = `'RDAT'` (see +[DxilContainer.h](../include/dxc/DxilContainer/DxilContainer.h)). The DXIL +container writer adds it via: + +```cpp +writer.AddPart(DFCC_RuntimeData, pRDATWriter->size(), + [&](AbstractMemoryStream *pStream) { pRDATWriter->write(pStream); }); +``` + +Within that container part, RDAT is *itself* a mini-container with its own +header, offset table, and sub-parts. The comment on `DxilRDATBuilder` +summarizes it well: + +> Like DXIL container, RDAT itself is a mini container that contains multiple +> RDAT parts. + +### What it contains + +RDAT primarily describes, for a library or shader: + +- **Resources** used (SRV/UAV/CBuffer/Sampler), including register binding, + space, kind and resource flags. +- **Functions / entry points**: name, unmangled name, shader kind, resources + referenced, functions called (dependencies), feature/shader-stage flags, + minimum shader target, payload/attribute sizes, and (v1.8+) stage-specific + information. +- **Subobjects** (RDAT v1.4+): state object config, root signatures, hit + groups, raytracing shader/pipeline configs, associations. +- **Node shader information** (v1.8+): launch type, node I/O, node attributes + for work graphs. +- **Per-stage signature/pipeline info** (experimental): VS/PS/HS/DS/GS/CS/MS/AS + info, signature elements, ViewID masks, thread group sizes, etc. +- **PDB info** (a separate group): embedded sources, libraries, compile + arguments, hashes. + +### How it is used + +- **Runtime/driver reflection**: A runtime can load the blob with + `DxilRuntimeData` and query typed record tables without parsing bitcode. +- **Subobject loading**: `LoadSubobjectsFromRDAT` + ([RDATDxilSubobjects.cpp](../lib/DxilContainer/RDATDxilSubobjects.cpp)) + reconstructs `DxilSubobjects` from the subobject table for RDXR/state object + creation. +- **Validation**: The validator re-checks the structure; `DxilRuntimeData::Validate()` + bounds-checks every reference. +- **Tooling/debug**: `RDATDumper` ([RDATDumper.cpp](../lib/DxilContainer/RDATDumper.cpp)) + produces a human-readable dump. + +--- + +## High-level structure + +An RDAT blob is laid out as follows: + +``` ++-------------------------------------------------------------+ +| RuntimeDataHeader | 8 bytes +| uint32_t Version | +| uint32_t PartCount | ++-------------------------------------------------------------+ +| uint32_t PartOffsets[PartCount] | 4 * PartCount bytes +| (each offset is relative to the start of the header) | ++-------------------------------------------------------------+ +| Part 0 | +| RuntimeDataPartHeader { Type, Size } | 8 bytes +| byte Data[ALIGN4(Size)] | ++-------------------------------------------------------------+ +| Part 1 ... | +| ... | ++-------------------------------------------------------------+ +``` + +Each part is one of a small set of "kinds": the **string buffer**, the **index +array buffer**, the **raw bytes buffer**, or a **record table** (many of these, +one per record type). Records inside tables never store variable-length data +inline; instead they store 4-byte *offsets/indices* into the string, index, or +raw-bytes parts. This is what keeps records fixed-size and strided, enabling +forward/backward compatibility. + +```mermaid +graph TD + Container["DXIL Container part: 'RDAT'"] --> Hdr["RuntimeDataHeader + offsets[]"] + Hdr --> P0["StringBuffer part"] + Hdr --> P1["IndexArrays part"] + Hdr --> P2["RawBytes part"] + Hdr --> P3["ResourceTable"] + Hdr --> P4["FunctionTable"] + Hdr --> P5["SubobjectTable / ... more tables"] + P4 -- "RDATString offset" --> P0 + P4 -- "RecordArrayRef / IndexArrayRef" --> P1 + P3 -- "record indices via index arrays" --> P1 + P5 -- "BytesRef offset+size" --> P2 +``` + +--- + +## Binary encoding + +All multi-byte integers are little-endian (matching the rest of the DXIL +container format and the target platforms). All sizes and offsets described as +"4-byte aligned" are padded up with `PSVALIGN4(x) = (x + 3) & ~3`. + +### `RuntimeDataHeader` + +Defined in [DxilRuntimeReflection.h](../include/dxc/DxilContainer/DxilRuntimeReflection.h). + +| Offset | Type | Field | Description | +|-------:|------------|------------|------------------------------------------------------| +| 0 | `uint32_t` | `Version` | RDAT format version. Currently `RDAT_Version_10 = 0x10`. | +| 4 | `uint32_t` | `PartCount`| Number of parts that follow. | +| 8 | `uint32_t[PartCount]` | (offsets) | Offset of each part, **relative to the start of this header**. Offsets are 4-byte aligned. | + +Size: `8 + 4 * PartCount` bytes. The value `0x10` for the version was chosen so +it "cannot be mistaken for a part count from the prerelease version" — old blobs +started with a raw part count, so `0x10` disambiguates. + +### `RuntimeDataPartHeader` + +Each part begins (at its offset) with: + +| Offset | Type | Field | Description | +|-------:|------------------------|--------|--------------------------------------------------------| +| 0 | `RuntimeDataPartType` (`uint32_t`) | `Type` | Identifies which kind of part this is. | +| 4 | `uint32_t` | `Size` | Size of the part data **not** including this header. Must be 4-byte aligned. | +| 8 | `byte[ALIGN4(Size)]` | (data) | Part payload. | + +The writer always stores `Size = PSVALIGN4(actualPartSize)`, and empty parts +(size 0) are omitted entirely (see `ComputeSize` / `FinalizeAndGetData` in +[DxilRDATBuilder.cpp](../lib/DxilContainer/DxilRDATBuilder.cpp)). + +### Overall size computation + +From `DxilRDATBuilder::ComputeSize`: + +``` +total = sizeof(RuntimeDataHeader) // 8 + + numNonEmptyParts * sizeof(uint32_t) // offset table + + Σ (sizeof(RuntimeDataPartHeader) + PSVALIGN4(part.GetPartSize())) +``` + +Only non-empty parts contribute; the header's `PartCount` equals +`numNonEmptyParts`. + +### Writing order + +The writer instantiates parts in the specific order the validator expects +(from `DxilRDATWriter`'s constructor): + +1. `StringBuffer` +2. `ResourceTable` +3. `FunctionTable` +4. `IndexArrays` +5. `RawBytes` +6. `SubobjectTable` (if allowed by validator version) +7. Remaining tables in declaration order (only those `<= maxAllowedType`) + +Ordering matters only for producing a byte-identical, validator-friendly layout; +readers locate parts by `Type` via the offset table, not by position. + +--- + +## Part types + +`RuntimeDataPartType` (a `uint32_t` enum) enumerates every part kind. The +non-table kinds are the shared buffers; the rest are record tables. + +| Value | Name | Introduced | Notes | +|-------:|-----------------------------|:----------:|------------------------------------------| +| 0 | `Invalid` | — | | +| 1 | `StringBuffer` | 1.3 | Shared UTF-8 string pool | +| 2 | `IndexArrays` | 1.3 | Shared array-of-uint32 pool | +| 3 | `ResourceTable` | 1.3 | `RuntimeDataResourceInfo` records | +| 4 | `FunctionTable` | 1.3 | `RuntimeDataFunctionInfo[2]` records; `Last_1_3` | +| 5 | `RawBytes` | 1.4 | Shared raw byte pool | +| 6 | `SubobjectTable` | 1.4 | `Last_1_4` | +| 7 | `NodeIDTable` | 1.8 | | +| 8 | `NodeShaderIOAttribTable` | 1.8 | | +| 9 | `NodeShaderFuncAttribTable` | 1.8 | | +| 10 | `IONodeTable` | 1.8 | | +| 11 | `NodeShaderInfoTable` | 1.8 | `Last_1_8` | +| 12 | `Reserved_MeshNodesPreviewInfoTable` | — | reserved | +| 13+ | `SignatureElementTable`, `VSInfoTable`, `PSInfoTable`, `HSInfoTable`, `DSInfoTable`, `GSInfoTable`, `CSInfoTable`, `MSInfoTable`, `ASInfoTable` | experimental | **experimental** | + +The PDB-info group uses a separate namespace via `RDAT_PART_ID_WITH_GROUP`, +which packs a `RuntimeDataGroup` into the high 16 bits: + +```cpp +RDAT_PART_ID_WITH_GROUP(group, id) = ((group << 16) | (id & 0xFFFF)) +``` + +- Group `Core = 0` — all the parts above. +- Group `PdbInfo = 1` — `DxilPdbInfoTable`, `DxilPdbInfoSourceTable`, + `DxilPdbInfoLibraryTable`. + +`RecordTableIndex` is a parallel dense enum used to index the reader's internal +`Tables[]` array; note that its ordering is **not** the same as the numeric +`RuntimeDataPartType` values (e.g. PDB tables are grouped in the middle of +`RecordTableIndex` but have high `RuntimeDataPartType` values). + +### StringBuffer part + +Type `1`. A flat blob of UTF-8 bytes. The first byte is always `'\0'` so that +offset `0` is a valid empty/null string: + +```cpp +StringBufferPart() { Insert(""); } // offset 0 == "" +``` + +- References into it are byte **offsets** (`RDATString`, `RDATStringArray`). +- Strings are null-terminated; the reader simply returns `table + offset`. +- Deduplicated on insert via an `unordered_map`. +- Validation requires the last byte to be `0`. + +### IndexArrays part + +Type `2`. A flat array of `uint32_t`. It stores *many* variable-length index +arrays back to back. Each array is encoded as: + +``` +[ count ][ value_0 ][ value_1 ] ... [ value_{count-1} ] +``` + +- A reference (`IndexArrayRef`, and the array-of-refs used by + `RecordArrayRef`/`RDATStringArray`) is the `uint32_t` **element index** of the + `count` word. +- `IndexTableReader::getRow(i)` returns a row starting at `table[i]` with length + `table[i]`, i.e. it reads the count then exposes the following `count` values. +- Arrays are deduplicated: `IndexArraysPart::AddIndex` appends the new array, + then uses an ordered `std::set` with a custom comparator (`CmpIndices`) that + compares count-then-elements; on a duplicate it rolls back the append and + returns the pre-existing offset. +- Part size in bytes = `4 * number_of_uint32_words`. + +This one pool is used both for numeric arrays (e.g. `NumThreads`, +`SemanticIndices`, `DispatchGrid`) and for arrays of record indices +(`RecordArrayRef`) and arrays of string offsets (`RDATStringArray`). + +### RawBytes part + +Type `5`. A flat blob of arbitrary bytes for binary payloads (root signature +blobs, ViewID masks, PDB data, hashes). + +- Referenced by `BytesRef { uint32_t Offset; uint32_t Size; }` — an 8-byte + field embedded in records. +- Deduplicated by content via `unordered_map`. +- Note: unlike the string buffer, there is no implicit leading entry and no + null terminator; `Size` fully bounds the data. + +### Record tables + +Types `3, 4, 6, 7…`. Each table part begins with a `RuntimeDataTableHeader` +followed by `RecordCount` fixed-size records: + +| Offset | Type | Field | Description | +|-------:|------------|---------------|----------------------------------------------| +| 0 | `uint32_t` | `RecordCount` | Number of records. | +| 4 | `uint32_t` | `RecordStride`| Size of each record in bytes; 4-byte aligned.| +| 8 | `byte[RecordCount * RecordStride]` | (records) | Packed records. | + +Key properties: + +- **Strided for extensibility.** `RecordStride` is stored per table so a newer + compiler can grow a record type (append fields) and older readers still walk + the table correctly. Readers only read fields that fit within both the + compiled struct size *and* the stored stride; `TableReader::Row` returns + `nullptr` if `sizeof(T) > stride`. +- **Deduplication.** When enabled (`m_bDeduplicationEnabled`), identical record + byte-images alias to the same index via an `unordered_map` + keyed on the raw record bytes. Whether dedup is allowed depends on the module + (`GetRecordDuplicationAllowed`). +- **Records contain only fixed-size fields**: scalars, enums (stored as their + underlying integer), fixed arrays, and 4-byte reference handles into the + shared pools (or the 8-byte `BytesRef`). + +`GetPartSize()` returns `0` for an empty table (so it is omitted), otherwise +`sizeof(RuntimeDataTableHeader) + RecordCount * RecordStride`. + +--- + +## Record and field reference model + +Records are **plain-old-data** structs generated by the X-macro system in +[RDAT_Macros.inl](../include/dxc/DxilContainer/RDAT_Macros.inl). The same +`.inl` definitions are expanded in multiple modes (`DEF_RDAT_TYPES_BASIC_STRUCT`, +`DEF_RDAT_TYPES_USE_HELPERS`, reader decl/impl, traits, validation, dump) so the +layout, readers, validators, and dumpers all stay in lockstep. + +Every field maps to one of these on-disk encodings: + +| Macro | On-disk size | Encoding | +|-------------------------------|-------------:|----------------------------------------------------------------| +| `RDAT_VALUE(type, name)` | `sizeof(type)` | Raw scalar (`uint8/16/32`, etc.). | +| `RDAT_VALUE_HEX` | `sizeof(type)` | Same; hint for dumping in hex. | +| `RDAT_ENUM(sTy, eTy, name)` | `sizeof(sTy)` | Enum stored in storage type `sTy` (e.g. `uint32_t`/`uint8_t`).| +| `RDAT_FLAGS(sTy, eTy, name)` | `sizeof(sTy)` | Bit flags stored in `sTy`. | +| `RDAT_STRING(name)` | 4 bytes | `RDATString` = byte offset into StringBuffer. | +| `RDAT_STRING_ARRAY_REF(name)` | 4 bytes | `RDATStringArray` = index-array offset; array of string offsets.| +| `RDAT_INDEX_ARRAY_REF(name)` | 4 bytes | `IndexArrayRef` = index-array offset; array of `uint32_t`. | +| `RDAT_RECORD_REF(type, name)` | 4 bytes | `RecordRef` = record index into that type's table. | +| `RDAT_RECORD_ARRAY_REF(type, name)` | 4 bytes | `RecordArrayRef` = index-array offset; array of record indices. | +| `RDAT_RECORD_VALUE(type, name)` | `sizeof(type)` | Nested record embedded inline (by value). | +| `RDAT_BYTES(name)` | 8 bytes | `BytesRef` = `{ uint32_t Offset; uint32_t Size; }` into RawBytes. | +| `RDAT_ARRAY_VALUE(type,count,...)` | `count*sizeof(type)` | Fixed-size inline array. | + +The sentinel `RDAT_NULL_REF = 0xFFFFFFFF` denotes a null reference for +record/index/string references. + +### Alignment guidance + +From the header comment in `RDAT_Macros.inl`: + +> Pay attention to alignment when organizing structures. `RDAT_STRING` and +> `*_REF` types are always 4 bytes. `RDAT_BYTES` is 2 × 4 bytes. + +Because `RecordStride` must be 4-byte aligned and records are packed with C +struct layout, field order in the `.inl` definitions is chosen to avoid internal +padding surprises. `uint8_t`/`uint16_t` fields are grouped so the record packs +cleanly. + +### Unions + +Records may contain a `union` (`RDAT_UNION` … `RDAT_UNION_END`) whose active +member is selected by another field (typically a `Kind`/`AttribKind` enum). The +`RDAT_UNION_IF`/`RDAT_UNION_ELIF` predicates (e.g. +`getShaderKind() == ShaderKind::Node`) drive the reader accessors, validation, +and dumping so only the active member is interpreted. On disk this is just a +fixed-size union occupying the size of its largest member. + +--- + +## Versioning and compatibility + +RDAT is explicitly built for mixed producer/consumer versions: + +- **Blob version** — `RuntimeDataHeader.Version` (`0x10`). Readers reject + anything below `RDAT_Version_10`. +- **Validator-version gating of parts** — `MaxPartTypeForValVer(Major, Minor, IsPrerelease)` + determines which part types a given validator version may emit: + - `< 1.3` → `Invalid` (no RDAT at all) + - `< 1.4` → up to `Last_1_3` (`FunctionTable`) + - `< 1.8` → up to `Last_1_4` (`SubobjectTable`) + - experimental/prerelease shader model or unbound validator → up to + `LastExperimental` + - otherwise → `LastRelease` (`Last_1_8`) + + The writer only instantiates tables whose `PartType() <= maxAllowedType`. +- **Record growth via stride** — Because each table records its `RecordStride`, + a record type can be *extended* by appending fields. Two mechanisms cooperate: + - **Derived records**: `RDAT_STRUCT_TABLE_DERIVED(type, base, table, Major, Minor)` + defines a record that inherits a base and adds fields, sharing the base's + table (e.g. `RuntimeDataFunctionInfo2` derives from + `RuntimeDataFunctionInfo` and requires validator 1.8). The table's stride is + bumped to the largest supported derived type. Readers upcast based on stride. + - A static assert enforces the core assumption: a derived record type must be + strictly larger than its base (`sizeof(derived) > sizeof(base)`), so stride + comparisons unambiguously indicate which fields are present. +- **Forward-compatible reads** — `TableReader::Row()` returns `nullptr` when + `sizeof(T) > stride`, and record readers invalidate themselves if the stored + size is smaller than the requested record type. Unrecognized part types are + simply skipped by the parser (`default: continue;`). + +--- + +## Record schemas + +The record definitions live in the `RDAT_*Types.inl` files. Below are the +most important ones with their on-disk field encodings. All `*Ref`/string fields +are 4-byte handles; `RDAT_BYTES` is 8 bytes (`Offset`+`Size`). + +### `RuntimeDataResourceInfo` (ResourceTable, v1.3+) + +| Field | Encoding | Meaning | +|--------------|-----------------|------------------------------------------------| +| `Class` | `uint32_t` enum | `hlsl::DXIL::ResourceClass` (SRV/UAV/CBuffer/Sampler) | +| `Kind` | `uint32_t` enum | `hlsl::DXIL::ResourceKind` | +| `ID` | `uint32_t` | Resource ID within its class | +| `Space` | `uint32_t` | Register space | +| `LowerBound` | `uint32_t` | First register | +| `UpperBound` | `uint32_t` | Last register | +| `Name` | `RDATString` | Global name | +| `Flags` | `uint32_t` flags| `DxilResourceFlag` (UAVCounter, ROV, Atomics64, …) | + +### `RuntimeDataFunctionInfo` (FunctionTable, v1.3+) + +| Field | Encoding | Meaning | +|-------------------------|------------------------------|------------------------------------------| +| `Name` | `RDATString` | Full (mangled) function name | +| `UnmangledName` | `RDATString` | Unmangled name | +| `Resources` | `RecordArrayRef` | Global resources used | +| `FunctionDependencies` | `RDATStringArray` | External functions called (by name) | +| `ShaderKind` | `uint32_t` enum | `hlsl::DXIL::ShaderKind` | +| `PayloadSizeInBytes` | `uint32_t` | Ray payload / callable param size | +| `AttributeSizeInBytes` | `uint32_t` | Hit attribute size | +| `FeatureInfo1` | `uint32_t` flags | Low 32 bits of feature flags | +| `FeatureInfo2` | `uint32_t` flags | High 32 bits of feature flags | +| `ShaderStageFlag` | `uint32_t` flags | Valid shader stages (`DxilShaderStageFlags`) | +| `MinShaderTarget` | `uint32_t` (hex) | Encoded minimum shader model target | + +`GetFeatureFlags()`/`SetFeatureFlags()` combine `FeatureInfo1|2` into a 64-bit value. + +### `RuntimeDataFunctionInfo2` (FunctionTable, derived, v1.8+) + +Extends `RuntimeDataFunctionInfo` with: + +| Field | Encoding | Meaning | +|------------------------------|-------------------|------------------------------------------------| +| `MinimumExpectedWaveLaneCount` | `uint8_t` | 0 = unspecified | +| `MaximumExpectedWaveLaneCount` | `uint8_t` | 0 = unspecified | +| `ShaderFlags` | `uint16_t` flags | `DxilShaderFlags` (NodeProgramEntry, UsesViewID, …) | +| *union selected by `ShaderKind`* | 4 bytes each | `RawShaderRef` (uint32) / `RecordRef` to `NodeShaderInfo`, `VSInfo`, `PSInfo`, `HSInfo`, `DSInfo`, `GSInfo`, `CSInfo`, `MSInfo`, `ASInfo` | + +The `uint8/uint8/uint16` grouping keeps the added prefix 4-byte aligned before +the 4-byte union. + +### Node/work-graph records (v1.8+) + +- `NodeID` (NodeIDTable): `Name` (`RDATString`), `Index` (`uint32_t`). +- `RecordDispatchGrid` (inline value): `ByteOffset` (`uint16_t`), + `ComponentNumAndType` (`uint16_t`, bitfields: bits 0:2 num components, + bits 2:15 `ComponentType`). +- `NodeShaderFuncAttrib` (NodeShaderFuncAttribTable): `AttribKind` + (`uint32_t` enum) + union (`RecordRef`, `IndexArrayRef` for + NumThreads/DispatchGrid/MaxDispatchGrid, or `uint32_t` values) selected by + `AttribKind`. +- `NodeShaderIOAttrib` (NodeShaderIOAttribTable): `AttribKind` + union + (record ref, inline `RecordDispatchGrid`, or `uint32_t`). +- `IONode` (IONodeTable): `IOFlagsAndKind` (`uint32_t`, packs + `NodeIOFlags`+`NodeIOKind`), `Attribs` (`RecordArrayRef`). +- `NodeShaderInfo` (NodeShaderInfoTable): `LaunchType` (`uint32_t` enum), + `GroupSharedBytesUsed` (`uint32_t`), `Attribs` + (`RecordArrayRef`), `Outputs`/`Inputs` + (`RecordArrayRef`). + +### Subobject records (SubobjectTable, v1.4+) + +`RuntimeDataSubobjectInfo`: + +| Field | Encoding | Meaning | +|---------|-----------------|--------------------------------------| +| `Kind` | `uint32_t` enum | `hlsl::DXIL::SubobjectKind` | +| `Name` | `RDATString` | Subobject name | +| *union selected by `Kind`* | varies | See below | + +Union members (inline `RDAT_RECORD_VALUE` structs): + +- `StateObjectConfig_t`: `Flags` (`uint32_t` `StateObjectFlags`). +- `RootSignature_t`: `Data` (`RDAT_BYTES`) — used for both global and local RS. +- `SubobjectToExportsAssociation_t`: `Subobject` (`RDATString`), `Exports` + (`RDATStringArray`). +- `RaytracingShaderConfig_t`: `MaxPayloadSizeInBytes`, + `MaxAttributeSizeInBytes` (`uint32_t` each). +- `RaytracingPipelineConfig_t`: `MaxTraceRecursionDepth` (`uint32_t`). +- `HitGroup_t`: `Type` (`uint32_t` `HitGroupType`), `AnyHit`, `ClosestHit`, + `Intersection` (`RDATString` each). +- `RaytracingPipelineConfig1_t`: `MaxTraceRecursionDepth` (`uint32_t`), + `Flags` (`uint32_t` `RaytracingPipelineFlags`). + +### Signature / per-stage records (experimental) + +- `SignatureElement` (SignatureElementTable): `SemanticName` (`RDATString`), + `SemanticIndices` (`IndexArrayRef`), `SemanticKind`/`ComponentType`/ + `InterpolationMode` (`uint8_t` enums), `StartRow` (`uint8_t`, `0xFF` = not + allocated), `ColsAndStream` (`uint8_t` bitfield: cols-1, start col, output + stream), `UsageAndDynIndexMasks` (`uint8_t` bitfield). +- `VSInfo`/`PSInfo`/`HSInfo`/`DSInfo`/`GSInfo`/`MSInfo`: `RecordArrayRef` + for input/output (and patch-const/prim) signatures, `RDAT_BYTES` ViewID and + input-to-output masks, plus stage-specific scalars (control-point counts, + tessellator domain/primitive, primitive topology, max vertex count, etc.). +- `CSInfo`/`ASInfo`: `NumThreads` (`IndexArrayRef`), `GroupSharedBytesUsed` + (and `PayloadSizeInBytes` for AS). + +### PDB info records (PdbInfo group) + +- `DxilPdbInfoLibrary` (DxilPdbInfoLibraryTable): `Name` (`RDATString`), + `Data` (`RDAT_BYTES`). +- `DxilPdbInfoSource` (DxilPdbInfoSourceTable): `Name`, `Content` (`RDATString`). +- `DxilPdbInfo` (DxilPdbInfoTable): `Sources` + (`RecordArrayRef`), `Libraries` + (`RecordArrayRef`), `ArgPairs` (`RDATStringArray`), + `Hash` (`RDAT_BYTES`), `PdbName` (`RDATString`), `CustomToolchainId` + (`uint32_t`), `CustomToolchainData` (`RDAT_BYTES`), `WholeDxil` (`RDAT_BYTES`). + +--- + +## Reading and validation + +### Parsing (`DxilRuntimeData::InitFromRDAT`) + +The parser uses a bounds-checked `CheckedReader` that throws on overrun/overlap: + +1. Read `RuntimeDataHeader`; reject `Version < RDAT_Version_10`. +2. Read the `PartCount` offsets. +3. For each part: `Advance(offset)`, read `RuntimeDataPartHeader`, then bound a + sub-reader to `part.Size` bytes. +4. Dispatch on `part.Type`: + - `StringBuffer` → init `StringTableReader`. + - `IndexArrays` → init `IndexTableReader` with `Size/4` words. + - `RawBytes` → init `RawBytesReader`. + - Any table type → `InitTable` reads a `RuntimeDataTableHeader` and binds a + `TableReader(data, RecordCount, RecordStride)`. + - Unknown → skipped (`default: continue;`). +5. In debug builds, `Validate()` is run automatically. + +### Access + +`RDATContext` holds the three buffer readers plus a `TableReader` per +`RecordTableIndex`. Typed access is generated per record type: + +- `RecordTableReader` iterates a table; `Row(i)` returns a reader. +- `RecordReader` validates that the stored stride/size is large enough for + the requested record type before exposing fields; otherwise it becomes a null + reader. +- Field accessors resolve handles: string offsets → `StringBuffer.Get`, index + refs → `IndexTable.getRow`, record refs → `Table().Row`, byte refs → + `RawBytes.Get` with `Size`. + +### Validation (`DxilRuntimeData::Validate`) + +- The string buffer, if present, must end in `'\0'`. +- Every table is walked; for each record, all references are bounds-checked: + - `ValidateRecordRef` — index `< table.Count()` (or `RDAT_NULL_REF`). + - `ValidateIndexArrayRef` — offset in range and the encoded array length fits + within the remaining index space. + - `ValidateRecordArrayRef` / `ValidateStringArrayRef` — the index array is + well-formed and each element is itself a valid record/string ref. + - `ValidateStringRef` — offset `< StringBuffer.Size()`. +- `RecursiveRecordValidator` re-validates each record at every version up to the + one supported by the table stride, so extended (derived) fields are checked + too. A static assert guarantees `sizeof(derived) > sizeof(base)`. + +--- + +## Appendix: Key constants + +| Name | Value | Meaning | +|----------------------|--------------|--------------------------------------------| +| `DFCC_RuntimeData` | `'RDAT'` | Container FourCC for the RDAT part | +| `RDAT_Version_10` | `0x10` | Current RDAT blob version | +| `RDAT_NULL_REF` | `0xFFFFFFFF` | Null record/index/string reference | +| `PSVALIGN4(x)` | `(x+3)&~3` | 4-byte alignment used for part/record sizes | +| `RuntimeDataGroup::Core` | `0` | Default part group | +| `RuntimeDataGroup::PdbInfo` | `1` | PDB info part group (packed into high 16 bits) |