Make public header components acyclic, and forbid new cycles

[Amaury Chamayou (achamayou)]

Motivation

Follow-up to #8476, for #3517.

#8476 made the src/ component graph acyclic, but check-source-dependencies.py only scans src/. Under include/ccf/, three groups of public header components still include each other in cycles. For example, ccf/ds/json.h includes ccf/crypto/base64.h, and ccf/crypto/sha256_hash.h includes ccf/service/map.h for a single serialiser, so the lowest-level headers pull in the whole typed KV API. Nothing stops new cycles either.

ccf/research/create_tx_claims_digest.h also included the private <kv/kv_types.h>, which is not installed, so applications could not use it. includes-checks.sh only checks quoted includes, so it missed this.

The resulting public header layering, and the includes this PR removes

Each node is a public header component, a directory under include/ccf/. + lists the top-level headers that an override assigns to a component. Other top-level headers are left out: an arrow means that a component's headers include another's, directly or through top-level headers. As in #8476, an arrow is omitted when a longer path implies it.

The dashed red arrows are the includes this PR removes. Three pointed up the layering and closed the cycles between crypto, ds, kv, pal and service, and one reached into a private header. Their replacements follow existing arrows: crypto -> ds for base64, kv -> crypto for the serialiser, and service -> pal for DID and Feed. The cycles through endpoint.h and node_context.h are resolved by the overrides instead.

flowchart TD
  research --> node
  research --> endpoints
  js --> endpoints
  js --> indexing
  node --> service
  endpoints --> service
  strategies --> indexing
  indexing --> kv
  service --> pal
  pal --> kv
  kv --> crypto
  crypto --> ds
  ds --> threading
  ds -. "json.h: base64" .-> crypto
  crypto -. "sha256_hash.h: BlitSerialiser" .-> service
  pal -. "uvm_endorsements.h: DID, Feed" .-> service
  research -. "create_tx_claims_digest.h" .-> private_kv
  endpoints["endpoints<br/>+ endpoint.h, endpoint_context.h,<br/>common_auth_policies.h"]
  strategies["indexing/strategies"]
  kv["kv<br/>+ tx.h"]
  private_kv["src/kv (private)"]
  classDef private stroke-dasharray:3 3
  class private_kv private
  linkStyle 13,14,15,16 stroke:#d1242f,stroke-width:2px,stroke-dasharray:5 5

Loading

Implementation summary

Public headers are checked as their own layer, not as part of the src/ component of the same name. A component's interface can legitimately be used by components that its implementation depends on: ccf/node_context.h exposes the indexing interface, while src/indexing depends on node. The checker now also requires that:

• public headers only include public headers, quoted or angled, and
• public header components form no cycle. Each directory under include/ccf/ is a component and each top-level header is its own, unless public_component_overrides in source-dependencies.json says otherwise.

Since public headers cannot include private ones and the src/ policy is acyclic, no cycle can span the two layers, so the whole include graph between components is acyclic.

Three declarations move down a layer to break the cycles. No names, namespaces or signatures change:

• base64 (ds -> crypto): the ccf::crypto base64 declarations move to a new ccf/ds/base64.h, which ccf/crypto/base64.h includes. json.h must keep encoding std::vector<uint8_t> as base64, so this dependency can only move, not disappear.
• BlitSerialiser<Sha256Hash> (crypto -> service): the specialisation moves next to its primary template in ccf/kv/serialisers/blit_serialiser.h, so ccf/crypto/sha256_hash.h no longer includes ccf/service/map.h. It is now visible wherever BlitSerialiser is.
• ccf::DID and ccf::Feed (pal -> service): these std::string aliases were all that ccf/pal/uvm_endorsements.h used from ccf/service/tables/uvm_endorsements.h. They move to the pal header, which the table header now includes.

Five ownership overrides settle the remaining cycles without code changes. endpoint.h, endpoint_context.h and common_auth_policies.h belong to endpoints, as ccf/endpoints/authentication/js.h builds on them. tx.h belongs to kv. ccf/indexing/strategies/ is its own component, as strategies take an AbstractNodeContext.

To limit source breaks:

• ccf/tx.h includes ccf/kv/map.h, set.h, unit_value.h and value.h, which it used to provide transitively. It also includes ccf/kv/abstract_handle.h, which it always needed: it destroys a std::unique_ptr<AbstractHandle> of a forward-declared type, and only compiled because the full type arrived transitively.
• ccf/research/create_tx_claims_digest.h includes ccf/claims_digest.h and ccf/tx.h instead of the private header.
• src/node/internal_tables_access.h includes the UVM endorsements table header it uses.

The two scans share their include parsing, and report source and public violations together, with one include per edge of a cycle. On main, the checker reports:

Public headers include private headers:
  include/ccf/research/create_tx_claims_digest.h:5: #include <kv/kv_types.h>
Public header components form a cycle: ccf/crypto -> ccf/service -> ccf/entity_id.h -> ccf/kv -> ccf/byte_vector.h -> ccf/ds -> ccf/crypto
  include/ccf/crypto/sha256_hash.h:6: #include "ccf/service/map.h"
  include/ccf/service/tables/members.h:7: #include "ccf/entity_id.h"
  include/ccf/entity_id.h:6: #include "ccf/kv/serialisers/blit_serialiser.h"
  include/ccf/kv/serialisers/serialised_entry.h:4: #include "ccf/byte_vector.h"
  include/ccf/byte_vector.h:5: #include "ccf/ds/siphash.h"
  include/ccf/ds/json.h:4: #include "ccf/crypto/base64.h"

Validated locally:

• All 201 CCF and sample source files of a Debug configuration compile with their build flags, including -Werror (clang 21, syntax only), as on main. Only declarations moved, so nothing was linked or run; CI covers the tests.
• Compiled on their own, the same 8 public headers fail as on main (for example, ccf/rest_verb.h uses fmt without including it). ccf/pal/uvm_endorsements.h and ccf/research/create_tx_claims_digest.h failed on main and now compile, as does the new ccf/ds/base64.h.
• Checker mutations are all rejected, with the offending lines: reintroducing the pal -> service include, an angled private include, an unresolved ccf/ include, a macro include, unknown overrides, and dropping the tx.h override.
• includes-checks.sh, clang-format 18.1.8, black, prettier, and the release notes, ASCII, copyright and TODO checks pass.

Safety and compatibility

Runtime behaviour does not change. Every moved declaration and definition is unchanged, so there is no ABI, wire, ledger, KV or JSON format change, and no effect on mixed-version operation or recovery. Sha256Hash keys are still serialised as hex, and byte vectors as base64 in JSON.

This is a source-only change, for code that relied on transitive includes. CHANGELOG.md lists it:

• ccf/tx.h, and headers that include it such as ccf/rpc_context.h and ccf/node_context.h, no longer provide ccf::ServiceMap, ccf::ServiceValue, ccf::ServiceSet or ccf::ServiceUnit.
• ccf/crypto/sha256_hash.h, ccf/crypto/hash_provider.h, ccf/claims_digest.h, ccf/receipt.h, ccf/service/local_sealing.h and several ccf/pal/ headers no longer provide the typed KV maps.
• ccf/app_interface.h, ccf/endpoint_registry.h, ccf/common_endpoint_registry.h and ccf/json_handler.h still provide everything they did, and the samples compile unchanged.

Losing transitive includes cannot silently change behaviour here. The headers that some includers lose contain only two specialisations, fmt::formatter<ccf::ByteVector> and BlitSerialiser<Sha256Hash>, and each is in the header that declares its type or its primary template. Code that can name the type still sees the specialisation; anything else fails to compile.

This lands in a patch release: 7.0.18 already contains a documented source break (#8309).

[Copilot]

Copilot review overview

🟡 Changes recommended

Four new changelog entries lack the required #8478 pull-request reference.

Review effort: Balanced
Findings: 2

Open (2)

• Missing introducing PR references in three changelog entries · New
• Missing introducing PR reference in changelog entry · New

What changed in this PR

Extends dependency enforcement to public headers and removes existing component cycles without changing runtime behavior.

Changes:

• Adds public-header dependency and private-include validation.
• Relocates declarations and adds compatibility includes.
• Updates release notes and checker documentation.

Custom instructions used: .github/copilot-instructions.md, .github/instructions/reviewing.instructions.md, .github/instructions/changelog.instructions.md, and the formatting/testing skills.

File	Description
.github/​skills/​formatting-and-linting/​SKILL.md	Updates include-check coverage.
CHANGELOG.md	Documents header and dependency changes.
include/​ccf/​crypto/​base64.h	Forwards to the new DS declaration header.
include/​ccf/​crypto/​sha256_hash.h	Removes the KV dependency.
include/​ccf/​ds/​base64.h	Adds relocated base64 declarations.
include/​ccf/​ds/​json.h	Uses the lower-layer base64 header.
include/​ccf/​kv/​serialisers/​blit_serialiser.h	Relocates the SHA-256 specialization.
include/​ccf/​pal/​uvm_endorsements.h	Owns DID and Feed aliases.
include/​ccf/​research/​create_tx_claims_digest.h	Replaces a private include.
include/​ccf/​service/​tables/​uvm_endorsements.h	Imports aliases from PAL.
include/​ccf/​tx.h	Makes required and compatibility includes explicit.
scripts/​check-source-dependencies.py	Checks public-header layering and cycles.
scripts/​includes-checks.sh	Describes expanded dependency checks.
scripts/​source-dependencies.json	Adds public component ownership overrides.
src/​node/​internal_tables_access.h	Adds its direct table dependency.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.