Skip to content

Behavior evidence gap: linking agent actions back to BOM components #1015

Description

@telleroutlook

Context

I'm one of the maintainers of WasmAgent/agentbom — a reference implementation library for describing AI agent inventories (tools, permission scopes, model dependencies, compliance posture, audit trail) as a single verifiable document. We've been tracking the CycloneDX Agent BOM proposal and have been building against it as an early implementer.

Following Steve Springett's suggestion on the existing thread, I'm opening this as a dedicated issue to track the specific gap I raised.


The gap: behavior evidence is not addressable in the current design

The current Blueprints RFC defines runtime evidence — attesting what an agent is (which components are loaded, which version is running). This is valuable and we implement it via our agentbom.evidence_layer.evidence_hashes field.

But in practice, the harder problem for AI agents is attesting what an agent did — specifically:

  • Which tool calls were made, in what order, with what inputs
  • Which memory cells were read/written (and whether they were tampered with between reads)
  • Whether a checkpoint/fork occurred, and from what parent state
  • Which policy bundle was active at the moment of a decision

These are not runtime inventory facts — they're behavioral trace records that need to be linked back to the BOM so a verifier can say: "the agent that ran this task was operating under this policy bundle, with these tools, and its actions are provably this sequence."


What we've built (implementation experience)

In wasmagent-protocol, we define the Agent Evidence Protocol (AEP) — a set of JSON schemas for:

Schema What it attests
aep-record A single agent action — links run_id, model_id, policy_bundle_digest, tool_manifest_digest, input/output refs with taint labels
checkpoint-evidence Verifiable resume point — state_digest, fork_of parent checkpoint, tamper-evident via ed25519 signature
memory-evidence Read/write ops with hash-chain (prior_digestdigest) making historical tampering detectable
replay-evidence Post-hoc replay verification record

In agentbom, each BOM document carries an evidence_layer with aep_references (AEP record IDs or hashes), plus an audit_log of structured events (tool_call, permission_check, prompt_injection_attempt). The BOM thus serves as the static inventory anchor that behavioral trace records reference back to.

The key structural insight from our implementation: behavior evidence doesn't need to live inside the BOM — it needs a stable reference point in the BOM (tool manifest digest, policy bundle digest, model snapshot) so that an external evidence record can prove it was generated under those exact conditions.


Concrete proposal for consideration

Before the August 20 TC54 vote, I'd suggest considering whether the Blueprints design should define:

  1. A behaviorEvidence reference type — a pointer (hash + URI) from a BOM component to an external behavioral trace record, analogous to how SBOM components carry externalReferences
  2. Stable digest anchors — explicit fields in the agent component for toolManifestDigest and policyBundleDigest, so external evidence records can reference a specific BOM snapshot
  3. Evidence envelope conventions — even if the trace format is out of scope for CycloneDX, defining the envelope schema (schema_version, trace_id, created_at_ms, optional signature) would let implementers build interoperable evidence records

These are additive — they don't require the trace format itself to be standardized, just the linking mechanism.


Why this matters for AI agents specifically

Software SBOM consumers already have mature tooling for runtime evidence (CVE scanners, SBOM diff). AI agent consumers face a different threat model: prompt injection, tool misuse, memory poisoning, and checkpoint replay attacks — all of which require behavioral evidence to detect or audit post-hoc. An agent BOM that can only attest "these tools were loaded" but cannot anchor "and these are the actions that were taken under this tool set" leaves a critical gap for the primary AI-specific risk scenarios.

Happy to share our schema files or discuss further.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions