Course: Course 2B — Securing & Attacking Harnesses and LLMs
Module: SDD-B07 — Agent SBOM and Supply Chain Assessment
Duration: 45–60 minutes
Environment: Python 3.10+. No GPU, no network, no external API keys. The agent configuration and the runtime snapshot are FIXTURE (provided) so the lab runs deterministically offline. A text editor and python3 -m json.tool for validation.
By the end of this lab you will have:
This lab is the concrete inventory practice that makes B4's trust surfaces and B11's governance obligations operable. You are building and assessing the artifact, not running an agent.
mkdir sdd-b07-lab && cd sdd-b07-lab
python3 --version # 3.10+ required
No venv, no dependencies, no GPU. The entire lab is self-contained Python with type hints. The agent config and the runtime snapshot are fixtures so the lab runs deterministically offline.
Build the data model and the generator that turns an agent's config-as-code into a CycloneDX AI BOM.
Create aibom.py:
from dataclasses import dataclass, field
from typing import Literal, Optional
import json
import hashlib
ComponentType = Literal[
"machine-learning-model", # CycloneDX ML-BOM extension
"data", # training/eval/retrieval data
"application", # function-call tools, MCP servers
"library", # harness/framework deps
"framework",
]
@dataclass
class ModelComponent:
"""An AI-BOM model component (NTIA: model identity, architecture, params)."""
name: str
version: str # PINNED checkpoint, not a floating alias
supplier: str
checkpoint_hash: str # proves the specific artifact
architecture: str # transformer, diffusion, etc.
parameter_count: str # e.g. "70B"
modalities: list[str] # input/output modalities
inference_runtime: str # vLLM, llama.cpp, TGI
model_card_ref: Optional[str] = None
fine_tune_adapters: list[dict] = field(default_factory=list)
@dataclass
class DataComponent:
"""An AI-BOM data component (NTIA: training data, provenance, licensing)."""
name: str
kind: Literal["training", "evaluation", "retrieval"]
supplier: str
provenance: str # where it came from
license: str
data_card_ref: Optional[str] = None
@dataclass
class ToolComponent:
"""An AI-BOM tool/MCP component. These ARE the attack surface (B4)."""
name: str
kind: Literal["function_tool", "mcp_server"]
supplier: str
transport: Optional[str] = None # for MCP servers: stdio, sse, websocket
exposes_tools: list[str] = field(default_factory=list) # MCP: tools it exposes
backing_implementation: Optional[str] = None
@dataclass
class HarnessComponent:
"""An AI-BOM harness/orchestration component (traditional SBOM, security-critical)."""
name: str
version: str
supplier: str
package_url: str # purl
slsa_attestation: Optional[str] = None # build provenance
@dataclass
class AgentConfig:
"""The agent's config-as-code — the source of truth for the AI BOM."""
model: ModelComponent
data: list[DataComponent]
tools: list[ToolComponent]
harness: list[HarnessComponent]
def _bom_ref(prefix: str, name: str, version: str) -> str:
return f"{prefix}:{name}@{version}"
def _component_entry(
bom_ref: str,
comp_type: ComponentType,
name: str,
version: str,
supplier: str,
properties: list[dict],
external_refs: Optional[list[dict]] = None,
) -> dict:
entry = {
"type": comp_type,
"bom-ref": bom_ref,
"name": name,
"version": version,
"supplier": {"name": supplier},
"properties": properties,
}
if external_refs:
entry["externalReferences"] = external_refs
return entry
def generate_aibom(config: AgentConfig, serial: str = "urn:uuid:sdd-b07-lab") -> dict:
"""Generate a CycloneDX AI BOM from the agent config.
The software SBOM enumerates libraries/frameworks. The AI BOM EXTENDS it
with machine-learning-model, data, and application (tool/MCP) components.
Every component is a trust dependency an attacker can reach.
"""
components: list[dict] = []
# Model component (CycloneDX ML-BOM type)
m = config.model
components.append(_component_entry(
_bom_ref("model", m.name, m.version),
"machine-learning-model",
m.name, m.version, m.supplier,
properties=[
{"name": "cdx:ai:model:architecture", "value": m.architecture},
{"name": "cdx:ai:model:parameters", "value": m.parameter_count},
{"name": "cdx:ai:model:checkpoint_hash", "value": m.checkpoint_hash},
{"name": "cdx:ai:model:modalities", "value": ",".join(m.modalities)},
{"name": "cdx:ai:model:inference_runtime", "value": m.inference_runtime},
],
external_refs=[{"type": "website", "url": m.model_card_ref}] if m.model_card_ref else None,
))
# Data components
for d in config.data:
components.append(_component_entry(
_bom_ref("data", d.name, d.kind),
"data",
d.name, d.kind, d.supplier,
properties=[
{"name": "cdx:ai:data:provenance", "value": d.provenance},
{"name": "cdx:ai:data:license", "value": d.license},
],
external_refs=[{"type": "website", "url": d.data_card_ref}] if d.data_card_ref else None,
))
# Tool & MCP components (these ARE the attack surface)
for t in config.tools:
props = [{"name": "ai:bom:tool:kind", "value": t.kind}]
if t.transport:
props.append({"name": "ai:bom:mcp:transport", "value": t.transport})
if t.exposes_tools:
props.append({"name": "ai:bom:mcp:exposes", "value": ",".join(t.exposes_tools)})
components.append(_component_entry(
_bom_ref("tool", t.name, t.kind),
"application",
t.name, t.kind, t.supplier,
properties=props,
))
# Harness components (traditional SBOM, security-critical for an agent)
for h in config.harness:
ext = []
if h.slsa_attestation:
ext.append({"type": "other", "url": h.slsa_attestation,
"comment": "SLSA build provenance"})
components.append(_component_entry(
_bom_ref("pkg", h.name, h.version),
"library",
h.name, h.version, h.supplier,
properties=[{"name": "purl", "value": h.package_url}],
external_refs=ext or None,
))
return {
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"serialNumber": serial,
"version": 1,
"metadata": {"component": {"type": "application", "name": "sample-agent"}},
"components": components,
}
Construct an AgentConfig for a representative agent: a fine-tuned model, two data components (one training, one retrieval), three function-call tools, two MCP servers, and two harness libraries. At least one harness component should have SLSA attestation; at least one should not (you will catch this in validation).
Write the config to agent_config.json (serialize the dataclasses) and generate the BOM:
bom = generate_aibom(config)
with open("aibom.json", "w") as f:
json.dump(bom, f, indent=2)
# validate it parses
python3 -m json.tool aibom.json > /dev/null
Build the validator that compares the AI BOM against what the agent actually loads at runtime.
@dataclass
class RuntimeSnapshot:
"""What the agent ACTUALLY loads at startup. The validation target."""
resolved_model_version: str # may drift from the BOM's pinned version
resolved_checkpoint_hash: str
registered_tools: list[str] # tool names the agent registered
connected_mcp_servers: list[str] # MCP servers the agent connected to
loaded_libraries: list[dict] # {name, version} for harness deps
timestamp: str
A runtime snapshot fixture is provided in runtime_snapshot.json. It INTENTIONALLY diverges from the BOM:
FindingSeverity = Literal["critical", "high", "medium", "low", "info"]
@dataclass
class Finding:
check: Literal["completeness", "version_drift", "provenance", "vuln_policy"]
severity: FindingSeverity
component: str
description: str
b4_surface: str # which B4 trust surface this maps to
def validate_aibom(bom: dict, runtime: RuntimeSnapshot) -> list[Finding]:
"""Run the four validation checks. Return findings (empty = clean).
1. COMPLETENESS — every tool/MCP in runtime must be in the BOM (undeclared?).
2. VERSION DRIFT — BOM-pinned model version == runtime-resolved version?
3. PROVENANCE — do model + harness components carry attestation?
4. VULN/POLICY — (simulated) any known-issue or policy-violation flags?
"""
findings: list[Finding] = []
# ... implement the four checks (see 2.3)
return findings
Check 1 — Completeness (undeclared components). For each MCP server in runtime.connected_mcp_servers, check it appears in the BOM's tool components. Any runtime server NOT in the BOM is an UNDECLARED COMPONENT finding. Severity scales with reachability — assume every connected server is contacted every turn (CRITICAL). Map the finding to B4's "MCP / indirect-injection surface."
bom_tool_names = {c["name"] for c in bom["components"] if c["type"] == "application"}
for server in runtime.connected_mcp_servers:
if server not in bom_tool_names:
findings.append(Finding(
check="completeness",
severity="critical",
component=server,
description=(f"UNDECLARED MCP server '{server}' connected at runtime "
f"but absent from the AI BOM. Un-assessed indirect-injection "
f"source (SDD-B03). Agent equivalent of unknown transitive dep."),
b4_surface="MCP / indirect-injection surface",
))
Also check the reverse: any BOM-declared tool NOT in the runtime is a STALE entry (lower severity — medium).
Check 2 — Version drift. Find the model component in the BOM; compare its version to runtime.resolved_model_version and its checkpoint hash to runtime.resolved_checkpoint_hash. A mismatch is a MODEL-VERSION DRIFT finding (HIGH). Map to B4's "model / refusal-layer surface."
Check 3 — Provenance. For the model component and each harness component, check for an SLSA attestation (in externalReferences with comment "SLSA build provenance") or a supplier checksum property. An un-attested model is CRITICAL (scaling attack); an un-attested harness library is MEDIUM.
Check 4 — Vulnerability/policy (simulated). For this lab, simulate a policy check: any tool whose supplier is not in an approved-supplier set is a MEDIUM finding. Define APPROVED_SUPPLIERS = {"acme-internal", "openai", "anthropic", "owasp"} and flag any tool/MCP from a supplier not in the set.
runtime = load_runtime_snapshot("runtime_snapshot.json")
findings = validate_aibom(bom, runtime)
for f in findings:
print(f"[{f.severity.upper()}] {f.check}: {f.component} -> {f.description}")
You should see at least: one CRITICAL undeclared MCP server, one HIGH model-version drift, one CRITICAL un-attested model (or MEDIUM un-attested library), and the stale-entry and policy findings.
Produce findings_report.json — the format a B12 engagement deliverable would carry:
def build_finding_report(findings: list[Finding], bom: dict, runtime: RuntimeSnapshot) -> dict:
"""Structure the findings for the engagement deliverable."""
undeclared = [f for f in findings if f.check == "completeness" and f.severity == "critical"]
drift = [f for f in findings if f.check == "version_drift"]
return {
"summary": {
"total_findings": len(findings),
"critical": sum(1 for f in findings if f.severity == "critical"),
"high": sum(1 for f in findings if f.severity == "high"),
"medium": sum(1 for f in findings if f.severity == "medium"),
},
"headline_finding": (
f"{len(undeclared)} undeclared component(s) and "
f"{len(drift)} model-version drift finding(s). "
f"The assessed agent is NOT the deployed agent."
),
"findings": [
{
"check": f.check,
"severity": f.severity,
"component": f.component,
"description": f.description,
"b4_trust_surface": f.b4_surface,
}
for f in findings
],
"b4_surface_map": build_b4_surface_map(bom, runtime),
}
The headline finding is the delta between the BOM and the runtime. An undeclared MCP server is an un-assessed indirect-injection source; model-version drift means the assessed agent is not the deployed agent. This report is the supply-chain deliverable of the engagement.
def build_b4_surface_map(bom: dict, runtime: RuntimeSnapshot) -> dict:
"""Map each BOM component to its B4 trust-surface node.
This is the bridge: the AI BOM populates B4's trust-surface map.
Each component is a node; the engagement scopes against these nodes.
"""
surfaces = {
"model_refusal_surface": [],
"function_call_surface": [],
"mcp_injection_surface": [],
"data_leak_surface": [],
"harness_vuln_surface": [],
}
for c in bom["components"]:
if c["type"] == "machine-learning-model":
surfaces["model_refusal_surface"].append(c["name"])
elif c["type"] == "data":
surfaces["data_leak_surface"].append(c["name"])
elif c["type"] == "application":
kind = next((p["value"] for p in c.get("properties", [])
if p["name"] == "ai:bom:tool:kind"), "function_tool")
if kind == "mcp_server":
surfaces["mcp_injection_surface"].append(c["name"])
else:
surfaces["function_call_surface"].append(c["name"])
elif c["type"] in ("library", "framework"):
surfaces["harness_vuln_surface"].append(c["name"])
# Flag runtime-only surfaces (undeclared) separately
bom_mcp = {c["name"] for c in bom["components"]
if c["type"] == "application"
and any(p["name"] == "ai:bom:tool:kind" and p["value"] == "mcp_server"
for p in c.get("properties", []))}
undeclared_mcp = set(runtime.connected_mcp_servers) - bom_mcp
if undeclared_mcp:
surfaces["UNDECLARED_mcp_injection_surface"] = sorted(undeclared_mcp)
return surfaces
The AI BOM populates B4's trust-surface map. The model maps to the refusal surface (SDD-B06). The function-call tools map to the function-call surface (SDD-B04). The MCP servers map to the indirect-injection surface (SDD-B03). The data components map to the data-leak surface. The harness libraries map to the traditional vulnerability surface. The undeclared MCP servers appear as a separate UN-ASSESSED surface — the finding.
Write detect_bom_drift(bom_at_t0: dict, bom_at_t1: dict) -> dict that compares two BOMs generated at different times and reports what changed: added components, removed components, version changes. The point: a production agent that adds a tool, connects to a new MCP server, or updates its model changes its AI BOM. The drift detector is the continuous-monitoring signal — every change is a re-assessment trigger.
aibom.py — the data model and CycloneDX generator (Phase 1)aibom.json — the generated AI BOM for the sample agent (Phase 1)validator.py — the four-check validator + runtime snapshot loader (Phase 2)findings_report.json — the undeclared-component and drift finding report (Phase 3)surface_map.json — the B4 trust-surface map from the AI BOM (Phase 4)drift_detector.py — the BOM-to-BOM drift detector (Phase 5)aibom.json) is valid CycloneDX 1.6 JSON, declares the model, data, tool/MCP, and harness components, and carries the NTIA minimum-element fields (architecture, parameters, checkpoint hash, provenance, licensing).# Lab Specification — SDD-B07: Agent SBOM and Supply Chain Assessment
**Course**: Course 2B — Securing & Attacking Harnesses and LLMs
**Module**: SDD-B07 — Agent SBOM and Supply Chain Assessment
**Duration**: 45–60 minutes
**Environment**: Python 3.10+. No GPU, no network, no external API keys. The agent configuration and the runtime snapshot are FIXTURE (provided) so the lab runs deterministically offline. A text editor and `python3 -m json.tool` for validation.
---
## Learning objectives
By the end of this lab you will have:
1. **Generated an AI BOM** (CycloneDX JSON) for a representative agent from its configuration — enumerating the model, data, tool/MCP, and harness components with the NTIA minimum-element fields.
2. **Validated the AI BOM against a runtime snapshot**, running the four checks (completeness, version drift, provenance, vulnerability/policy) and producing structured findings.
3. **Detected undeclared components** — a tool and an MCP server the agent uses at runtime but the BOM does not declare — and classified their severity by reachability.
4. **Detected model-version drift** — the BOM-declared version differs from the runtime-resolved version — and explained why the assessed agent is not the deployed agent.
5. **Mapped the AI BOM's components to B4's trust surfaces**, producing the trust-surface node list that feeds a red-team engagement scope.
6. **Produced an undeclared-component finding report** in the format a B12 engagement deliverable would carry.
This lab is the concrete inventory practice that makes B4's trust surfaces and B11's governance obligations operable. You are building and assessing the artifact, not running an agent.
---
## Phase 0 — Setup (3 min)
```bash
mkdir sdd-b07-lab && cd sdd-b07-lab
python3 --version # 3.10+ required
```
No venv, no dependencies, no GPU. The entire lab is self-contained Python with type hints. The agent config and the runtime snapshot are fixtures so the lab runs deterministically offline.
---
## Phase 1 — The agent configuration and the CycloneDX AI BOM generator (12 min)
Build the data model and the generator that turns an agent's config-as-code into a CycloneDX AI BOM.
### 1.1 The data model
Create `aibom.py`:
```python
from dataclasses import dataclass, field
from typing import Literal, Optional
import json
import hashlib
ComponentType = Literal[
"machine-learning-model", # CycloneDX ML-BOM extension
"data", # training/eval/retrieval data
"application", # function-call tools, MCP servers
"library", # harness/framework deps
"framework",
]
@dataclass
class ModelComponent:
"""An AI-BOM model component (NTIA: model identity, architecture, params)."""
name: str
version: str # PINNED checkpoint, not a floating alias
supplier: str
checkpoint_hash: str # proves the specific artifact
architecture: str # transformer, diffusion, etc.
parameter_count: str # e.g. "70B"
modalities: list[str] # input/output modalities
inference_runtime: str # vLLM, llama.cpp, TGI
model_card_ref: Optional[str] = None
fine_tune_adapters: list[dict] = field(default_factory=list)
@dataclass
class DataComponent:
"""An AI-BOM data component (NTIA: training data, provenance, licensing)."""
name: str
kind: Literal["training", "evaluation", "retrieval"]
supplier: str
provenance: str # where it came from
license: str
data_card_ref: Optional[str] = None
@dataclass
class ToolComponent:
"""An AI-BOM tool/MCP component. These ARE the attack surface (B4)."""
name: str
kind: Literal["function_tool", "mcp_server"]
supplier: str
transport: Optional[str] = None # for MCP servers: stdio, sse, websocket
exposes_tools: list[str] = field(default_factory=list) # MCP: tools it exposes
backing_implementation: Optional[str] = None
@dataclass
class HarnessComponent:
"""An AI-BOM harness/orchestration component (traditional SBOM, security-critical)."""
name: str
version: str
supplier: str
package_url: str # purl
slsa_attestation: Optional[str] = None # build provenance
@dataclass
class AgentConfig:
"""The agent's config-as-code — the source of truth for the AI BOM."""
model: ModelComponent
data: list[DataComponent]
tools: list[ToolComponent]
harness: list[HarnessComponent]
```
### 1.2 The CycloneDX generator
```python
def _bom_ref(prefix: str, name: str, version: str) -> str:
return f"{prefix}:{name}@{version}"
def _component_entry(
bom_ref: str,
comp_type: ComponentType,
name: str,
version: str,
supplier: str,
properties: list[dict],
external_refs: Optional[list[dict]] = None,
) -> dict:
entry = {
"type": comp_type,
"bom-ref": bom_ref,
"name": name,
"version": version,
"supplier": {"name": supplier},
"properties": properties,
}
if external_refs:
entry["externalReferences"] = external_refs
return entry
def generate_aibom(config: AgentConfig, serial: str = "urn:uuid:sdd-b07-lab") -> dict:
"""Generate a CycloneDX AI BOM from the agent config.
The software SBOM enumerates libraries/frameworks. The AI BOM EXTENDS it
with machine-learning-model, data, and application (tool/MCP) components.
Every component is a trust dependency an attacker can reach.
"""
components: list[dict] = []
# Model component (CycloneDX ML-BOM type)
m = config.model
components.append(_component_entry(
_bom_ref("model", m.name, m.version),
"machine-learning-model",
m.name, m.version, m.supplier,
properties=[
{"name": "cdx:ai:model:architecture", "value": m.architecture},
{"name": "cdx:ai:model:parameters", "value": m.parameter_count},
{"name": "cdx:ai:model:checkpoint_hash", "value": m.checkpoint_hash},
{"name": "cdx:ai:model:modalities", "value": ",".join(m.modalities)},
{"name": "cdx:ai:model:inference_runtime", "value": m.inference_runtime},
],
external_refs=[{"type": "website", "url": m.model_card_ref}] if m.model_card_ref else None,
))
# Data components
for d in config.data:
components.append(_component_entry(
_bom_ref("data", d.name, d.kind),
"data",
d.name, d.kind, d.supplier,
properties=[
{"name": "cdx:ai:data:provenance", "value": d.provenance},
{"name": "cdx:ai:data:license", "value": d.license},
],
external_refs=[{"type": "website", "url": d.data_card_ref}] if d.data_card_ref else None,
))
# Tool & MCP components (these ARE the attack surface)
for t in config.tools:
props = [{"name": "ai:bom:tool:kind", "value": t.kind}]
if t.transport:
props.append({"name": "ai:bom:mcp:transport", "value": t.transport})
if t.exposes_tools:
props.append({"name": "ai:bom:mcp:exposes", "value": ",".join(t.exposes_tools)})
components.append(_component_entry(
_bom_ref("tool", t.name, t.kind),
"application",
t.name, t.kind, t.supplier,
properties=props,
))
# Harness components (traditional SBOM, security-critical for an agent)
for h in config.harness:
ext = []
if h.slsa_attestation:
ext.append({"type": "other", "url": h.slsa_attestation,
"comment": "SLSA build provenance"})
components.append(_component_entry(
_bom_ref("pkg", h.name, h.version),
"library",
h.name, h.version, h.supplier,
properties=[{"name": "purl", "value": h.package_url}],
external_refs=ext or None,
))
return {
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"serialNumber": serial,
"version": 1,
"metadata": {"component": {"type": "application", "name": "sample-agent"}},
"components": components,
}
```
### 1.3 Your task — build a sample agent config
Construct an `AgentConfig` for a representative agent: a fine-tuned model, two data components (one training, one retrieval), three function-call tools, two MCP servers, and two harness libraries. At least one harness component should have SLSA attestation; at least one should not (you will catch this in validation).
Write the config to `agent_config.json` (serialize the dataclasses) and generate the BOM:
```python
bom = generate_aibom(config)
with open("aibom.json", "w") as f:
json.dump(bom, f, indent=2)
# validate it parses
python3 -m json.tool aibom.json > /dev/null
```
---
## Phase 2 — The runtime snapshot and the four-check validator (15 min)
Build the validator that compares the AI BOM against what the agent actually loads at runtime.
### 2.1 The runtime snapshot
```python
@dataclass
class RuntimeSnapshot:
"""What the agent ACTUALLY loads at startup. The validation target."""
resolved_model_version: str # may drift from the BOM's pinned version
resolved_checkpoint_hash: str
registered_tools: list[str] # tool names the agent registered
connected_mcp_servers: list[str] # MCP servers the agent connected to
loaded_libraries: list[dict] # {name, version} for harness deps
timestamp: str
```
A runtime snapshot fixture is provided in `runtime_snapshot.json`. It INTENTIONALLY diverges from the BOM:
- One MCP server the BOM declares is NOT connected at runtime (stale BOM entry).
- One MCP server the runtime connects to is NOT in the BOM (UNDECLARED component — the finding).
- The resolved model version differs from the BOM's pinned version (MODEL-VERSION DRIFT).
- One harness library has no SLSA attestation in the BOM (PROVENANCE finding).
### 2.2 The validator
```python
FindingSeverity = Literal["critical", "high", "medium", "low", "info"]
@dataclass
class Finding:
check: Literal["completeness", "version_drift", "provenance", "vuln_policy"]
severity: FindingSeverity
component: str
description: str
b4_surface: str # which B4 trust surface this maps to
def validate_aibom(bom: dict, runtime: RuntimeSnapshot) -> list[Finding]:
"""Run the four validation checks. Return findings (empty = clean).
1. COMPLETENESS — every tool/MCP in runtime must be in the BOM (undeclared?).
2. VERSION DRIFT — BOM-pinned model version == runtime-resolved version?
3. PROVENANCE — do model + harness components carry attestation?
4. VULN/POLICY — (simulated) any known-issue or policy-violation flags?
"""
findings: list[Finding] = []
# ... implement the four checks (see 2.3)
return findings
```
### 2.3 Implement the four checks
**Check 1 — Completeness (undeclared components).** For each MCP server in `runtime.connected_mcp_servers`, check it appears in the BOM's tool components. Any runtime server NOT in the BOM is an UNDECLARED COMPONENT finding. Severity scales with reachability — assume every connected server is contacted every turn (CRITICAL). Map the finding to B4's "MCP / indirect-injection surface."
```python
bom_tool_names = {c["name"] for c in bom["components"] if c["type"] == "application"}
for server in runtime.connected_mcp_servers:
if server not in bom_tool_names:
findings.append(Finding(
check="completeness",
severity="critical",
component=server,
description=(f"UNDECLARED MCP server '{server}' connected at runtime "
f"but absent from the AI BOM. Un-assessed indirect-injection "
f"source (SDD-B03). Agent equivalent of unknown transitive dep."),
b4_surface="MCP / indirect-injection surface",
))
```
Also check the reverse: any BOM-declared tool NOT in the runtime is a STALE entry (lower severity — `medium`).
**Check 2 — Version drift.** Find the model component in the BOM; compare its `version` to `runtime.resolved_model_version` and its checkpoint hash to `runtime.resolved_checkpoint_hash`. A mismatch is a MODEL-VERSION DRIFT finding (HIGH). Map to B4's "model / refusal-layer surface."
**Check 3 — Provenance.** For the model component and each harness component, check for an SLSA attestation (in `externalReferences` with comment "SLSA build provenance") or a supplier checksum property. An un-attested model is CRITICAL (scaling attack); an un-attested harness library is MEDIUM.
**Check 4 — Vulnerability/policy (simulated).** For this lab, simulate a policy check: any tool whose supplier is not in an approved-supplier set is a MEDIUM finding. Define `APPROVED_SUPPLIERS = {"acme-internal", "openai", "anthropic", "owasp"}` and flag any tool/MCP from a supplier not in the set.
### 2.4 Run it
```python
runtime = load_runtime_snapshot("runtime_snapshot.json")
findings = validate_aibom(bom, runtime)
for f in findings:
print(f"[{f.severity.upper()}] {f.check}: {f.component} -> {f.description}")
```
You should see at least: one CRITICAL undeclared MCP server, one HIGH model-version drift, one CRITICAL un-attested model (or MEDIUM un-attested library), and the stale-entry and policy findings.
---
## Phase 3 — The undeclared-component finding report (8 min)
### 3.1 The report structure
Produce `findings_report.json` — the format a B12 engagement deliverable would carry:
```python
def build_finding_report(findings: list[Finding], bom: dict, runtime: RuntimeSnapshot) -> dict:
"""Structure the findings for the engagement deliverable."""
undeclared = [f for f in findings if f.check == "completeness" and f.severity == "critical"]
drift = [f for f in findings if f.check == "version_drift"]
return {
"summary": {
"total_findings": len(findings),
"critical": sum(1 for f in findings if f.severity == "critical"),
"high": sum(1 for f in findings if f.severity == "high"),
"medium": sum(1 for f in findings if f.severity == "medium"),
},
"headline_finding": (
f"{len(undeclared)} undeclared component(s) and "
f"{len(drift)} model-version drift finding(s). "
f"The assessed agent is NOT the deployed agent."
),
"findings": [
{
"check": f.check,
"severity": f.severity,
"component": f.component,
"description": f.description,
"b4_trust_surface": f.b4_surface,
}
for f in findings
],
"b4_surface_map": build_b4_surface_map(bom, runtime),
}
```
### 3.2 The point
The headline finding is the delta between the BOM and the runtime. An undeclared MCP server is an un-assessed indirect-injection source; model-version drift means the assessed agent is not the deployed agent. This report is the supply-chain deliverable of the engagement.
---
## Phase 4 — Map the AI BOM to B4's trust surfaces (7 min)
### 4.1 The surface map
```python
def build_b4_surface_map(bom: dict, runtime: RuntimeSnapshot) -> dict:
"""Map each BOM component to its B4 trust-surface node.
This is the bridge: the AI BOM populates B4's trust-surface map.
Each component is a node; the engagement scopes against these nodes.
"""
surfaces = {
"model_refusal_surface": [],
"function_call_surface": [],
"mcp_injection_surface": [],
"data_leak_surface": [],
"harness_vuln_surface": [],
}
for c in bom["components"]:
if c["type"] == "machine-learning-model":
surfaces["model_refusal_surface"].append(c["name"])
elif c["type"] == "data":
surfaces["data_leak_surface"].append(c["name"])
elif c["type"] == "application":
kind = next((p["value"] for p in c.get("properties", [])
if p["name"] == "ai:bom:tool:kind"), "function_tool")
if kind == "mcp_server":
surfaces["mcp_injection_surface"].append(c["name"])
else:
surfaces["function_call_surface"].append(c["name"])
elif c["type"] in ("library", "framework"):
surfaces["harness_vuln_surface"].append(c["name"])
# Flag runtime-only surfaces (undeclared) separately
bom_mcp = {c["name"] for c in bom["components"]
if c["type"] == "application"
and any(p["name"] == "ai:bom:tool:kind" and p["value"] == "mcp_server"
for p in c.get("properties", []))}
undeclared_mcp = set(runtime.connected_mcp_servers) - bom_mcp
if undeclared_mcp:
surfaces["UNDECLARED_mcp_injection_surface"] = sorted(undeclared_mcp)
return surfaces
```
### 4.2 The point
The AI BOM populates B4's trust-surface map. The model maps to the refusal surface (SDD-B06). The function-call tools map to the function-call surface (SDD-B04). The MCP servers map to the indirect-injection surface (SDD-B03). The data components map to the data-leak surface. The harness libraries map to the traditional vulnerability surface. The undeclared MCP servers appear as a separate UN-ASSESSED surface — the finding.
---
## Phase 5 — Stretch: the BOM-drift detector (optional, 5 min)
Write `detect_bom_drift(bom_at_t0: dict, bom_at_t1: dict) -> dict` that compares two BOMs generated at different times and reports what changed: added components, removed components, version changes. The point: a production agent that adds a tool, connects to a new MCP server, or updates its model changes its AI BOM. The drift detector is the continuous-monitoring signal — every change is a re-assessment trigger.
---
## Deliverables
- `aibom.py` — the data model and CycloneDX generator (Phase 1)
- `aibom.json` — the generated AI BOM for the sample agent (Phase 1)
- `validator.py` — the four-check validator + runtime snapshot loader (Phase 2)
- `findings_report.json` — the undeclared-component and drift finding report (Phase 3)
- `surface_map.json` — the B4 trust-surface map from the AI BOM (Phase 4)
- (optional) `drift_detector.py` — the BOM-to-BOM drift detector (Phase 5)
## Success criteria
- [ ] The generated AI BOM (`aibom.json`) is valid CycloneDX 1.6 JSON, declares the model, data, tool/MCP, and harness components, and carries the NTIA minimum-element fields (architecture, parameters, checkpoint hash, provenance, licensing).
- [ ] The validator runs all four checks (completeness, version drift, provenance, vuln/policy) and produces at least one finding per check class against the fixture runtime snapshot.
- [ ] The undeclared-component finding correctly identifies the MCP server present at runtime but absent from the BOM, classified CRITICAL and mapped to the MCP/indirect-injection surface (SDD-B03).
- [ ] The model-version drift finding correctly flags the mismatch between the BOM-pinned version and the runtime-resolved version, classified HIGH and mapped to the model/refusal-layer surface (SDD-B06).
- [ ] The provenance finding flags the un-attested model or harness component and articulates the scaling-attack risk.
- [ ] The B4 surface map populates all five surface classes from the BOM components and flags the undeclared MCP servers as a separate un-assessed surface.
- [ ] Every finding ties back to a specific BOM component and a specific B4 trust surface — no assertion-only claims.