Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Graph Engineering & Evidence-Gated Learning

The primary architectural standard for this framework is now graph engineering: one explicit, declarative engagement graph defines the topology, state channels, reducers, conditional routers, bounded loops, and memory boundaries for every runtime. The source of truth is graph/redteam.graph.json, validated against schemas/graph.schema.json by tools/graph/validate-graph.mjs.

Why graph engineering

The graph makes orchestration inspectable instead of implicit prompt choreography:

Canonical topology

The canonical graph contains 15 nodes, including the shared context stage between inventory and specialist fan-out.

The topology is:

  1. START -> validate_scope — the subscription, scope, engagement mode, and read-only role posture are validated before any Azure access.

  2. memory_load — prior methodology memory is loaded as read-only context.

  3. preflight_inventory — the Inventory & Scope agent performs sequential permission checks and resource enumeration.

  4. build_security_context — a compact handoff names source references and unavailable signal families. The default context contains an unverified inventory reference, not fetched security telemetry. A custom host can supply verified, bounded summaries with evidence provenance. Defender, Entra, Sentinel, and other family names are coverage slots, not automatic integrations. Missing signals remain coverage gaps; raw telemetry and secrets do not belong in this prompt context.

  5. plan_specialists -> run_specialist — a LangGraph-style Send fan-out maps over the in-scope read-only roster and dispatches one specialist worker per domain in parallel. The runners apply both scope.domains and scope.resource_types: a specialist must match both when both filters are present. ARM types match case-insensitively, including provider /* patterns on either side. These filters narrow agent dispatch; they do not replace resource-level scope validation before a read.

  6. collect_raw — raw specialist outputs fan back in through a deterministic merge_findings reduce.

  7. evaluate — the evaluator-optimizer loop head runs deterministic checks plus a critic score over candidate findings.

  8. route_after_evaluate — if revision < max_revisions and quality is below quality_threshold, the graph reflects back to plan_specialists; otherwise it proceeds.

  9. judge — an Agent-as-a-Judge gate re-checks candidate findings using targeted read-only evidence queries. False-positive observations enter methodology memory as inert candidates; historical observations do not automatically suppress current findings.

  10. authorize_active / route_active — a human-in-the-loop interrupt gates the optional active lanes. Before pausing for approval, the runner fails closed unless the selected lane’s enabled flag and attestation ID are present. Read-only or rejected runs route straight to correlation.

  11. correlate -> report — confirmed findings are correlated into RBAC and attack paths, then rendered into deliverables.

  12. reflexion_debrief -> END — the run records an inert episode. Stable lessons are promoted only after matching evidence from at least two distinct runs for the same agent.

State channels and reducers

The graph state follows a LangGraph-style channel model. Concurrent writes are safe because each channel declares its reducer in the graph contract.

ChannelTypeReducerRole
scopeobjectlastValidated subscription, mode, domain, exclusion, and read-only role context.
memoryobjectlastMethodology memory loaded from prior runs.
inventory_refstringlastPath to the preflight resource inventory.
security_contextobjectlastShared source references, verification state, and coverage gaps; verified summaries when supplied by the host.
raw_findingsarrayappendPer-specialist JSONL outputs accumulated by Send fan-in.
candidate_findingsarraymerge_findingsDeterministically deduped findings before critique and judge.
critiqueobjectlastEvaluator quality score and notes that drive reflection.
revisionnumberlastBounded reflection iteration counter.
confirmed_findingsarraymerge_findingsFindings promoted by the false-positive judge.
attack_pathsarrayappendCross-domain authorization and attack-path chains.
report_refsarrayappendRendered deliverable paths.

The default security-context/v1 object has status: summary-only. A nonempty inventory reference produces inventory.status: referenced and ARM status: unverified; no reference produces missing and unavailable, respectively. The default builder does not open the inventory file or establish that it exists. All eight signal-family slots remain explicit, including unavailable families. Node hosts can supply securityContextFn; asynchronous callbacks require runGraphAsync, while runGraph requires a synchronous callback. Hosts remain responsible for verifying evidence and bounding or redacting supplied summaries. | approved | boolean/null | last | Human decision at the gated active-lane interrupt. |

Self-improving loops

The graph deliberately borrows from prior art in self-improving agents while excluding unsafe runtime self-modification:

AEF-compatible learning contract

The current AEF integration pins the read-only aef-core source at 07b291198cfdeee9dc82095a9366931cf14b2a92 in tools/aef/source-lock.json. Its bootstrap builds an isolated runtime from that source; it does not modify the upstream checkout. The contract contributes inert candidates, independent run attribution, per-agent consolidation, and auditable promotion or rollback. See the integration review for source provenance and measured limits. The separate target-owned code-evolution runner is bounded and explicitly invoked; it does not enable the upstream evolution engine or run automatically after each assessment.

Observability: the Agent Observatory

Graph execution is only trustworthy if it is inspectable. Every node transition, handoff, guarded tool call and memory transition is appended to a per-session event log (runs/live-events.jsonl), and the Observatory is a read-only projection of exactly that log:

node tools/dashboard/server.mjs --session engagements/<session> --port 4318
PanelShows
Topology3D coordination map; each animated packet is a recorded exchange with source, destination, direction and byte count
TimelineAppend-only event history, filterable by agents, handoffs, tools, memory, runs, evolution and evaluation
MemoryRetrieved / candidate / promoted counts, each row expandable to its source record and evidence hash
ReviewEvaluator rounds with rubric, input hash, critique and the routing decision
FindingsThe session’s findings with severity drill-down, plus the rendered report
EvolutionPropose → test → accept-or-reject decisions for the bounded optimizer

The Findings panel reads the session’s reports/findings.json and mirrors the generated report. Its summary tiles and severity chips are filters, so a reader can move from a count straight to the findings behind it; Open HTML report opens the full report in a new tab with PDF and HTML downloads inside it, named <subscription>-<assessment date> so reports from different subscriptions or dates are never confused for one another.

Three properties keep it presentable to an audience without overclaiming:

Findings text is model-authored, so the panel treats it as untrusted data: it reaches the DOM as text only, never as markup. The report is served under a document-scoped policy that permits its own inline assets but allows no network access at all — a tighter sandbox than opening the same file from disk.

Memory firewall: the immutable boundary

The one immutable boundary is the read-only enforcement system. Learning may write only memory/methodology/; it may not modify the guard core, egress allowlist, cluster allowlist, read-only role requirements, or anything under guardrails/**. The graph schema describes this as a memory-write target, and tools/graph/validate-graph.mjs enforces structural and referential integrity, including the memory firewall.

Methodology learning cannot execute generated code, create tools, or rewrite the guard. It can change what prior observations agents retrieve; it cannot change what they are allowed to do. The separate target-owned evolution runner has its own bounded mutation surface and verification gates, outside the assessment’s methodology-memory loop.

One graph, two engines

The same graph/redteam.graph.json drives two execution models:

  1. Dependency-free Node runner — tools/graph/run-graph.mjs executes the topology with simulated dispatch by default. It is suitable for deterministic graph tests, not proof of an Azure assessment. The separate tools/graph/run-live.mjs entry point uses the Claude native adapter with scoped preflight and guarded reads. Standalone live CLI adapters for Copilot, Codex, and Cursor are not implemented; those runtimes use their native agent orchestration separately.

  2. First-class LangGraph target — integrations/langgraph/ compiles the same JSON graph into a Python StateGraph, using LangGraph concepts such as Send, reducers, checkpointers, interrupts, and Store-style memory while reusing the same read-only guard through a subprocess bridge. Its dependencies are isolated from the Node core, and its specialist bodies remain host integration stubs. Compilation is not live execution.

Prior art and ecosystem

This architecture is inspired by:

The framework adapts those ideas to a red-team setting by keeping the learning surface narrow and the enforcement surface deterministic, fail-closed, and shared by every runtime.