The primary architectural standard for this framework is now graph engineering: one
explicit, declarative engagement graph defines the topology, state channels, reducers,
conditional routers, bounded loops, and memory boundaries for every runtime. The source of
truth is graph/redteam.graph.json, validated against
schemas/graph.schema.json by
tools/graph/validate-graph.mjs.
Why graph engineering¶
The graph makes orchestration inspectable instead of implicit prompt choreography:
Determinism — node order, routers, fan-out, and fan-in are declared in JSON and checked by the validator before execution.
Resumability — parallel specialist work is backed by the durable JSONL task manifest in
tools/orchestration/manifest.mjs, so interrupted fan-out can be resumed and reduced deterministically.Explicit state and reducers — each shared channel declares how writes merge, including append-only fan-in and finding-level dedupe.
Bounded self-improvement — reflection is a loop with explicit exit criteria (
max_revisions: 2,quality_threshold: 0.85), not an open-ended conversation.Reviewable safety boundaries — the graph declares where memory may be read or written, while read-only enforcement remains outside the self-improvement surface.
Canonical topology¶
The canonical graph contains 15 nodes, including the shared context stage between inventory and specialist fan-out.
The topology is:
START -> validate_scope— the subscription, scope, engagement mode, and read-only role posture are validated before any Azure access.memory_load— prior methodology memory is loaded as read-only context.preflight_inventory— the Inventory & Scope agent performs sequential permission checks and resource enumeration.build_security_context— a compact handoff names source references and unavailable signal families. The default context contains an unverified inventory reference, not fetched security telemetry. A custom host can supply verified, bounded summaries with evidence provenance. Defender, Entra, Sentinel, and other family names are coverage slots, not automatic integrations. Missing signals remain coverage gaps; raw telemetry and secrets do not belong in this prompt context.plan_specialists -> run_specialist— a LangGraph-style Send fan-out maps over the in-scope read-only roster and dispatches one specialist worker per domain in parallel. The runners apply bothscope.domainsandscope.resource_types: a specialist must match both when both filters are present. ARM types match case-insensitively, including provider/*patterns on either side. These filters narrow agent dispatch; they do not replace resource-level scope validation before a read.collect_raw— raw specialist outputs fan back in through a deterministicmerge_findingsreduce.evaluate— the evaluator-optimizer loop head runs deterministic checks plus a critic score over candidate findings.route_after_evaluate— ifrevision < max_revisionsand quality is belowquality_threshold, the graph reflects back toplan_specialists; otherwise it proceeds.judge— an Agent-as-a-Judge gate re-checks candidate findings using targeted read-only evidence queries. False-positive observations enter methodology memory as inert candidates; historical observations do not automatically suppress current findings.authorize_active/route_active— a human-in-the-loop interrupt gates the optional active lanes. Before pausing for approval, the runner fails closed unless the selected lane’s enabled flag and attestation ID are present. Read-only or rejected runs route straight to correlation.correlate -> report— confirmed findings are correlated into RBAC and attack paths, then rendered into deliverables.reflexion_debrief -> END— the run records an inert episode. Stable lessons are promoted only after matching evidence from at least two distinct runs for the same agent.
State channels and reducers¶
The graph state follows a LangGraph-style channel model. Concurrent writes are safe because each channel declares its reducer in the graph contract.
| Channel | Type | Reducer | Role |
|---|---|---|---|
scope | object | last | Validated subscription, mode, domain, exclusion, and read-only role context. |
memory | object | last | Methodology memory loaded from prior runs. |
inventory_ref | string | last | Path to the preflight resource inventory. |
security_context | object | last | Shared source references, verification state, and coverage gaps; verified summaries when supplied by the host. |
raw_findings | array | append | Per-specialist JSONL outputs accumulated by Send fan-in. |
candidate_findings | array | merge_findings | Deterministically deduped findings before critique and judge. |
critique | object | last | Evaluator quality score and notes that drive reflection. |
revision | number | last | Bounded reflection iteration counter. |
confirmed_findings | array | merge_findings | Findings promoted by the false-positive judge. |
attack_paths | array | append | Cross-domain authorization and attack-path chains. |
report_refs | array | append | Rendered deliverable paths. |
The default security-context/v1 object has status: summary-only. A nonempty inventory
reference produces inventory.status: referenced and ARM status: unverified; no reference
produces missing and unavailable, respectively. The default builder does not open the
inventory file or establish that it exists. All eight signal-family slots remain explicit,
including unavailable families. Node hosts can supply securityContextFn; asynchronous
callbacks require runGraphAsync, while runGraph requires a synchronous callback. Hosts
remain responsible for verifying evidence and bounding or redacting supplied summaries.
| approved | boolean/null | last | Human decision at the gated active-lane interrupt. |
Self-improving loops¶
The graph deliberately borrows from prior art in self-improving agents while excluding unsafe runtime self-modification:
Self-Refine — each specialist performs a bounded refinement pass on its own draft findings before writing raw output.
Evaluator-optimizer —
evaluatecombines deterministicrun-checksoutput with a critic score and stages a bounded, inert parameter candidate;route_after_evaluateroutes back to targeted specialist planning only while the loop is undermax_revisionsand belowquality_threshold.Agent-as-a-Judge —
judgere-verifies candidate findings with bounded targeted read-only queries, distinguishes confirmed from unverified results, and records inert false-positive candidates in methodology memory. Reuse still requires the distinct-run evidence gate.Reflexion / ExpeL-style debrief —
reflexion_debriefrecords run-attributed experiences. Stateless consolidation promotes only stable signatures reproduced in at least two distinct runs, and never pools evidence between agents.
AEF-compatible learning contract¶
The current AEF integration pins the read-only aef-core source at
07b291198cfdeee9dc82095a9366931cf14b2a92 in tools/aef/source-lock.json. Its bootstrap builds
an isolated runtime from that source; it does not modify the upstream checkout. The contract
contributes inert candidates, independent run attribution, per-agent consolidation, and
auditable promotion or rollback. See the integration review for source
provenance and measured limits. The separate target-owned code-evolution runner is bounded and
explicitly invoked; it does not enable the upstream evolution engine or run automatically
after each assessment.
Observability: the Agent Observatory¶
Graph execution is only trustworthy if it is inspectable. Every node transition, handoff,
guarded tool call and memory transition is appended to a per-session event log
(runs/live-events.jsonl), and the Observatory is a read-only projection of exactly that log:
node tools/dashboard/server.mjs --session engagements/<session> --port 4318| Panel | Shows |
|---|---|
| Topology | 3D coordination map; each animated packet is a recorded exchange with source, destination, direction and byte count |
| Timeline | Append-only event history, filterable by agents, handoffs, tools, memory, runs, evolution and evaluation |
| Memory | Retrieved / candidate / promoted counts, each row expandable to its source record and evidence hash |
| Review | Evaluator rounds with rubric, input hash, critique and the routing decision |
| Findings | The session’s findings with severity drill-down, plus the rendered report |
| Evolution | Propose → test → accept-or-reject decisions for the bounded optimizer |
The Findings panel reads the session’s reports/findings.json and mirrors the generated
report. Its summary tiles and severity chips are filters, so a reader can move from a count
straight to the findings behind it; Open HTML report opens the full report in a new tab
with PDF and HTML downloads inside it, named <subscription>-<assessment date> so reports
from different subscriptions or dates are never confused for one another.
Three properties keep it presentable to an audience without overclaiming:
Loopback only. It binds
127.0.0.1, serves bounded metadata, and makes no Azure or model calls of its own.Truthful labelling. Live stream, historical replay and fixture data are always labelled distinctly, so a recorded replay is never shown as live activity.
No synthesised traffic. If an exchange was not recorded, nothing is drawn. Unobserved nodes render as having no recorded execution rather than as idle-but-healthy, and a severity with no findings is never presented as evidence that the risk is absent.
Findings text is model-authored, so the panel treats it as untrusted data: it reaches the DOM as text only, never as markup. The report is served under a document-scoped policy that permits its own inline assets but allows no network access at all — a tighter sandbox than opening the same file from disk.
Memory firewall: the immutable boundary¶
The one immutable boundary is the read-only enforcement system. Learning may write only
memory/methodology/; it may not modify the guard core, egress allowlist, cluster allowlist,
read-only role requirements, or anything under guardrails/**. The graph schema describes this
as a memory-write target, and tools/graph/validate-graph.mjs enforces structural and
referential integrity, including the memory firewall.
Methodology learning cannot execute generated code, create tools, or rewrite the guard. It can change what prior observations agents retrieve; it cannot change what they are allowed to do. The separate target-owned evolution runner has its own bounded mutation surface and verification gates, outside the assessment’s methodology-memory loop.
One graph, two engines¶
The same graph/redteam.graph.json drives two execution models:
Dependency-free Node runner —
tools/graph/run-graph.mjsexecutes the topology with simulated dispatch by default. It is suitable for deterministic graph tests, not proof of an Azure assessment. The separatetools/graph/run-live.mjsentry point uses the Claude native adapter with scoped preflight and guarded reads. Standalone live CLI adapters for Copilot, Codex, and Cursor are not implemented; those runtimes use their native agent orchestration separately.First-class LangGraph target —
integrations/langgraph/compiles the same JSON graph into a PythonStateGraph, using LangGraph concepts such asSend, reducers, checkpointers, interrupts, and Store-style memory while reusing the same read-only guard through a subprocess bridge. Its dependencies are isolated from the Node core, and its specialist bodies remain host integration stubs. Compilation is not live execution.
Prior art and ecosystem¶
This architecture is inspired by:
LangGraph for
StateGraph,Send, reducers, checkpointers, interrupts, and Store-style memory.Reflexion, Self-Refine, Agent-as-a-Judge patterns, ExpeL, and evaluator-optimizer loops for iterative improvement.
The broader awesome-LangGraph ecosystem of graph-based agent systems.
The framework adapts those ideas to a red-team setting by keeping the learning surface narrow and the enforcement surface deterministic, fail-closed, and shared by every runtime.