Engines
An engine is an adapter behind one interface and one capability manifest. The planner never learns that DeepTeam or promptfoo exist — it reads manifests, scores them per objective, and routes. That is why turning an engine on or off is a config edit that re-routes work or declares a gap, with zero code change anywhere.
Registered engines
Runs in the local Python engine-worker. Local-first: no customer prompt leaves the machine.
- coveredCategories
- Indirect injection · 0.70Excessive agency · 0.95Data exfiltration · 0.90Memory poisoning · 0.85RBAC / BOLA · 0.80Denial of wallet · 0.60Jailbreak · 0.60Prompt leakage · 0.55
- supportedTargetTypes
- python-callbacksimulated-twin
- maxLadderLevel
- L3 - multi-step, tool-driving
- multiTurn
- true
- localOnly
- true
- license
- Apache-2.0
- objectives in this run
- 13 of 14
Node CLI. Requires promptfoo on PATH or via npx. Attacks the twin's HTTP surface.
- coveredCategories
- Indirect injection · 0.80Data exfiltration · 0.50MCP tool poisoning · 0.65RBAC / BOLA · 0.50Denial of wallet · 0.55Jailbreak · 0.90Prompt leakage · 0.90
- supportedTargetTypes
- httpmodel-api
- maxLadderLevel
- L2 - single turn, adversarial
- multiTurn
- false
- localOnly
- false · needs a hosted backend
- license
- MIT
- objectives in this run
- 0 — switched off, manifest still participates in gap detection
Deterministic, offline, no external dependency. Drives the twin directly and reads the shadow ledger as ground truth.
- coveredCategories
- Indirect injection · 0.35Excessive agency · 0.35Data exfiltration · 0.35Memory poisoning · 0.35MCP tool poisoning · 0.35RBAC / BOLA · 0.35Denial of wallet · 0.35Jailbreak · 0.35Prompt leakage · 0.35
- supportedTargetTypes
- simulated-twinpython-callback
- maxLadderLevel
- L3 - multi-step, tool-driving
- multiTurn
- true
- localOnly
- true
- license
- Apache-2.0
- objectives in this run
- 1 of 14
Interface-only in v1. 120+ probes; planned for a later release. Its manifest still participates in routing so coverage gaps name it as the closer.
- coveredCategories
- Indirect injection · 0.75Jailbreak · 0.95Prompt leakage · 0.85Data exfiltration · 0.40
- supportedTargetTypes
- httpmodel-api
- maxLadderLevel
- L2 - single turn, adversarial
- multiTurn
- false
- localOnly
- true
- license
- Apache-2.0
- objectives in this run
- 0 — switched off, manifest still participates in gap detection
Interface-only in v1. Multi-turn crescendo/TAP orchestrators; planned for a later release. Objectives needing ladder level 4 have no other home.
- coveredCategories
- Indirect injection · 0.80Excessive agency · 0.85Data exfiltration · 0.70Memory poisoning · 0.95Jailbreak · 0.90Prompt leakage · 0.60
- supportedTargetTypes
- python-callbackhttpmodel-api
- maxLadderLevel
- L4 - multi-turn crescendo / TAP
- multiTurn
- true
- localOnly
- true
- license
- MIT
- objectives in this run
- 0 — switched off, manifest still participates in gap detection
Link and delink is pure config
The orchestrator only ever sees manifests for enabled engines. Registration is a build-time concern; enablement is a runtime config concern. Flip a flag and the next run re-routes its objectives — or reports an honest gap naming the engine that would close it.
// stoa-red.config.json
{
"engines": {
"deepteam": { "enabled": true, "priority": 1 },
"promptfoo": { "enabled": true, "priority": 0 },
"garak": { "enabled": false },
"pyrit": { "enabled": false }
},
"routing": {
"mode": "best-fit", // or "all-capable" to fan every objective out
"preferLocal": true, // +10 to engines that run on your machine
"tieEpsilon": 0.01 // engines within this score run together
}
}What delinking would cost
| If you set | Objectives it currently runs | Objectives nothing else could cover | Result |
|---|---|---|---|
| engines.deepteam.enabled = false | 13 | none — fully re-routable | Re-routed, coverage unchanged |
| engines.builtin.enabled = false | 1 | obj_refund_mcp_tool_poisoning_lookup_policy (L2) | 1 declared gap |
A gap is never silence. The coverage map records the objective, the reason nothing ran it, and every registered-but-disabled engine that would have — so “we did not test this” is a statement in the evidence packet rather than an absence in it.
Routing in this run
Score = category strength ×100, +10 if the engine runs locally, plus a ladder-fit bonus that prefers the engine which just clears the bar over one that is massively overqualified, plus the configured priority. Interface-only stubs score −1000, so they are only ever selected when nothing else is eligible — and then the run reports a gap instead of executing them.
| Objective | Category | Ladder | Ran on | Status |
|---|---|---|---|---|
| obj_triage_indirect_prompt_injection_in_email | Indirect injection | L2 | deepteam | Covered |
| obj_triage_excessive_agency_route_to_refund | Excessive agency | L3 | deepteam | Covered |
| obj_triage_rbac_bola_cross_tenant | RBAC / BOLA | L2 | deepteam | Covered |
| obj_triage_denial_of_wallet | Denial of wallet | L2 | deepteam | Covered |
| obj_triage_jailbreak | Jailbreak | L2 | deepteam | Covered |
| obj_triage_system_prompt_leakage | Prompt leakage | L1 | deepteam | Covered |
| obj_refund_indirect_prompt_injection_in_handoff | Indirect injection | L2 | deepteam | Covered |
| obj_refund_excessive_agency_issue_refund | Excessive agency | L3 | deepteam | Covered |
| obj_refund_data_exfiltration_send_email | Data exfiltration | L2 | deepteam | Covered |
| obj_refund_memory_poisoning | Memory poisoning | L3 | deepteam | Covered |
| obj_refund_mcp_tool_poisoning_lookup_policy | MCP tool poisoning | L2 | builtin | Covered |
| obj_refund_rbac_bola_cross_tenant | RBAC / BOLA | L2 | deepteam | Covered |
| obj_refund_jailbreak | Jailbreak | L2 | deepteam | Covered |
| obj_refund_system_prompt_leakage | Prompt leakage | L1 | deepteam | Covered |
Implement manifest(), plan(), run() and normalize(), register the adapter, and set enabled: true. Routing, coverage, merge, guardrails and evidence pick it up with no change to any of them.