Human-in-execution
A human designs the first boundary and runs local tests manually. AI may observe notes but has no execution authority.
The shift
From “AI writes a shell tool†to “AI scaffolds inside a governed chainâ€
The human-in-command defines the boundary: what the tool may do, where it may run, how evidence is produced, which failures are classified, which secrets are redacted, and which claims may be made after execution.
AI may scaffold designs, tests, schemas, docs, wrappers and even implementation candidates, but its authority increases only when the surrounding chain becomes more standardized and observable.
The goal is not to let AI run arbitrary Bash. The goal is to build a minimal, bounded, repeatable command-execution capability that can later be safely used by AI scaffolds under governance.
Architecture flow
Authority shifts left only after standardization
Human command, standardized chain, delegated AI authority
Mermaid source
flowchart LR HIC["Human-in-command"] --> B["Boundary contract"] B --> S["Standardized chain"] S --> P["Policy"] S --> PR["Process"] S --> PROC["Procedure"] S --> CFG["Configuration"] S --> WI["Work instruction"] S --> EV["Evidence schema"] EV --> G["Gate decision"] G --> AIA["Delegated AI authority"] AIA --> SC["AI scaffold proposes Rust change"] SC --> RV["Human or gate review"] RV --> EX["Bounded execution test"] EX --> OUT["Evidence / outcome"] OUT --> HIC
Maturity ladder for the bash_exec example
Mermaid source
flowchart TD L0["Level 0: Human-in-execution"] --> L1["Level 1: AI-in-the-loop"] L1 --> L2["Level 2: AI-in-execution"] L2 --> L3["Level 3: Human-in-the-loop by gate"] L3 --> L4["Level 4: AI-in-command within bounded mandate"] L4 --> L5["Level 5: Human-in-command governs by evidence"] L0 --> V0["Manual bash_exec design and tests"] L1 --> V1["AI suggests contracts and test cases"] L2 --> V2["AI scaffolds implementation candidates"] L3 --> V3["Human approves gates and exceptions"] L4 --> V4["AI selects approved routines"] L5 --> V5["Governed command-execution capability"]
Worked stages
How the steps work
AI-in-the-loop
AI proposes schemas, failure modes and test cases. Human decides what enters the contract.
AI-in-execution
AI may scaffold implementation candidates, docs and tests inside a disposable branch or workspace. It cannot promote or claim validation.
Human-in-the-loop by gate
Human review is required for capability expansion, command class changes, environment access, timeout changes and redaction changes.
AI-in-command, delegated
AI may select approved routines for pre-release checks, but only inside a bounded mandate and with evidence output.
Human-in-command by evidence
The human governs through status, audit, evidence, failure trends and explicit update decisions rather than performing every step manually.
Boundary contract
Minimum safe contract for a Rust bash_exec tool
Allowed intent. Execute narrowly bounded diagnostic, build, validation or evidence commands in a declared working directory, with timeout, output capture, redaction and typed failure classification.
Forbidden intent. Unbounded shell access, hidden background execution, credential discovery, secret printing, policy bypass, destructive mutation without explicit approval, or commands outside the declared workspace boundary.
| Contract field | Example for bash_exec | Gate |
|---|---|---|
| Command boundary | Command string is accepted only with explicit working directory, timeout and caller claim. | Reject missing cwd, timeout or claim. |
| Runtime boundary | Use absolute executable paths in service/SYSTEM contexts where PATH may be absent. | Reject PATH-dependent assumptions for production/service use. |
| Output boundary | Capture stdout/stderr separately, truncate safely, redact secret-like patterns. | Reject token/cookie/JWT/private-key leakage. |
| Failure taxonomy | timeout, exit_code, spawn_error, permission_denied, command_not_found, contract_error, policy_refusal. | Every failure returns typed JSON. |
| Evidence | run_id, cwd, executable path, exit code, duration, redaction flag, output paths, failure_class. | No validation claim without evidence path. |
| Authority | AI may scaffold or run approved checks; human approves boundary expansion and promotion. | AI-in-command only within delegated mandate. |
Rust design shape
Implementation shape, without granting unbounded shell power
Input model
BashExecRequest {
command: String,
cwd: AbsolutePath,
timeout_ms: u64,
max_output_bytes: usize,
caller_claim: String,
dry_run: bool
}
Execution model
Spawn a declared shell/executable with bounded timeout. Do not rely on PATH in service contexts. Capture stdout/stderr to memory and optionally evidence files. Kill the process tree on timeout where supported.
Redaction model
Redact before display, before evidence write and before status summaries. Treat secret-detection failures as policy failures, not merely warnings.
Failure model
Every error returns a structured class and does not collapse into generic failure. The failure mode is design feedback.
Promotion model
AI may propose a promotion candidate only after local tests, pre-release smoke and evidence validation. Human-in-command approves production boundary changes.
Planes
Where the bash_exec work sits
Planes for the bash_exec capability
Step-by-step maturity growth
How the capability matures from human execution to governed AI authority
This example uses four review lenses. They keep the work from becoming “AI wrote some code” and turn it into a controlled capability design.
| Lens | Question | bash_exec example |
|---|---|---|
| PDD Plan / Purpose Design | What is the intended capability and boundary? | A bounded command-execution tool for diagnostics, build checks, validation and evidence capture. Not an unrestricted shell. |
| TDD Test / Technical Correctness | What proves the implementation behaves correctly? | Unit, integration and runtime tests for cwd, timeout, exit codes, stdout/stderr capture, redaction and typed errors. |
| BDD Behaviour / Boundary Behaviour | What should callers and operators experience? | Caller gets structured JSON with run_id, status, failure_class, output paths and redaction flags. Policy refusals are clear. |
| ODD Outcome / Operational Delivery | What outcome may be claimed after execution? | Only claim validated when boundary evidence exists: service runtime smoke, secret guard, timeout guard and truthful status/audit. |
PDD → TDD → BDD → ODD maturity loop
Level 0 · Human-in-execution
Maturity step. A human writes the first tool boundary and executes examples manually. AI has no execution authority.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | Define why the tool exists: bounded diagnostics/build/evidence command execution. | The purpose is written as “run shell commands”, which is too broad. | Reject until allowed and forbidden command classes are explicit. |
| TDD | Human writes fixtures for success, non-zero exit, timeout, missing cwd and spawn error. | Only happy-path command is tested. | No next level without negative tests. |
| BDD | Caller sees predictable JSON instead of raw terminal behaviour. | stderr text is interpreted as failure even when exit code is zero, or vice versa. | Behaviour contract separates exit_code, stderr and failure_class. |
| ODD | Evidence file proves command, cwd, duration, exit code and redaction state. | Developer says “it worked locally” without boundary evidence. | No validation claim without evidence path. |
Level 1 · AI-in-the-loop
Maturity step. AI helps reason, classify and compare. It may propose schemas, failure classes and tests, but cannot execute or approve.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | AI proposes a request/response schema and identifies actor placement. | AI expands scope to generic terminal automation. | Human-in-command rejects scope drift. |
| TDD | AI suggests property cases: output truncation, UTF-8, timeout, redaction fixtures. | AI optimizes for code elegance, not operational evidence. | Tests must map to boundary claims. |
| BDD | AI writes example caller stories: dry-run refusal, timeout message, policy refusal. | Policy refusal looks like a tool crash. | Refusals must be typed and understandable. |
| ODD | AI suggests evidence fields and audit vocabulary. | Evidence misses process path, cwd or redaction flag. | Evidence schema review required. |
Level 2 · AI-in-execution
Maturity step. AI may scaffold implementation candidates in a disposable workspace. It may not promote, mutate production or claim validation.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | AI creates a Rust module skeleton from the approved schema. | AI adds convenience features outside the contract. | Diff review rejects unapproved authority expansion. |
| TDD | AI scaffolds unit tests and integration tests. | Tests use interactive PATH assumptions that fail as SYSTEM/service. | Service-runtime assumptions must be explicit and tested. |
| BDD | AI generates examples for callers and error responses. | Errors are fluent but not machine-actionable. | All outputs must be typed JSON. |
| ODD | AI generates local evidence and test summaries. | AI reports success from tests only, without runtime smoke. | Unit tests alone cannot close outcome. |
Level 3 · Human-in-the-loop by gate
Maturity step. The human no longer does every task, but must approve gates: authority expansion, destructive commands, redaction changes, environment changes and promotion.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | Gate defines what changed: command class, cwd rule, timeout or evidence field. | Small code change silently changes authority. | Every authority-impacting change needs approval. |
| TDD | Tests are run by automation, reviewed by human. | Green tests are treated as production safety. | Runtime smoke and evidence boundary still required. |
| BDD | Human verifies that operator-facing failures are actionable. | Failure text blames the user but does not identify contract error. | Failure classification must guide correction. |
| ODD | Human confirms which governance claim can be made. | Status says validated when only configured or tested is true. | Status/audit must reflect boundary truth. |
Level 4 · AI-in-command within bounded mandate
Maturity step. AI may select approved routines and orchestrate pre-release checks inside a delegated mandate. It cannot redefine governance truth, risk tolerance or policy boundaries.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | AI chooses the approved validation plan based on the change type. | AI chooses a faster plan because it infers the change is small. | Plan selection must be rule-bound. |
| TDD | AI runs approved test packs and records evidence. | AI retries until success and hides failed attempts. | All attempts and failures are evidence. |
| BDD | AI validates caller stories and refusal behaviour. | AI accepts ambiguous behaviour as good enough. | BDD assertions must be explicit. |
| ODD | AI assembles the promotion preflight package. | AI prepares a claim larger than evidence supports. | Claim/evidence consistency gate. |
Level 5 · Human-in-command governs by evidence
Maturity step. The human governs through evidence, trend, audit and boundary updates instead of manually executing every task.
| Lens | Example work | Typical mistake / failure mode | Gate |
|---|---|---|---|
| PDD | Governance reviews whether the tool purpose or boundaries need revision. | Repeated exceptions are treated as noise. | Repeated exception creates routine or policy review. |
| TDD | Test suite evolves from observed failure modes. | Regression tests are not added after incidents. | Every accepted failure learning creates a fixture or rationale. |
| BDD | Operator experience improves from repeated support questions. | Users keep misusing the tool because contract language is unclear. | Behaviour docs and work instruction updated. |
| ODD | Evidence trends show whether the capability is stable, risky or ready to scale. | Scale is approved because automation exists, not because evidence supports it. | Scale requires stability evidence and rollback path. |
The maturity path is not automatic. A system can stay at Level 1 or 2 forever when risk, evidence quality, reversibility or operator trust is insufficient. Maturity grows only when the chain becomes more standardized and the evidence proves that authority can safely shift.
Review template
Gate checklist for this example
| Gate | Question | Pass evidence |
|---|---|---|
| G1 Boundary | Is the allowed command class explicit? | Policy + request schema + refusal tests. |
| G2 Service reality | Does it work without PATH and from the declared cwd? | Runtime smoke with absolute executable path. |
| G3 Failure modes | Are timeout, spawn and exit failures typed? | Failure fixtures and evidence JSON. |
| G4 Redaction | Can stdout/stderr leak secrets? | Secret-like fixture rejected or redacted. |
| G5 Authority | What can AI scaffold, execute, approve or promote? | Actor placement record. |
| G6 Governance truth | What claim may be made after the run? | Status/audit updated only after boundary evidence. |