Bounded AI Red-Team Exercise

Authorize the target and blast radius

Name the application components, identities, data, tools, models, environments, attack classes, test window, prohibited techniques, rate and cost limits, evidence rules, and emergency contacts. Exercise the assembled system—not only model output—including retrieval, memory, tool selection, argument validation, authorization, side effects, logging, and recovery.

Prioritize realistic abuse cases such as direct and indirect prompt injection, sensitive-context extraction, unsafe content, policy evasion, data poisoning, excessive agency, privilege misuse, cross-user access, tool-result spoofing, denial of service, and uncertain write outcomes. Use inert canaries, synthetic protected data, reversible targets, and least-privilege test identities.

Use explicit rules of engagement

Assign a test lead, system owner, safety observer, evidence custodian, and incident authority. Every finding needs a minimized reproducer, affected boundary, observed impact, severity rationale, detection evidence, containment status, and retest owner. Do not publish exploit detail that would create avoidable harm before remediation.

Authorization and approver: [REFERENCE -> ROLE]
Target and excluded systems: [EXACT BOUNDARY]
Environment and test identities: [ISOLATED/CONTROLLED -> LEAST PRIVILEGE]
Threats exercised: [INJECTION | EXFILTRATION | TOOL ABUSE | AVAILABILITY | OTHER]
Allowed data and canaries: [SYNTHETIC VALUES ONLY]
Prohibited actions: [REAL HARM, UNAUTHORIZED ACCESS, PERSISTENCE, PUBLIC DISCLOSURE]
Rate, cost, time, and tool budgets: [LIMITS]
Observers and escalation: [ROLE -> CONTACT PATH]
Owner and cadence: [TEST LEAD, SYSTEM OWNER, SAFETY OBSERVER -> DATE]
Stop criteria: [HUMAN HARM | REAL DATA | OUT-OF-SCOPE ACCESS | UNCONTROLLED SIDE EFFECT]
Evidence handling: [REDACTED STORE -> RETENTION]
Recovery and rollback: [ISOLATE -> REVOKE TEST ACCESS -> RESTORE -> RECONCILE]
Finding closure: [FIX -> REGRESSION CASE -> INDEPENDENT RETEST -> OWNER]

Expected evidence and verification

Expected evidence includes authorization, rules of engagement, environment proof, timestamped redacted traces, findings, containment actions, regression cases, mitigation owners, and independent retest results. Verify that controls fail safely at the application boundary, detectors alert as designed, unauthorized tool calls never execute, and uncertain side effects are reconciled before testing resumes.

Stop immediately on credible human harm, unexpected real personal or confidential data, out-of-scope access, loss of test isolation, uncontrolled cost or load, or a side effect that cannot be confirmed. Recovery means isolating the workflow, disabling or narrowing the test identity and tools, preserving redacted evidence, reconciling external state, restoring a verified configuration, notifying the incident authority, and resuming only after explicit approval.