Promote measured behaviour, not a convincing demo.
A fixed synthetic record compares a candidate with its control across six defensive scenarios. The regulated-SOC portfolio separately requires ten adversarial scenario classes for each persona. Deterministic checks, isolation policy and safety vetoes run before an accountable person can approve release.
- Scenarios
- 06 matched
- Objective checks
- 18 passed
- Safety vetoes
- 00
- Decision
- Human approval
Uncertainty and escalation
Separate illustrative comparison: six matched, fabricated binary classifications, not the release-gate results above. No model ran. Lower latency and fewer decided misses can come at the cost of more work deferred to analysts.
| Measure | Control | Candidate |
|---|---|---|
| Brier score (lower is better) | 0.233 | 0.118 |
| Calibration error (5 fixed bins) | 0.167 | 0.100 |
| False negatives among decided malicious cases | 50.0% | 0.0% |
| Abstention rate | 16.7% | 33.3% |
| Malicious cases deferred | 1 | 1 |
| P50 latency (ms) | 600 | 300 |
| P95 latency (ms) | 900 | 450 |
Brier score and calibration error use all six predictions, including abstentions. False-negative rate excludes abstentions; deferred malicious cases remain visible, not counted as resolved. Percentiles use nearest rank. Small synthetic samples cannot establish calibration, effectiveness or production thresholds.
Deterministic result
Candidate is eligible for human review.
Eligibility is not approval. A customer deployment must use representative reviewed data, accepted thresholds and an accountable release owner.