02.1
ScoringTwenty-two scorers, seven dimensions
Cognitive phase, perception focus, skill usage, behaviour triggers, empathy, length and a model-as-judge pass. The dimensions are about how a response reads to a person.
Human-like
#What it is
Sentinel evaluates LLM output. Suites, cases, scorers, baselines and red-team generation, as a Go library with an optional Forge integration.
#What it does
- Seven scoring dimensions aligned with the Cortex human model: cognitive phase, perception focus, skill usage, behaviour triggers, empathy, length and LLM-as-judge.
- 22 built-in scorers with configurable thresholds, plus custom scorers.
- Eight scenario types: factual, creative, safety, summarization, classification, extraction, conversation and reasoning.
- Baselines: save a run and detect regressions across prompt versions and model changes, which is the difference between "this feels better" and "this is 4% worse on extraction".
- Red team: five attack generators covering prompt injection, jailbreak, PII extraction, hallucination and bias, with bypass detection.
- 16 plugin hooks and three store backends: memory, SQLite, PostgreSQL.
#The uncomfortable finding
Running the red-team suite against Shield, the boundary layer of plain deny lists consistently catches more than the classifier layers. Semantic detection is the interesting part and the written-down list is the part that works. I keep expecting that to change and it has not yet.