Short-Circuiting Safety Layers for Agent Input and Output

#Abstract

Content safety for language model applications is typically implemented as a pipeline in which every detector runs on every input and a combination rule produces a scalar decision. This report argues for an ordered construction in which detectors are sorted by cost and precision and permitted to terminate evaluation early, subject to an admissibility condition. The primary benefit claimed is single-cause attribution rather than latency. A replicated measurement finding is reported that runs against the design's own framing.

#Two deficiencies of flat pipelines

The standard arrangement runs every detector on every input, then combines the results with a weighted sum, a maximum, or a small learned model, and compares the combination against a threshold. It has two deficiencies, and the second is the one that motivated this work.

Latency floor. The slowest detector sets the cost of every request, including the overwhelming majority that no detector would have blocked. For a deployment where a classifier layer costs a few hundred milliseconds, this is paid on every benign request in order to catch the rare hostile one.

Lost attribution. When all detectors run and are combined, the decision is a scalar. A high-precision pattern match and a low-confidence classifier score become one number, and no downstream consumer can recover which detector was responsible.

The consequences of the second compound. Per-detector tuning becomes impossible, because moving one threshold moves the composite for every input, including inputs that detector had no opinion about. A user who was blocked cannot be told why in terms that mean anything, since the honest answer is a weighted sum. A regression test cannot be written against a specific behaviour, only against the composite, so a test that passes tells you the aggregate has not moved rather than that the behaviour is still correct. And an incident review has nothing to review: the question after a wrongful block is always which detector fired, and a flat pipeline answers with a number.

A safety system that cannot say why it acted is not auditable, and a system that is not auditable cannot be improved except by guessing.

#Construction

Six layers ordered by cost and precision, each able to terminate:

1. Boundary, under 5ms. Deny lists over topics, actions and data.
2. Instinct, under 10ms. Injection, jailbreak and exfiltration patterns.
3. Reflex, under 10ms. Operator-authored condition and action rules.
4. Awareness, under 50ms. Personal data, topic, sentiment and intent, predominantly annotating.
5. Values, under 100ms. Toxicity, brand safety and honesty constraints.
6. Judgment, under 500ms. Grounding, relevance and compliance scoring.

The saving is a consequence rather than the justification. Ordering by cost alone would be an optimisation and would be unsound, because it would let a cheap, imprecise detector decide cases a later detector would have decided differently.

#The admissibility condition

Termination is permitted only where the layer's precision is high enough that later evidence could not have changed the decision.

Read plainly: given that this detector fired, the probability the input is genuinely unsafe must be at least 1 minus a tolerance the operator sets. A cheap detector with poor precision must not short circuit however fast it is, and stating this as a condition rather than a convention is what keeps the ordering principled rather than merely convenient.

Three consequences follow, and the third is the one that makes the condition operationally useful rather than decorative.

First, admissibility is a property of a detector against a deployment, not of a detector in the abstract. The same regular expression has different precision against a customer support assistant and against a security research tool, so the same layer may terminate in one profile and annotate in another. This is why the layer's ability to terminate is configured per profile rather than fixed in the layer.

Second, the condition is one-sided. It licenses early termination on a block, not on an allow. A detector that does not fire has said nothing about whether later detectors would, so evaluation continues. An early-exit design that also short circuits the allow path is a different and much stronger claim, and this construction does not make it.

Third, the condition is measurable and therefore falsifiable. Precision given firing is estimable from labelled adversarial evaluation, which means a layer configured to terminate can be shown to be misconfigured. That is the entire practical value of writing the condition down: without it, "this detector is precise enough to act alone" is an opinion, and with it, it is a number somebody can check and a configuration somebody can be shown to have got wrong.

#Attribution as the primary output

A terminating chain produces a decision with one named cause: layer, detector, confidence. That record is auditable, presentable to an affected user, and directly expressible as a regression test. Emerging regulatory expectations for consequential systems ask for traceability of exactly this kind [1] [2].

People do not complete a risk assessment before flinching, and the flinch is allowed to be the whole answer. What matters afterwards is being able to say which flinch it was.

#Why input filtering alone is insufficient

Direct prompt injection is the more tractable case [3]. Indirect injection, where instructions arrive inside content the system retrieves, cannot be caught by examining the user's message at all [4]. Retrieved content and generated output therefore pass through the same layers as user input, which corresponds to treating injection and output handling as distinct concerns [5].

#Profiles

Safety requirements are not uniform across deployments. Composing the six primitives into a named profile keeps configuration reviewable as a unit, rather than as flag combinations over a shared pipeline whose interactions nobody can state. This is economy of mechanism applied to policy [6].

#A finding that contradicts the framing

Across repeated adversarial evaluation against configured profiles, the boundary layer of plain deny lists accounted for more true positives than the classifier layers.

This is unwelcome for a design that leads with a cognitive framing, and it has replicated on every occasion it has been measured. It is reported because a design note that omits its own contrary evidence is not worth reading.

#What that finding does and does not imply

The result invites an overreading in both directions, so it is worth being careful about what it supports.

It does not imply the classifier layers are worthless. The layers are ordered, so the boundary layer sees every input first and the classifiers see only what survives it. Any statistic comparing their true positive counts is therefore computed over different populations, and the boundary layer's advantage is partly an artefact of position. This is the censoring problem noted in the limitations, and it is not a small correction: the classifiers are being judged on the residual after the easy cases have been removed.

It does not imply that deny lists generalise. A deny list catches what somebody thought to write down. Its precision is excellent and its recall against novel phrasing is whatever the author's imagination allowed, which is why the classifier layers exist. The measurement counts what was caught, not what was missed, and the missed set is by construction unobserved.

What it does support is narrower and still useful. In the deployments measured, a large share of genuinely hostile inputs were not sophisticated. They were direct, repetitive, and matched patterns already known, which is consistent with what systematic red teaming reports about the distribution of attempts [7]. A design that assumes the adversary is always novel will over-invest in the tail and under-invest in the cheap mechanism that handles the bulk.

The lesson I take is not that the cognitive framing is wrong. It is that the framing made me expect the interesting layers to do the work, and the boring layer did the work.

The comparison that would settle this is a shadow evaluation in which every input reaches every layer regardless of termination, so that per-layer recall is measured over a common population. That is described below as a limitation because it has been run only on samples, not continuously.

#Threats this construction does not address

The layers filter content. Several relevant threats are not content properties and are therefore outside what any arrangement of these detectors can catch.

Multi-turn escalation. Each message in a conversation may be individually benign while the trajectory is not. Every layer here evaluates a single input, so a sequence assembled across turns passes every layer. Stateful evaluation over conversation history is a different design and is not present.

Training data extraction. Prompts that elicit memorised training data are not distinguishable by topic or intent from legitimate requests for the same information [8]. Nothing in the six layers addresses this and nothing in their framing would suggest it should.

Harm from correct, permitted output. The layers decide whether output is safe under a policy. Whether the policy is right, and whether outputs it permits cause harm in aggregate through accumulation or distribution, is a question about the deployment rather than about any input [9]. Treating a content filter as an answer to that question is a category error that a well engineered filter makes easier to commit, because it produces a reassuring green result.

The evaluator as attack surface. Where a layer is itself a language model, the content it evaluates can attempt to instruct it. This is the same indirect injection problem as above, relocated into the defence, and the mitigations are structural: the evaluating model is given the content as data with no tool access and no ability to affect the pipeline other than by returning a score [10].

#Limitations

No recall-parity comparison. The claim that ordering preserves recall relative to a flat pipeline is asserted rather than measured. The experiment required is a comparison at equal recall on a common adversarial set, and it has not been run.

Censored measurement. An input terminated early is never examined by later layers, so any statistic computed over blocked inputs is censored by the layer that blocked them. Shadow evaluation of a random sample restores an unbiased estimate at sample cost and does not restore per-request attribution.

Hallucination is not addressed. Grounding scoring is one layer among six, and the underlying behaviour is a persistent property of these systems rather than a defect removable by filtering [11].

The admissibility threshold is set, not derived. The operator chooses ε. Nothing in the construction says what it should be, and the right value depends on the relative cost of a wrongful block against a missed harm, which is a deployment judgement rather than a technical one. Making that judgement explicit is an improvement over leaving it implicit in a combination rule, and it is not the same as answering it.

Comparison against alternatives is absent. No comparison is offered against a single well-tuned classifier trained on the same policy, which is what several production moderation systems actually deploy [12], nor against model-level alignment approaches that shift the work from filtering to training [13]. Those are different points in the design space, and the argument here is about the structure of a filtering layer rather than about whether filtering is the right layer to invest in.