Non-determinism - guidance vs hard rules

Triggers tells you when an agent should call consult(). This page is about a trap that sits underneath it: if you leave that decision to the model, it is not reliable.

The same soft instruction - “consult a human when the stakes are high” - produces different behaviour across models, and even across runs of the same model with the same input. Smaller and faster models under-consult relative to larger ones. In our own trials, a cardiac-risk question was waved straight through until the tester pushed back a second time - the model simply didn’t judge it worth a human on the first pass.

Caution

If a person’s safety depends on the model choosing to consult, you have a non-deterministic safety gate - which is to say, no gate at all. Move that decision into code.

The convention: guidance vs hard rule

Split every “should we consult here?” decision into one of two buckets, and treat them differently.

Hard rule - decided in your code

For any category where a miss is unacceptable - irreversible actions, regulated domains, anything touching money, auth, PII, or safety-of-life - the decision to consult must live in your code, not in the model’s prompt. A code gate fires 100% of the time for the category it guards: zero model-dependence, zero run-to-run swing.

This is the single most important idea on this page. The deterministic gate is exactly the should_consult router from Triggers - a plain function over tags / action-kind / hedge-detection that your agent calls before it acts:

# The gate is code, not a prompt. It cannot "forget" to fire.
if should_consult(user_question, inferred_tags, intended_action, llm_draft):
    answer = consult(
        question=user_question,
        require_human=True,          # no silent LLM green-light on a gated call
        target_responses=2,
        approval_method="consensus",
        wait_seconds=600,
    )
    ...

Because the branch is code, it is deterministic: the gated category consults every time, on every model, on every run. You have removed the variance for the cases you cannot afford to get wrong.

Guidance - left to the model

For the gray zone - where a code gate would over-trigger and burn credits on questions the LLM handles fine - you can lean on the model. But treat it as guidance: expect variance, make the instruction as explicit as you can (templates below), and never put an unacceptable-miss decision here.

Prompt templates for the guidance path

When you do rely on the model to decide, an explicit, enumerated policy beats a vague “use your judgment.” Drop a block like this into the system prompt:

CONSULT POLICY
You have a tool, consult(), that asks a vetted human expert.
Call consult() BEFORE answering whenever ANY of these is true:
  - the question is legal, medical, financial, tax, or compliance in nature;
  - acting on a wrong answer would be costly or hard to undo;
  - you are about to state a fact you cannot verify from the provided context;
  - your own draft contains a hedge ("I'm not sure", "consult a professional",
    "it depends", "generally speaking");
  - the user has asked you to double-check or get a second opinion.
When in doubt, consult. A needless consult costs one credit; a confident wrong
answer in these areas costs far more. Do NOT reassure the user that something is
fine in these areas without a human answer.

A lightweight self-check, run on the model’s own draft before it reaches the user, narrows the swing further:

SELF-CHECK (run on your draft, not the user's question)
If acting on this draft without a human review could harm someone, break a
rule, or be expensive to reverse - stop and call consult() instead of sending.

These templates reduce the variance. They do not remove it - an LLM’s discretionary judgment is never fully deterministic. That is precisely why the unacceptable-miss categories belong in the hard-rule bucket, not here.

Recommended config by stakes tier

Pick the call settings by how much a wrong answer costs - not by how confident the model sounds. The same settings make the outcome reliable regardless of which model your agent runs, because the gate is in code and require_human removes the LLM-fallback escape hatch.

Stakes	Gate in code?	require_human	target_responses	approval_method
Trivial (draft, suggestion)	No consult	-	-	-
Costly to undo (send, post, stage)	Yes	False	1	fastest
High-stakes judgment (customer-facing, money-adjacent)	Yes	False	2-3	consensus
Irreversible / safety / regulated	Yes	True	3	all_must_agree

Tip

On a gated call, set require_human=True. You would rather the call fail and your agent retry than have an LLM fallback quietly green-light the one decision you gated specifically because a human had to see it.

Reducing the swing

There is a concrete, testable claim here: moving a category from guidance to a hard rule drops its consult-rate variance to zero. Where a soft-prompt model might consult on, say, 6 of 10 borderline cardiac questions (and a faster model on 2 of 10), a code gate keyed on the medical tag consults on 10 of 10, every model, every run. The swing for that category is gone because the decision left the model.

For the residual guidance zone, the templates above plus routing the decision through a stronger model (even if a cheaper model does the rest of the work) narrow the remaining spread. Measure it on your own traffic: log how often each category actually fired a consult, and promote any category with an unacceptable miss rate into the code gate.

The honest summary: you cannot make an LLM’s judgment deterministic, so don’t ask it to be the safety gate. Encode the decisions you can’t afford to miss; guide the rest.

Next step

Aggregation patterns → - once a consult fires, how to fold one-or-many human answers back into your agent’s flow.