Guardrails and red flags

The agent’s clinical safety limits and detection of critical situations

OlivIA applies two clinical safety mechanisms: guardrails (what the agent must never do) and red flags (critical situations that trigger immediate escalation).

Guardrails

Guardrails are prohibited agent behaviors. Categories:

Category	Prevents the agent from…
no_diagnosis	giving a diagnosis
no_medication	recommending or adjusting medication
no_prognosis	issuing a prognosis
no_contradicting_clinician	contradicting the clinical team

Each guardrail has a severity (warning or critical) and optional guidance. If a protocol configures none, a safe default set applies (no diagnosis and no medication as critical; no prognosis and no contradicting the clinician as warning).

How they are enforced — two layers:

1. Preventive: the rules are injected into the agent’s instructions as inviolable rules that override the protocol context and any patient request.
2. Detective (judge): before speaking, a judge evaluates the phrase the agent is about to say against each category. If it detects a violation, the phrase is blocked. The system is fail-safe: if the judge is unavailable, the phrase is treated as unsafe and not spoken.

Red flags

A red flag is a critical clinical criterion that, when detected on the call, triggers an immediate response. It is defined per protocol with a criterion and a transfer phone (up to 10 per version). Declaring a criterion is what enables its detection: if a protocol declares none, the agent has no ability to escalate by red flag.

Common suggested criteria: suicidal ideation, severe chest pain, severe breathing difficulty, severe bleeding — plus any custom criteria you define.

What happens when a red flag is detected:

1. The alert is logged and escalated immediately to the clinical team.
2. The run is closed as unresolved.
3. The agent says the red_flag.reassurance phrase (if configured).
4. The call is transferred to the criterion’s phone (or the organization’s emergency phone). If there is no number or the transfer fails, the agent closes the call safely, without medical guarantees.