> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hana.health/llms.txt
> Use this file to discover all available pages before exploring further.
# Clinical Safety Architecture
> Ensuring patient safety through calibrated confidence, hallucination reduction, and rigorous training data controls in voice-based clinical AI.
## Ensuring Safety
Ensuring patient safety is paramount in deploying voice-based clinical AI. Unlike text tools where clinicians review outputs before they reach patients, HANA speaks directly to patients in real-time. To uphold rigorous safety standards, we incorporate advanced methods that reduce errors, mitigate risks, and enhance reliability. Our dual-model architecture and constrained generation collectively improve accuracy, reduce hallucinations, and ensure well-calibrated, trustworthy outputs.
## Improved Calibration
Calibrated confidence refers to the alignment between the system's predicted confidence scores and its actual observed accuracy. A well-calibrated system provides confidence levels that clinicians can reliably interpret as true indicators of prediction correctness. Unlike models prone to overconfidence, especially at higher confidence intervals, a calibrated model avoids falsely reassuring clinical teams with unjustifiably high certainty.
Improved calibration directly supports safer and more informed clinical decisions, crucial in high-stakes scenarios where accurate confidence assessment helps prevent potential patient harm due to over-reliance on flawed outputs.
### How HANA Achieves Calibration
**Pre-Call Reasoning Engine:**
* Generates clinical question sets with associated confidence weights per question
* Each question tagged with expected information yield score
* Protocol branching decisions carry explicit confidence thresholds
* If confidence falls below threshold, system escalates to human review rather than guessing
**Real-Time Conversation Engine:**
* Monitors response quality against pre-computed expectations
* Tracks semantic alignment between patient responses and expected clinical patterns
* Flags unexpected responses for post-call clinical review
* Never improvises clinical guidance — falls back to approved protocol language
### Confidence Scoring
Scoring Dimensions:
* **Semantic Match (0.0–1.0)**: Does the patient response match the expected clinical entity?
* **Completeness (0.0–1.0)**: Did the patient provide all required information?
* **Consistency (0.0–1.0)**: Does the response contradict prior answers?
Composite Score = weighted\_avg(semantic, completeness, consistency)
* IF composite \< 0.7 → Re-ask with clarification
* IF composite \< 0.4 → Flag for human follow-up
* IF composite \< 0.2 → Graceful handoff to staff
### Calibration Validation
HANA validates calibration through continuous production monitoring:
* **Binned accuracy analysis**: Group predictions by confidence level and measure actual accuracy per bin
* **Expected Calibration Error (ECE)**: Track weighted average gap between predicted confidence and observed accuracy
* **Per-protocol calibration**: Each clinical protocol maintains its own calibration curve, updated with every completed conversation
* **Drift detection**: Automated alerts when calibration degrades beyond acceptable thresholds
## Reducing Hallucinations
By constraining the conversation engine to pre-approved response templates and grounding all entity references in verified data sources, HANA minimizes individual model biases and substantially reduces hallucinations.
### Anti-Hallucination Architecture
```mermaid theme={null}
graph TB
INPUT["Patient Conversation"] --> L1
subgraph Stack["Safety Stack"]
direction TB
L1["Layer 1: Protocol Constraints\nApproved questions only · No freeform advice"]
L2["Layer 2: Entity Grounding\nEHR validation · Drug DB · SNOMED/ICD-10"]
L3["Layer 3: Output Validation\nSemantic check · Post-utterance verification"]
L4["Layer 4: Human Oversight\nFull logging · 24h flagged review"]
L1 --> L2 --> L3 --> L4
end
subgraph Agents["Observational Safety Agents (Rule-Based)"]
DS["Direct Signals\nSelf-harm · Crisis · Medication misuse"]
AS["Associative Signals\nCompound risk patterns"]
VCF["Vocal + Content Fusion\nText-prosody discrepancy"]
end
DS --> ESC["Escalation Protocol"]
AS --> ESC
VCF --> ESC
Agents -.->|Parallel Monitoring| Stack
```
**Layer 1: Protocol Constraints**
* Only approved questions can be asked
* Only protocol-defined branches can be taken
* No freeform clinical advice generation
**Layer 2: Entity Grounding**
* All patient references grounded in EHR data
* Medication names validated against drug database
* Clinical terms mapped to SNOMED CT / ICD-10 ontologies
**Layer 3: Output Validation**
* Real-time semantic check against protocol
* Post-utterance verification before voice delivery
* Automated flagging of ungrounded statements
**Layer 4: Human Oversight**
* All conversations logged and auditable
* Flagged conversations reviewed within 24 hours
* Protocol updates triggered by error patterns
### Hallucination Monitoring
Metrics below are measured continuously across production conversations and updated as conversation volume grows. Current figures reflect internal evaluation across pilot deployments.
| Metric | Target | Observed |
| ------------------------------- | ------- | -------- |
| Ungrounded clinical entity rate | \< 0.1% | 0.03% |
| Protocol deviation rate | \< 0.5% | 0.2% |
| Patient-reported misinformation | 0 | 0 |
| Automated safety flag rate | \< 2% | 1.1% |
These metrics are derived from automated evaluation pipelines (LLM-as-judge + rule-based validation) and clinical team review of flagged conversations. Methodology and sample details available upon request.
## Observational Safety Agents
Beyond the conversation engine's built-in safety constraints, HANA deploys independent observational agents that monitor every conversation in parallel. These agents are architecturally separate from the conversation engine — they observe but do not generate responses — providing an independent safety layer.
### Why Rule-Based, Not LLM-Based
A critical design decision: HANA's observational safety agents are **rule-based, not LLM-powered**. This is intentional:
* **Full traceability**: Every risk flag traces back to the exact words, phrases, or patterns that triggered it. No black-box reasoning in safety-critical decisions.
* **Deterministic behavior**: Given the same input, the safety agent always produces the same output. No stochastic variation in risk detection.
* **Auditability**: Clinicians and compliance teams can inspect exactly why a conversation was flagged, what threshold was triggered, and what protocol was activated.
* **Regulatory defensibility**: In clinical safety, you need to explain *why* a decision was made. Rule-based systems provide this by construction.
### Risk Detection Architecture
**Direct Signal Detection:**
* Explicit self-harm language, suicidal ideation expressions, and crisis indicators
* Medication misuse signals (e.g., "I took too many pills")
* Acute distress indicators requiring immediate clinical intervention
**Associative Signal Detection:**
* Compound signals that individually seem benign but together indicate risk — for example, expressions of hopelessness combined with questions about methods or locations that suggest harmful intent
* Context-dependent risk patterns developed from clinical literature and clinical team collaboration
* These associative rules were developed over months of work with behavioral health clinicians across diverse patient populations
**Vocal + Content Fusion:**
* When vocal intelligence detects prosodic stress markers and the rule-based agent detects risk-adjacent language, the combined signal is escalated even if neither alone would trigger
* Text-prosody discrepancy (patient says "I'm fine" with flat affect and vocal tremor) generates a clinical review flag
### Escalation Protocols
Each healthcare organization configures escalation protocols during onboarding:
* **24/7 clinical coverage**: If the organization has round-the-clock staff, the agent can initiate a warm transfer to a clinician
* **Emergency services**: For organizations without 24/7 coverage, the agent follows a defined protocol — acknowledges its limitations, provides emergency contact information, and notifies the clinical team
* **Clinical team notification**: All risk flags generate structured notifications with the triggering signals, conversation context, and patient information
HANA's safety agents are guardrails, not clinical tools. They detect risk signals and activate clinic-defined protocols. They do not provide clinical assessment, diagnosis, or therapeutic intervention. The clinical response is always the responsibility of the healthcare organization's care team.
***
## Training Data Safety
To further ensure the safety and compliance of our clinical AI systems, we apply rigorous controls to the data used in training and evaluation.
### Data De-identification
* All real-world clinical data undergoes thorough de-identification, removing or obfuscating PII and PHI in accordance with HIPAA Safe Harbor and Expert Determination standards
* Re-identification risk assessment performed on each de-identified dataset
* Every de-identification operation logged with timestamp, method, and verification status
### Synthetic Data Generation
* Expert-designed clinical conversation templates covering common and edge-case scenarios
* Demographic diversity across age, gender, language, and cultural backgrounds to reduce model bias
* Complexity gradients from simple appointment confirmations to complex multi-condition medication reviews
* Adversarial examples with deliberately ambiguous patient responses for robust edge-case handling
### Scenario Coverage Without Risk
* By blending de-identified data with synthetic examples, we ensure high scenario diversity while eliminating risks of privacy breaches or data leakage
* Training data distributions monitored for demographic, geographic, and clinical condition balance
* All training data practices audited against HIPAA, GDPR, and SOC 2 Type II requirements