HIPAA Architecture for an LLM Voice Agent

A voice agent for a clinic hears a name, a date of birth and a reason for calling, sends that speech to a transcription model and a language model, reads it against a chart, and writes an appointment back. Every one of those hops touches protected health information. The HIPAA question is not how to keep PHI away from the model. It is how to account for every place it goes.

That reframing matters, because the tempting architecture, the one that promises the model never sees a patient detail, is mostly fiction for a system that has to book a real appointment for a real person. What follows is the architecture we actually run: where PHI travels, what contract sits over each hop, how little of it we keep, and for how long. I work on this layer, so the figures are the ones in the code, not a brochure.

None of it leans on the phrase HIPAA compliant, because that phrase is a feeling. The parts a security team can act on are a signed agreement, a named set of controls, and a retention window that deletes on schedule.

PHI moves through a short chain on every call: the telephony carrier, a speech-to-text model, a language model, our own booking layer, and the practice system it writes to. Each is a place a name or a symptom exists for a moment, and each is a place the law cares about.

Where it goes	What it is there	The control
Telephony carrier	Call audio in transit	Agreement in place, TLS in transit
Speech-to-text	Audio turned into a transcript	Agreement, no training on the data
Language model	Transcript and tool arguments	Agreement, no training on the data
Booking layer	The record read and written	Encrypted, audit-logged, in-region
Practice system	The appointment written back	Official API, scoped credentials

The honest reading of that table is that PHI does reach the models. A transcript is PHI, and a language model cannot book from a transcript it was never given. So the design is not absence, it is accountability: a contract over each hop, the smallest amount of data that makes the turn work, and a short clock on anything kept. That last stage, the write path, is also where the record is created, which is what puts the whole system on a particular side of a legal line.

Conduit, or business associate

HHS draws a line between a business associate and a mere conduit. A conduit only transports data, the way a courier carries a sealed envelope, and needs no agreement. Anything that creates, receives, maintains or reads PHI is a business associate, and needs a signed one.

A voice agent is on the wrong side of that line for a conduit, and not narrowly. It transcribes the call, resolves the caller against a record, reads availability, writes an appointment and stores a transcript.

Under HHS guidance none of that is transient transport, so a business associate agreement is required rather than optional, and 2care signs one before any protected health information is exchanged.

The logic runs one hop further. Every subprocessor that handles PHI on our behalf, the carrier and the models in the call path, is itself under an agreement, because a business associate agreement is only as real as the ones sitting behind it.

What the model and voice providers are held to

The providers in the PHI path are held to two things a clinic can check: a signed agreement, and a commitment that patient data is not used to train any model. Those are contractual facts, not toggles, which is the whole point. An architectural promise that PHI never reaches a model would be easier to make, and false.

What the design does control is how much the model sees. A question that is not a booking, opening hours or what to bring, is answered by a model handed only the clinic's own attached documents and the caller's question, never the patient's chart. A booking tool is called with the specific fields it needs, a provider id and a slot, not the whole record. Anything clinical is handed to a person rather than reasoned about, the boundary described in what an escalation contains. The model is given what the turn requires and no more, which shrinks the blast radius of any single call without pretending the radius is zero.

What we store, and for how long

Every tool call the agent makes is recorded, request and response, so a booking can be reviewed and a bad turn debugged. Those rows hold real PHI: names, dates of birth, phone numbers, free-text symptoms. Keeping them is a deliberate bargain, and the other half of the bargain is deleting them.

90 days

default life of a tool-call evidence row before a daily sweep hard-deletes it

128,000 bytes

the cap on any single logged payload, so one response cannot balloon the store

0

standing access to those rows without an admin role

3

regions the data never leaves

Reads of that evidence are gated to an admin role, individual sensitive fields can be redacted at the point of logging, and any single payload is capped at 128,000 bytes so a runaway response cannot bloat what is retained. A daily sweep hard-deletes rows past 90 days across the store, rather than hiding them behind a flag. Call audio and transcripts sit under a separate clock the practice sets, and a deletion there is carried through to backups within the stated window. Retention that runs on a schedule, and can be checked, is worth more than a policy nobody enforces.

Telemetry that assumes its payload is PHI

Most observability leaks quietly. A tracing tap that streams the agent's prompts and tool arguments to a third-party dashboard is exporting PHI to a subprocessor nobody signed for. Ours starts from the opposite assumption.

The deep tracing view, the one that shows every model and tool call inside a single turn, is off by default. When it is switched on it points only at an in-network collector, never a third-party endpoint, and the spans, which do carry PHI, are redacted there and kept on their own clock. The system of record is the primary database, not the trace store, so the trace is a debugging convenience that can be turned off without touching the audit trail. Treating the observability stack as PHI, rather than as harmless metadata, is the part most systems skip.

Residency you cannot break by accident

Data residency fails the day it depends on a developer remembering to set a region. It is enforced a level lower here, so a mistake in application code cannot move a European patient's record to the United States.

Each region runs as its own cell with its own datastore, and cross-region access is denied at the cloud account level rather than by convention, so a query reaching across a boundary is refused by policy before it runs. Vendor keys live in an encrypted secrets store, fetched at runtime by the compute role and cached for 5 minutes, never baked into an environment file or the infrastructure state. Each cell resolves its own region, so a hardcoded value cannot route one region's traffic through another. US data stays in the US, EU and UK data in the EU and UK, Indian data in India, with no silent path between them.

Where this architecture still has limits

Three honest limits, because a HIPAA page with no failure mode reads as marketing to anyone who has run a security review.

2care is not through a SOC 2 Type II audit yet, and does not claim the badge before it is earned. The controls are real and reviewable today; the attestation is still in progress, and the page will say so the day that changes.

PHI genuinely reaches third-party models and carriers, so the guarantee at those hops is contractual, an agreement and a no-training commitment, not architectural absence. A practice that requires no patient data to touch any outside provider at all cannot be served honestly by this design, or by most voice agents.

And the retention that protects patients also limits forensics. A row swept at 90 days is a row nobody can inspect at a hundred and twenty. That is a real trade, and the deletion is the right side of it, but it is a trade rather than a free win.

Why the boundary is an artifact, not a claim

The reason 2care leads with a signed agreement and a named list of controls, rather than the words HIPAA compliant, is that a security team cannot act on a feeling. It can act on an agreement it can read, a retention window it can verify, a data-flow diagram it can follow, and a sub-processor list it can check.

Every claim in this post is one of those. The agreement that predates the first call, the 90 day sweep that runs whether or not anyone is watching, the account-level rule that keeps regions apart. The security review pack on the compliance page is this same architecture written for a procurement reviewer, under NDA. The test of a HIPAA architecture is whether a stranger on your security team can verify it without having to trust us, and that is the bar 2care builds to clear.

Frequently asked questions

Will you sign a BAA?

Yes, and before any protected health information is exchanged, not after go-live. The agreement also binds our own subprocessors, the carrier and the models in the call path, because a business associate agreement is only meaningful if the agreements behind it exist as well.

Does patient data reach the language model?

Yes. A transcript is protected health information, and the agent cannot book from one it was never given. What the design controls is scope and contract: the model sees only what the turn needs, under a signed agreement, and the data is not used to train any model.

Do you train models on our patients' data?

No. Patient data is not used to train shared models, and the providers in the call path are held to that alongside the agreement. It is a commitment you can hold us to, rather than a setting that can quietly be flipped.

Where is our data stored?

In the region you operate in: the United States for US practices, the EU or UK for European ones, and India for Indian deployments. Cross-region access is denied at the cloud account level, so data does not move between regions without contractual safeguards in place.

How long are call recordings kept?

Call audio and transcripts are retained for the window the practice sets, and deletion reaches backups within it. The separate tool-call evidence kept for quality review is hard-deleted on a 90 day default sweep that runs daily.

A count you can run this week

Before you evaluate any voice vendor, list every place a patient detail travels on a single booking call: the carrier, the transcription model, the language model, the database, the practice system. That is 5 systems on one call. For each, ask for the signed agreement and the retention window. The length of that list, and how many boxes a vendor can fill, tells you more than any compliance logo on the page.

Bring your security team to a demo and ask for the review pack behind this post: the architecture overview, sub-processor list, data-flow diagram and retention policy. Start at the security and compliance page, or book a demo and bring the hard questions early.