Case Study - A Clinical Chat That Only Ever Sees the Patient You’re Looking At

Overview

The client's clinician-facing EMR gives providers a workspace for reviewing a patient's chart. Clinicians needed a way to ask natural-language questions about a specific patient's case and get answers grounded in that patient's actual record — medications, notes, labs, and care plan.

The problem

A clinical chat tool that answers from general medical knowledge instead of the patient in front of the clinician is worse than no tool at all — it invites a confidently wrong answer into a clinical decision. The harder constraint sits underneath that: the retrieval layer has to be scoped per patient at the data-access level, not just prompted to behave that way, because a scoping mistake here is a HIPAA event, not a bug ticket. On top of that, it needed to feel immediate — a multi-second silent wait while a full record's worth of context gets assembled doesn't read as "real-time" in a clinical workflow.

Why it took a physician-engineer

Deciding what "this patient's context" should actually contain — which fields from demographics, diagnoses, medications, care plans, notes, and documents matter enough to put in front of the model, and how to weight them — is a clinically informed judgment, not a generic RAG default. So is deciding when a long-running conversation needs its context refreshed rather than just truncated: the system triggers a rolling summary every ten messages specifically so earlier clinically relevant detail doesn't silently fall out of the window. A purely technical build ships a generic document-QA bot; this needed to ship a clinically curated one.

Approach

Per-patient scoping was built as a hard boundary at the storage and retrieval layer — S3 and DynamoDB reads are already scoped to the patient in the request, not filtered after the fact — so there's no code path where a retrieval query can reach across patients. Retrieval itself uses in-Lambda cosine similarity rather than a managed vector database: a deliberate simplicity-and-cost trade-off appropriate to per-patient corpus sizes, revisited only if latency at scale demands it.

Streaming over a raw WebSocket, rather than polling or a blocking request, was chosen specifically because generation takes several seconds and a clinical chat interface needs to show progress, not a spinner. Rolling summarization was added once real clinical conversations got long enough that keeping full history in context would blow the token budget.

Under the hood

The frontend connects to an API Gateway WebSocket, with connection state tracked in DynamoDB. Patient documents are chunked and embedded at ingestion time via AWS Bedrock Titan; each chat query embeds the question and retrieves the nearest chunks from that patient's embedded set only, via in-Lambda cosine similarity. Context assembly pulls from the patient's S3 snapshot plus DynamoDB (goals, check-ins, survey responses) into a single prompt sent to the model. Every tenth message in a conversation triggers a second call to regenerate a rolling summary, keeping long clinical conversations within budget without losing earlier context.

Outcome

What used to mean searching a chart manually across notes, labs, and documents became a single natural-language question, answered in a live streaming response, grounded exclusively in that patient's own record — with no path in the system architecture for it to be otherwise.