When AI Automation Cut Conversion in Half — and How We Rebuilt It Around Real Clinic Workflows

The problem was not "build a chatbot"

The clinic was handling roughly 200 calls and 230 chats per day. Operators were spending a large share of their time on repetitive communication: appointment confirmations, simple booking questions, routine service questions, and requests around test results. At the same time, they had less time to work with the patients where a strong human operator actually mattered commercially.

The owner had already tried an off-the-shelf AI communication product. The result was worse than doing nothing. The system failed to escalate even when patients explicitly asked for a person. Booking requests were still pushed to human staff, so very little real workflow was removed.

Most importantly, chat-to-appointment conversion dropped from 46% to 21%. The experiment was stopped.

Before we built anything, we agreed on what the system would not do

This was one of the most important parts of the project. Before implementation, we set expectations with the owner very explicitly.

The automation would not replace human operators. It could remove a large amount of routine work and might reduce the amount of operator time required, but humans would remain part of the workflow. The agent would handle a predefined set of routine communication. Anything outside that scope would be escalated rather than improvised.

And we did not promise that AI would generally communicate better than a strong human operator. Its practical advantage was different: it could answer almost immediately, access the same operational information every time, and complete routine actions consistently.

That changed the target from "make the bot as autonomous as possible" to "automate the parts that are safe and useful, and make the handoff reliable everywhere else."

Why we started with chat instead of voice

Voice was deliberately moved to phase two. For the languages used by this client, the generated voices we tested were still noticeably synthetic enough that patients could recognize the automation.

Chat let us solve the harder part first: intent handling, knowledge retrieval, conversation state, escalation, and workflow integration. Once those pieces work, voice becomes another interface on top of an existing decision system rather than a separate AI experiment — the same reasoning behind how we sequence our own AI Voice Receptionist projects.

Building the knowledge and retrieval layer

First: build a clinic knowledge layer

The EHR did not contain enough structured operational information to answer ordinary patient questions reliably. Details about doctors, services, branches, and equipment existed, but not in a form suitable for automated communication.

We created a separate curated knowledge base with its own editing interface. We also helped define the internal update flow: what happens when a doctor joins or leaves, when a service changes, when equipment changes, or when a branch changes its offering.

That maintenance process mattered as much as the initial data load. A knowledge base that is correct only on launch is a temporary demo. The clinic later reused the same knowledge layer for website integration — the same approach we use for the knowledge layer behind our AI Chat Agent for Clinics.

Second: use real conversations as precedents, not generic scripts

Generic scripts were not enough. Patients phrase the same intent in many different ways, services behave differently, objections differ, and the same patient can move between informational, emotional, and commercial states inside one conversation.

The clinic provided roughly two months of communication history: more than 12,000 dialogs. We used those conversations as a retrieval layer for similar interaction patterns and tested the system against both historical dialogs and simulated conversations.

This was not model fine-tuning. Historical interactions remained external memory that could be retrieved at inference time.

We embedded conversation situations, not just isolated messages

A patient message by itself often tells you the intent, but not the response strategy that worked. For example, "that is too expensive" contains an objection, but it does not tell the system how a good operator handled a similar objection.

The useful retrieval unit is therefore closer to a conversation turn or small conversational situation: patient message plus operator response, with surrounding metadata where available. This changes the retrieval question from "which text is semantically similar?" to "which previous situation resembles this one, and what response pattern was effective there?"

The embedding unit was a paired conversation turn (patient message + operator reply), carrying metadata where available: conversation outcome, intent, service, conversation stage, clinic-specific context, and quality/approved status. A small set of similar historical exchanges gets retrieved and injected as examples for response generation.

To be precise about what this is not: these embeddings support retrieval. They are not model fine-tuning.

Historical data was useful — and dirty

The hard part was not generating embeddings. It was deciding which historical answers deserved to become precedent.

The clinic history contained strong replies, mediocre replies, and bad replies. Some were too aggressive, some weak, some outdated, and some simply reflected operator habits we did not want the system to reproduce.

If every historical conversation is treated as authoritative, retrieval becomes an efficient mechanism for reproducing yesterday's mistakes. We therefore cleaned and iterated the dataset before it became a useful behavioral layer. Several rounds of historical testing and simulated conversations were required before the system communicated close to a good operator in that clinic.

How the pieces fit together

Embeddings provide similarity. The LLM provides language. The application provides control.

Retrieval was never supposed to decide policy. It only helped find useful precedents. The model handled natural language and local interpretation. The surrounding application handled boundaries, workflow state, escalation, and integration.

That separation became critical in healthcare because the dangerous failure is often not a slightly imperfect sentence. It is the system continuing autonomously in a conversation where it should have stopped.

Use embeddings for similarity and precedent. Use the LLM for language and local reasoning. Use deterministic rules for boundaries and workflow control.

Escalation was a core feature, not an exception

The failed off-the-shelf system had already demonstrated the cost of poor escalation: patients could explicitly ask for a person and the bot would continue talking.

We treated escalation as a first-class system state. For this clinic, automation stopped when a patient explicitly asked for a human, when the conversation crossed into medical advice, when symptom-based discussion would require a service recommendation, when negative/problematic input needed an operator, or when sensitive results/consultation information required human verification of the patient's right to receive it.

A serious healthcare agent therefore needs explicit non-automation logic. The design problem is not only deciding what the agent can answer. It is deciding where it must stop.

Model output should be data, not just text

The model should return the user-facing reply together with machine-readable control fields. A practical response schema can include intent, conversation stage, escalation flag, escalation reason, and confidence alongside the text that will be sent to the patient.

That makes the next system action explicit. Application code does not need to infer workflow state from prose. It also creates a useful failure mode: if the model output cannot be parsed or violates the expected structure, the system can fail toward escalation rather than improvising.

{
  "reply": "Sure — I can help with that.",
  "intent": "booking_intent",
  "stage": "booking",
  "should_escalate": false,
  "confidence": 0.91
}

Conversation state matters

A patient interaction is not a sequence of unrelated messages. It has state. A simplified lifecycle can move through stages such as new, qualifying, nurturing, booking, booked, escalated, closed, or lost.

The model can help classify the current state, but the application should own the allowed transitions. This prevents behavior such as sending a patient back into qualification after they have already moved into booking.

A communication agent becomes useful when it can actually do something

One of the biggest weaknesses of the original off-the-shelf implementation was that booking requests still went to a human operator.

We integrated the replacement system directly with the clinic's EHR booking API so it could retrieve available slots and create the booking instead of merely recognizing booking intent. That is the difference between chat automation and workflow automation through chat.

Results

The original clinic baseline was approximately 46% chat-to-appointment conversion. The off-the-shelf AI experiment reduced it to 21%.

After rebuilding the system around clinic-specific knowledge, historical conversation precedents, explicit escalation, and real workflow integration, approximately 54% of incoming chats are now handled without human escalation. Average first response time fell from approximately six minutes to below ten seconds. Chat-to-appointment conversion recovered and increased slightly above the original baseline to approximately 51%.

Those metrics have to be read together. Maximum automation was not the objective. A system that handles more conversations but destroys conversion is not a successful automation.

Lessons learned

Set the automation boundary before writing the automation

The project started with an explicit agreement that AI would not replace every operator, would only handle predefined routine communication, and would escalate everything else. That constraint prevented the usual drift toward "let the bot try."

Historical communication beats generic scripts — after cleaning it

The clinic already had thousands of examples of how its patients ask, object, disappear, return, and book. But good precedent had to be separated from bad precedent before retrieval became useful.

Retrieve situations, not merely similar vocabulary

Conversation-turn retrieval can surface both a similar patient message and a response pattern that worked in a comparable situation.

Boundaries matter more than clever replies

A slightly awkward sentence is recoverable. Giving medical advice, continuing after a patient asks for a person, or mishandling sensitive information is not.

Workflow completion is the real unit of automation

Detecting booking intent is not booking automation. The workflow is automated when the appointment has actually been created.

Never expect miracles from automation

Removing roughly half of routine chats from operators while improving response time and preserving conversion was already a meaningful result. The system did not need to automate 100% to create value.

For a similarly detailed look at what breaks and why, see how we handled long-context medical reasoning in the InnMap case study — a different clinical problem, but the same underlying discipline about knowing what a model is and isn't reliable for.

What comes next

The project is ongoing. Phase two moves into voice automation. The communication layer is also evolving toward more explicit manually configured sales logic on top of the historical-dialog retrieval layer.

That points toward a layered architecture: clinic knowledge + historical precedent + conversation state + explicit business rules + workflow integrations + LLM. None of those pieces is particularly magical. Together, they make the agent behave much more like part of a real clinic operation than a standalone chatbot.

About the client