The front desk
Pediatric autism clinics run on phone calls and text messages. A family gets a referral, and someone at the front desk has to reach them, explain what happens next, check which location is closest, find an assessment slot and book it. Then they answer the follow-up questions, chase the ones who go quiet, and keep the clinic's own system up to date. Multiply that across eight clinics and you get a backlog measured in months. When I started, it was about eighteen.
I didn't want to build this from a spec. So I went in on weekends and watched: which questions came up every day, where staff were copying the same information between screens, which conversations needed a human and which really didn't.
What I was responsible for
Everything outside the code: scoping, negotiating the contract, the relationship with the clinics, and a standup at 8am every day. Inside the code, I built the voice agent end to end and roughly 60% of the SMS system. Tyler Scaringella and Nick Brady built the rest with me.
8am standup, every day
Follow one text through the system
Drawn from the actual codebase. The message is an example. Click any box for what it does.
Step 1 of 8
A parent texts back
Say: “Hi, yes, can we book the assessment? Mornings are best.” The SMS provider sends it to our server as a webhook.
Decisions I'd make again
Four agents became one
The first prototype used CrewAI with four agents in a row: one to classify the message, one to schedule, one to decide on escalation, one to write the reply. That meant every text paid for four model calls in sequence. We replaced it with a single tool-calling agent in LangGraph and moved everything deterministic out of the model altogether.
The model doesn't get a vote on consent
Recording consent, verifying the guardian, working out which sibling a parent means, and handling STOP and HELP all happen in ordinary code before the agent runs. It's less clever, but those are the parts where being wrong has legal or human consequences, and plain code is easy to test.
Book with a conditional update, not a check-then-write
Two families can ask for the same Tuesday morning. A booking only succeeds if the slot is still marked free at the moment we write, and if the follow-up write to the patient fails, the slot is released. Then the booking is pushed to the clinic's system.
Read from a local copy of the calendar
Instead of calling the clinic's scheduling API mid-conversation, a background thread copies availability into our database every five minutes. It marks the day's slots taken, then re-adds whatever the clinic says is open. That keeps a slow external API out of the conversation. The cost is that the copy can be a few minutes stale, so the booking step still goes through the clinic's system.
Keyword search over a single document
Clinic information (addresses, services, what to expect at an assessment) lives in one long markdown file. The agent searches it with keyword scoring and synonyms, not embeddings. The trade-off: it misses paraphrases that embeddings would catch, but when a parent got a strange answer you could see exactly which of the hundred-odd sections matched, and why.
What was harder than expected
Threads and event loops
The server handles requests on threads, and the calendar client was async with a shared lock and cached auth token. That produced the classic Python error: a future attached to a different event loop. The fix was unglamorous. We made the sync path fully synchronous, put the token behind a thread lock, and gave each thread its own database client.
Parents send three texts in a row
“Hi.” “Can we do Thursday?” “Actually Friday.” Each text starts its own agent run, so replies can arrive out of order or answer a question that's already changed. We designed per-number locks and a short queue, then deliberately didn't build them until real traffic showed it was a problem worth the complexity.
Carriers have opinions
US texting rules mean certain words trigger automatic opt-outs. If a parent typed QUIT, the carrier could block the number without us knowing. So the server catches those words and asks them to reply STOP if they mean it. CANCEL is left alone, because parents use it to mean an appointment, not the conversation.
Testing something that talks back
Unit tests don't tell you whether a conversation goes well. So we built a harness where another model plays a parent: anxious, Spanish-speaking, confused or aggressive, each with their own situation and preferred times. Many of these run in parallel against the agent. Each run ends with an outcome code: the parent was satisfied, it hit a 25-message cap, or the agent ended it. Alongside it sat a regression suite keyed to the project's requirement IDs.
What happened
- Ran across eight clinics, handling hundreds of patient conversations a week over text and phone.
- Saved an estimated 500+ hours of staff time a month.
- Cut into a backlog that had reached about eighteen months.
- Once it was stable, I handed the system and what we'd learned to an internal engineering team.
What I'd change
- Let staff take over. Escalations went to Teams as one-way cards. Staff could see everything, but there was no clean way to pause the agent and reply as a person. That should have been in the first version.
- Run the tests in CI. We had the simulator and a regression suite, but our CI only guarded which branches could merge. Wiring the regression suite into every pull request would have caught a few things earlier.
- Keep conversation state out of process memory. Some state lived in the server process, which is fine on one machine and wrong the moment you add a second.
- Put clinic details in the database. Locations and addresses started out in code. That was fine for one organisation and painful for anything beyond it.
- Measure the time saved from day one. The hours figure is an estimate. I'd instrument it properly next time.