Key Points
- Four layers: apps, an on-phone engine and encrypted vault, an AI gateway, and (planned) a server linked to ABDM.
- The model drafts in the cloud; verification runs on the phone.
- Nothing reaches the signed record without passing the rules, and a signed record never changes.
High-Level Design
Apps
Doctor appListen, review, sign, PDF · built
Desk appRegister, consent, vitals
Pharmacy appDispense what was signed
Patient appRecord, reminders, emergency card
On the phone
Audio pipeline16 kHz capture, voice activity detection, chunks of up to ~10 s
Rules engineName matching, verification, safety checks, note assembly
Encrypted vaultConsultations, signed records, catalogue
Gateway
AI gatewayHolds all service keys, routes speech and drafting, meters usage, enforces limits
AI services
Speech to textTranslate and code-mixed modes, India-hosted
Drafting modelSarvam-105B, India-hosted
Planned server
Record serverSync, backup, sharing between apps · India region
ABDM linkABHA, health facility and professional registries, FHIR R4, consent manager
Drug registryNational Drug Registry lookups
One Consultation, End to End
- Swipe to start. Consent is already recorded at the desk. Listening begins.
- Capture and chunk. The phone records 16 kHz mono audio; a voice-activity detector cuts it at natural pauses into chunks of up to about 10 seconds.
- Transcribe. Each chunk goes through the gateway to speech recognition in translate mode, which returns English in natural word order.
- Gate. If the English contains a catalogue name, a molecule, a number or a dosing word, the same audio is sent once more in code-mixed mode to keep the original words and numbers. The last chunk after Stop is sent to both at once.
- Live strips. As chunks return, the screen fills two strips: "Already taking" (what the patient says they take) and "Today's orders".
- Draft. On Stop, both transcripts go to the drafting model, which returns a structured note: complaints, history, examination, diagnosis, current medicines with an action, and prescription lines with every field.
- Verify on the phone. The rules engine checks every line and assigns a state: verified, confirm, or cannot sign.
- History paragraph. Written only from verified facts.
- Fallback. If the model fails or takes longer than 10 seconds, a rules-only note is shown and every medicine needs confirmation.
- Review and sign. The doctor resolves flagged lines and slides to sign.
- Store and print. The signed record, each line's spoken words, its interpretation, every reason given and the catalogue source are stored; the prescription PDF is generated.
| Measure | Value | Status |
|---|---|---|
| Name matching per consultation | about 0.16 s | Measured on a laptop; target under 0.2 s on the phone |
| Note on screen after Stop | 5 s or less | Target, not yet measured on the phone |
| Model fallback threshold | 10 s | Design rule |
| Lines that block signing | 3 or fewer | Target; a real block is never downgraded to meet it |
| Invented medicines allowed | 0 | Any invention fails a test run |
Cost and Connectivity
| Question | Answer |
|---|---|
| Speech cost per consultation | About ₹0.50 a minute for translation alone, about ₹0.81 a minute with the second original-language pass[1]. A two- to five-minute consultation costs roughly ₹1 to ₹4 in speech fees. |
| Drafting cost per consultation | Not yet measured; it will be measured on recorded consultations before any pricing decision. |
| What if the internet is slow or down? | The demo needs internet to transcribe. If the drafting model fails or takes longer than 10 seconds, the doctor gets a rules-only note in which every medicine must be confirmed. There is no offline mode yet; whether one is needed is a question for the pilot. |
| Can the model run on India's own infrastructure? | Yes in principle: the drafting model's weights are open[2], so it can be self-hosted if a hospital or the law requires it. |
Low-Level Design: The Data Model
| Entity | Holds | Rule |
|---|---|---|
| Patient | Identity, ABHA link, allergies with reactions, long-term conditions, "also taking" list | A phone number is never treated as unique; patients are never merged automatically |
| Consent | What was consented to, consent text version, who recorded it, when | No recording without it |
| Consultation | Doctor, patient, start and stop, vitals from the desk | One consultation, one note |
| Transcript chunk | Audio reference, English text, original-language text, timing | Audio deleted after signing (planned for pilot) |
| Draft line | Brand, generic, strength, dose, grid, frequency, food timing, duration, when-needed condition and daily maximum, route | Every field links to the words that support it |
| Verification result | State per line, rule that fired, candidate choices | The model's label is checked, never silently changed |
| Signed record | Final lines, spoken words, interpretations, reasons, catalogue sources, signature, time | Immutable; a correction is a new signed version |
| Audit event | Every confirmation, override reason and allergy-record change, with person and time | Append-only |
Low-Level Design: Matching Medicine Names
- A spoken word becomes a confirmed name only if it matches exactly, or is close in both spelling (score of at least 0.80) and sound (at least 0.90), and beats the next candidate by at least 0.10.
- Ordinary English words are never replaced: "shall call" can only suggest Shelcal, never become it.
- Suffixes are part of the name. Pan and Pan-D, or Telma and Telma-H, are different medicines; a suffix is never dropped or added.
- Merged numbers are split only when the catalogue confirms the strength: "Pan 40100" becomes Pan 40, 1-0-0.
- Uncertain names become up to three one-tap choices, each shown with its composition. 90 sourced look-alike pairs always appear together. Context may rank choices, never fill one in.
- The matching list covers about 500 common outpatient brands with spoken forms in English, Hindi and Telugu and spellings in five Indian scripts (4,649 spellings), plus the clinic's own stock list.
Low-Level Design: The Verifier's Rules
| Check | Rule | Result |
|---|---|---|
| New order or current medicine | "I take X" is a current medicine, never a new order; a brand with dosing detail from the doctor is an order | Confirm if unclear |
| Negation | "Not Dolo" applies only to that medicine | Removed, with the words shown |
| Missing strength, dose, frequency or duration | Never auto-completed; never filled in by popularity | Confirm with choices |
| Strength the model made up | A strength not in the words is treated as invented | Blocks signing |
| When-needed (SOS) medicine | Needs a condition and a daily maximum | Choices: if fever, if pain, if needed |
| Allergy | Checked by ingredient and drug class | Cannot sign |
| Banned combination | Against 711 notified combinations, by ingredients and form | Cannot sign |
| Weekly drug written daily | For example methotrexate or vitamin D 60,000 IU | Confirm with a reason |
| Children's liquid medicines | Need concentration and weight; anyone under 18 until a paediatrician rules otherwise | Blocks until added |
| Duplicates | Same ingredient in today's orders or in what the patient already takes | Confirm |
| Drug outside the doctor's system | For example an allopathic drug by an AYUSH doctor | Warning, not a block |
Technology Stack
| Layer | Technology |
|---|---|
| Apps | Flutter (Dart) with Kotlin for Android platform code; Material 3 design |
| On-phone engine | Python rules engine embedded in the app; Dart audio pipeline |
| On-phone storage | SQLCipher encrypted database |
| Gateway | Cloudflare Workers, with key-value storage for key state and a SQL database for usage metering |
| AI services | Sarvam speech recognition and Sarvam-105B |
| Server (planned) | Python and PostgreSQL in an India region; FHIR R4 for ABDM |
| Standards | ABHA, ABDM registries and consent, SNOMED CT India extensions, NAMASTE, ICD-11 |
| Wearables (planned) | Android Health Connect and Apple HealthKit |
How It Is Built and Tested
- Rulings live in the repository. Every product decision is written down, and a build checklist fails the build if a rule is broken.
- About 1,788 automated tests across the apps and the engine, plus image tests that guard approved screens at normal size, 200% text, tablet and stress layouts.
- A frozen clinical test set of 131 cases; any change to an existing test needs the founder's written acceptance.
- A benchmark harness that replays cached transcripts, so most accuracy changes are measured without new speech costs.
- Done means verified on a real Android phone through the real app with a real gesture.
Sources
- 1Heath internal benchmarks, October to November 2026. Synthetic consultations; method described on the Models and data page. pawankalyan.dev
- Artificial Analysis. Sarvam 105B and Sarvam 30B: everything you need to know, 2 April 2026. artificialanalysis.ai