Case study · Rimbalo

Speech recognition that works in airplane mode, on Android and iOS

Rimbalo is dictation for lawyers and financial advisors, on Android and iOS. The brief was demanding on purpose: everything from microphone to finished document runs on the phone, nothing leaves it, and the app never writes a word the user didn't say.

• 39 minof real recordings played through the app as one session on three phones, the oldest from 2016
• 81%of the code written once, in shared Kotlin
• 0network permissions in the Android build; a build check fails if one appears
• 0.29 smedian wait for text after speaking, offline, on an iPhone SE 3

The brief

• Professionals whose clients' words can't go to someone else's server, so speech recognition has to happen on the phone, with every radio off.
• Client names and legal citations spelled the user's way, not the model's.
• It has to keep up with speech on phones people actually own, not just this year's flagship.
• The finished output is a document a lawyer signs, so nothing in it can be invented.

What I delivered

All of it, from the model to the store listing. That matters more than it sounds: on a project like this, the problems live in the gaps between the model, the app and the phone, and one person who owns all three closes them faster than three specialists handing work across.

1. The model, chosen by measurement. Two NVIDIA speech models at every quantisation, scored on the same 186 clips before anything was built on either. The smaller English model plus the user's own word list beat the bigger model's download of nearly 500 MB.
2. The model, made to run on phones. Converted to LiteRT, then fixed three times for failures that only appeared on real devices. Each fix was proven numerically identical to the original before it shipped.
3. The engine, in shared Kotlin. Voice activity detection, audio features and a decoder that matches NVIDIA's reference token for token, held there by a test in CI. Android and iOS run the same code.
4. The app. Compose Multiplatform for both platforms: dictation, voice commands, word lists, and Legal and Financial Advisor packs that build the finished document from what was said, with subscriptions through both stores.
5. Delivery and release. Models outside the app binary, through Play Asset Delivery and Apple-hosted Background Assets. Store listings, App Review notes and the review recording for both stores.

The hard part, briefly

At half precision, two unrelated phone GPUs, a Pixel 7 and an iPhone, returned the same wrong answer, identical to 16 digits. Two different driver stacks agreeing on garbage ruled out a driver bug and pointed at the model itself: real speech pushed a value inside LayerNorm past what half precision can hold. The fix was exact mathematics, not a workaround, and output similarity went from −0.03 to 0.9997.

Then I tested it the way a user would: 54 real recordings, 39 minutes of speech, played through the app as one continuous session on three phones, the oldest a 2016 Pixel. Every phone kept up for the whole session, and memory stayed flat.

The engineering notes have the details, the tables and the other two fixes.

The feature I took out

The first design had an on-device language model write summaries. I measured three candidates before building on any of them. The best one still put a false statement in 22% of routine summaries, and prompt changes didn't fix it. Nothing on the phone could catch those errors before a lawyer signed the note, so the model came out and every section is now built by rules from the user's own words.

What this could look like for you

• Your product has to work without a connection

  A device in a barn, a basement, a backyard or a plane. I'll put the whole pipeline on the phone and measure it on the hardware your customers own, starting with the oldest phone you support.

• You want AI in the app, but the data can't leave it

  I'll measure whether a model on the phone is good enough for your feature, in numbers your team can check, including when the answer is no.

• Your problem crosses layers

  Audio, a model, the app and two phone platforms that all have to agree. You get one engineer who works across all of them, rather than a meeting between the people who each own one.

• Your launch has a date

  Rimbalo's first release went from first commit to both stores' review in 21 days, with the measurements to show no step was skipped to get there.