On-device transcription

On-device transcription is speech-to-text that runs entirely on your own computer or phone, using a model stored locally, so the audio never has to be sent to a server to be turned into words. You will also see it called local transcription, offline transcription, or on-device speech recognition.

It is the strongest privacy property available in this category, and it is also the most commonly misread phrase on a meeting-tool marketing page. The reason is worth stating up front, because everything else follows from it: “on-device” is a claim about where one stage of the work happens — and the stage people are most sensitive about is usually the one still in the cloud.

The stage it refers to

Every AI notetaker runs the same four-stage pipeline: capture the audio, transcribe it, work out who was speaking, then summarize. “On-device” names the second stage. The other three each have their own answer, and a tool can give a different one for each.

Stage	Can it run on your machine?	What usually happens
Capture	Always local	Sound has to be picked up by hardware you own — a microphone, or system audio capture. This stage is local in every tool, including cloud ones, which is why “captured on your device” is a much weaker claim than it sounds.
Transcription	Yes, increasingly well	The stage the phrase actually refers to. Speech models small enough to run on a laptop or phone have become genuinely usable.
Speaker separation	Often	Diarization clusters voices acoustically and is not especially heavy, though names come from elsewhere entirely — see how notetakers know who is speaking.
Summarization	Rarely, today	This is a language model reading the transcript, and it is the stage most likely to be a call to someone else’s server.

Read the table in the order it is written and the misreading becomes obvious. A product can say, accurately, that transcription happens on your device — and then send the resulting transcript to a cloud model to write the summary. The audio never left, which is a real thing to have; the words did. And the transcript is the more concentrated version of the conversation, not the less: it is the meeting with the noise removed and the meaning intact, searchable and quotable, which is exactly why it is the artifact people worry about in who can see my meeting notes.

None of that makes on-device transcription a marketing trick. It makes it a specific, checkable claim about one step, and the useful question to ask a vendor is not “is it on-device” but “which stages are, and what leaves the machine at the end of each one.”

Where the computing happens is not where the data lives

The second thing the phrase gets asked to carry, and can’t: on-device processing says nothing about storage.

A note transcribed entirely on your laptop can still sync to a cloud account, back up, be shared with a colleague, be indexed for search, and sit in a vendor’s storage under whatever retention policy applies. The transcription was local; the transcript’s life afterwards is a separate design decision with a separate answer. Three properties get bundled together under “private” and they come apart cleanly:

• Where it is processed — on-device or in a cloud. This is what the term means.
• Where it is stored — your disk, your account in someone’s cloud, your employer’s tenant.
• Who can read it — decided by the account it lands in, not by which chip did the maths.

Local processing gives you the first. It gives you the second only if the product also stores locally, which many local-transcription features do not. It gives you the third only incidentally.

What it genuinely buys you

Real, and worth being clear about, because this is the part that isn’t marketing.

• It is the only privacy property that doesn’t depend on a promise. Every other protection in this category — no training on your data, a short retention window, encryption at rest, a named subprocessor list — is a commitment made by a company that can revise its terms, be acquired, or be legally compelled. Those commitments matter, and retention limits in particular are the sturdiest of them, because data that no longer exists can’t be repurposed. On-device processing is the version of that argument taken one step further: data that was never sent can’t be repurposed by anyone, and you don’t have to trust anything to know it.
• It works with no connection. A local model doesn’t care about hotel Wi-Fi, and there is no per-minute API cost behind it, which is why the free, self-run route is built on local models.
• It sidesteps a whole class of compliance conversation. If audio and text never leave the machine, there is no vendor to review for that stage, no data-transfer question, and nothing for a security team to approve.

What you give up

• Model size. The largest, most accurate speech models don’t fit comfortably on a laptop, so local transcription generally runs a smaller one. The gap has narrowed a lot and is smallest on clean audio; it widens on exactly the hard cases described in how accurate are AI meeting notes — crosstalk, distant microphones, proper nouns, unfamiliar accents.
• Your machine does the work. Transcription is continuous compute for the length of the call, on a computer already encoding video, sharing a screen, and running everything else you have open. On a laptop that means fans, heat, and battery.
• Language coverage is usually narrower. Local models tend to support fewer languages, and less evenly — the language pipeline caveats apply here with less room to manoeuvre.
• You get one stage. A local speech model hands you text. Getting audio into it, separating speakers, and turning the transcript into notes are separate problems, which is what makes a self-run setup a good weekend project and a poor Tuesday-morning plan.

Things that get called on-device but aren’t

• “Local-first” or “stored on your device.” A storage claim, not a processing one. The audio may still have gone out to be transcribed and come back.
• “End-to-end encrypted.” The data still leaves your machine; encryption governs who can read it in transit and at rest. Strong, and a different property.
• “We don’t train on your data.” A restriction on use, not on location — see do AI notetakers train on my meeting data, where the three meanings of “training” come apart the same way the three meanings of “private” do here.
• “Private cloud,” “your own tenant,” “edge processing.” All describe someone else’s computer with a fence around it. Sometimes a strong fence; still not your device.
• Hybrid processing. Some products handle straightforward audio locally and route harder passages, or particular features, to a server. That is a reasonable engineering choice and not dishonest, but it means “on-device” and “always on-device” are different claims. It is also the kind of behaviour that changes between releases, so it is worth re-checking rather than remembering.

Where you’ll meet it in practice

Apple’s Notes app transcribes recordings on the device, which is a genuine privacy advantage over any cloud pipeline and one of the clearer reasons to reach for it — see Canary vs Apple Notes for where that does and doesn’t help on a video call. Open-source Whisper-family models run locally through community desktop wrappers and are the standard answer for people who want nothing to leave the machine at all. Operating systems increasingly expose local dictation and speech APIs that other apps can build on. Meanwhile most hosted notetakers — bot-based and bot-free alike — transcribe in a cloud, because that is where the biggest models are.

Why real-time summaries are still cloud work

Worth separating the two halves honestly, because they are at different stages of maturity. Running speech-to-text locally is close to solved for ordinary meeting audio. Running a capable language model locally, fast enough to keep rewriting a summary at several resolutions while the conversation continues, on a machine that is simultaneously in a video call, is not — real-time meeting summarization is the stage where the hardware gap still bites. That is a description of today’s constraints rather than a permanent state of affairs, and it is the reason live-summary tools, this one included, are cloud tools.

Canary is a real-time, bot-free meeting summarizer. It captures your computer’s system audio (no bot in the call, no plugin) and shows a live, multi-resolution rolling summary — from what’s being said right now to the whole call — so you can catch up the instant your name is called.

Being straight about where it sits on this page’s axis: Canary is not an on-device product. Audio is streamed in short chunks for transcription and then discarded; what is stored is the text transcript and the rolling summaries, encrypted at rest. Capture is on-demand rather than always-on, speech-to-text and summarization run on third-party API tiers that don’t train on your content by default, and the providers are named in the privacy policy. Free-tier notes are purged after a week and account deletion is permanent. That is a set of commitments, and by this page’s own argument commitments are a weaker thing than locality. If fully on-device is a hard requirement for you, no real-time cloud summarizer meets it today, and the right move is to say so in your evaluation and run a local model — accepting that what you’ll get is a transcript rather than a live summary. Are AI meeting notetakers safe? walks through the rest of the checklist, and the complete guide to bot-free meeting notes covers the capture side.

A note on consent

On-device processing changes who can read the recording. It changes nothing about whether the other people in the call knew it was being made — and if anything it makes capture harder to notice, since there is no bot in the participant list, no banner, and no server involved to leave a trace anywhere. Local processing protects the data from third parties; it does not create agreement from the people whose words were captured. Tell participants you’re capturing the meeting: one sentence is enough, and consent rules vary by region — some require only one participant’s consent, others everyone’s.