Private AI

Local AI Transcription: Keep Audio and Transcripts Private

How to turn meetings, interviews and recordings into private transcripts, summaries and a searchable archive without sending the audio to a public API.

Audio is one of the most sensitive things a company produces. A meeting recording is not “a file”. It is unguarded speech: names, deals, medical details, HR issues, passwords said out loud, the joke that should never have been recorded. Sending that to a public speech-to-text API can be the right call when the content is already public. It is the wrong default for everything else.

Local AI transcription means speech-to-text runs on infrastructure you control, and the transcript stays there unless you choose to move it. That is one of the few private AI workflows that is both technically mature and operationally boring, which is the combination you want. If you need the work done as an engagement, that is a core path on local LLM setup for business. This article is how the system actually fits together.

What the pipeline looks like

Audio / Video
     ↓
Speech-to-Text
     ↓
Transcript
     ↓
Local LLM
     ↓
Summary / Actions / Searchable Knowledge

The first two stages are automatic speech recognition. The last two are optional and where language models earn their keep. Do not glue them into one opaque product. You want to keep the transcript even if the summary is wrong.

Whisper-type speech recognition

The practical local stack is in the Whisper family: OpenAI’s open Whisper models, faster-whisper, WhisperX, and the various C++ and TensorRT ports. They are not the only speech models. They are the ones you can actually deploy, evaluate, and run offline without a research team.

What they are good at: clear speech, common business languages, reasonably clean recordings, batch jobs.

What they struggle with: heavy overlap, strong accents on small models, domain jargon, far-field table mics, and music under speech. A larger model and a better microphone fix more of this than a prompt will.

“Whisper” is not a product you buy. It is a model family you run. The product is the pipeline: ingest, decode, store, summarise, search, retain.

Local speech-to-text versus a hosted API

Hosted speech APIs are fast to adopt and often more accurate on messy audio, especially with vendor-specific extras (custom vocabularies, telephony models). They also mean the audio bytes go to someone else.

Local STT is slower to stand up and usually a bit worse on the worst recordings. It wins when the audio must not travel, when volume would dominate an API bill, or when the site is offline.

Hybrid is allowed: internal meetings locally; a public webinar on a vendor. Write that down so people do not invent a third path in a hurry.

Audio preprocessing

Garbage in, garbage out is more true here than in chat.

Before decode:

• Format: convert to a consistent PCM rate (16 kHz mono is the usual ASR input). Do not feed a 4K video file to the recogniser and hope.
• Loudness: normalise; clip less.
• Segmentation: split long files on silence so a two-hour board meeting is not one giant decode with no checkpoints.
• Language hint: if you know the language, pass it. Autodetect is fine for mixed corpora; it is a waste of time for a German-only interview series.
• Source separation: optional. Useful when a podcast has a loud intro track. Rarely worth it on a quiet Zoom.

Store the original file. Store the normalised work copy. Do not throw away the source because a 200 MB WAV felt inconvenient. Storage is cheaper than a lost recording.

Language detection

Whisper-class models can guess the language. Trust that guess as a hint, not as a filing system. For a company that records in two languages, run detection per segment, then pin the decode language for that segment. Mixed-language meetings will still bleed. If the work is “every interview is French”, skip detection.

Timestamps

You almost always want timestamps. They make the transcript a navigable document instead of a wall of text, and they let a player jump to the moment.

Word-level timestamps are possible with some pipelines and expensive. Sentence or segment timestamps are enough for meetings. Store them in a structured format (JSON alongside a readable .txt or .vtt), not only as baked-in subtitle files.

Diarization

Diarization is “who spoke when”. It is a separate model, not a Whisper feature people think they are entitled to.

It works reasonably on distinct speakers, good mics, little overlap. It collapses when two people talk at once, when everyone sounds similar on a laptop mic, or when the number of speakers is wrong.

Label speakers as SPEAKER_00 unless you have a reliable mapping (known participants, a roster, a push-to-talk source). Manual correction after the fact is part of a serious workflow, not a failure. If your users expect broadcast-TV speaker labels on a twelve-person hybrid meeting, say no before the pilot.

Summaries and action items

This is where a local LLM sits after STT:

• short summary for people who will not read the transcript;
• decisions;
• action items with owners if the transcript supports that;
• a title.

The model should see the transcript, not the audio. Keep prompts boring and structured. Ask for JSON or a markdown template you parse. Then show the human the transcript next to the summary. Summaries without a link back to a timestamp are how errors become minutes.

Do not auto-send the summary to the whole company. Routing is a product decision. The model does not know who was in the room.

Storage and retention

Audio and transcripts are records. They need a policy:

• where the files live;
• who can list and play them;
• encryption at rest, same standard as the rest of your file estate;
• retention: “keep forever” is not a policy, it is an accident;
• legal hold, if you are in a business that has those.

Default logs should not contain transcript text. Search indexes should respect the same ACLs as the files.

Search

A folder of .txt files is already better than the recordings alone. A proper search layer is better still: full-text first, then embeddings if people ask questions rather than keywords.

“Find the meeting where we talked about the Warsaw contract” is a retrieval problem on transcripts. It is the same private search design as documents, with timestamps as citations. See private AI document search for the RAG half; do not invent a second stack.

Batch processing

Interactive “drop a file, wait” is for demos and for journalists. Companies have folders.

Run a queue: new file in an ingest bucket, job object in a database, worker pulls, writes transcript and artefacts, marks done, notifies. Idempotent jobs, so a crash does not double-bill GPU time or double-create documents. Overnight batches are how you get utilisation out of a GPU that would sit idle between meetings.

GPU vs CPU

Whisper on CPU is real. It is also slow. A long meeting on a laptop CPU is a lunch break. On a server CPU it is still a background job.

GPU inference (or a fast encoder implementation) is what makes “every interview this week” plausible. You do not need a datacentre card for a few hours of audio a day; you do need to look at the hardware requirements for local LLMs and add STT as its own line, because a transcription worker and a chat model will fight for VRAM if they share one card.

Rule: measure real-time factor on your audio, not on a clean TED talk. A real-time factor of 0.2 means an hour of audio takes about twelve minutes. That number is the capacity plan.

Privacy

Local STT keeps the waveform off a vendor’s disk. That is the point. It does not encrypt the air in the meeting room. Tell people they are recorded. Do not put a hidden transcriber on a phone in a pocket and call it innovation.

If a recording should not exist, do not transcribe it. The most private transcript is the one you never made.

Integration

Where the output goes decides whether anyone uses this.

Useful destinations:

• the team’s existing document store, with a link back to audio;
• a case file in a matter-management or CRM system, as an attachment;
• an internal search index;
• a ticketing system, as a note, after a person clicks “send”.

Useless destinations:

• a GPU host’s local disk that only the engineer can SSH to;
• a Slack channel that is wider than the meeting was;
• an email to “the company”.

APIs should accept a file or a URL on the internal network, return a job id, and later a transcript URI. That is enough to hang a UI on. You do not need a platform.

Need a private transcription workflow for your company? That is a standard pilot in the private AI deployment work: a real corpus of recordings, local speech-to-text, a summary step, and a place the team can actually find the result next week.