Choosing a transcription app that's strong in Japanese — what Kotoba Whisper is

If you’re looking for a transcription tool that’s “accurate in Japanese,” the thing to understand is how to choose between a Japanese-specialized model and a general-purpose one. The Japanese-specialized Kotoba Whisper is light and fast and strong on read-aloud audio, while in natural meetings with overlapping speech and noise, general-purpose models with more parameters (medium / large-v3) can be more stable. OffReco is a Mac app that runs both of these fully on-device and lets you switch, without sending anything to the cloud. This article covers why Japanese transcription is hard, what Kotoba Whisper actually is, and how to improve your results.

Why Japanese transcription is hard

Compared with English, Japanese speech recognition has a few built-in difficulties.

• Many homophones: words that sound identical but differ in meaning and kanji are everywhere. Picking the right written form requires understanding the surrounding context.
• Proper nouns and jargon: company names, product names, and internal terms often aren’t in the dictionary, so they’re easy to mis-transcribe.
• Casual spoken language: fillers, restarts, and back-channel responses trip up models that assume clean written language.

General-purpose Whisper covers many languages with a single model, so it can miss these Japanese-specific patterns. A model retrained on Japanese data can do better here — though in meetings with lots of noise or overlapping speech, a general-purpose model with more parameters can be more stable. Which one wins depends on your recording environment, so being able to try both is reassuring.

What Kotoba Whisper is

Kotoba Whisper is a model based on OpenAI’s Whisper but specialized for Japanese (OffReco uses v2.0). Technically, it’s distilled from Whisper’s top model, large-v3, with a lighter decoder and retraining on Japanese data.

Two things stand out:

• Light and fast: per the model card, it’s about 6.3x faster than large-v3, while its Japanese error rate (CER) is reported to be on par with or better than large-v3 (for example, CER 9.2 on CommonVoice 8 Japanese). Because it’s lightweight, it runs at practical speeds on many Macs without a dedicated GPU (source: HuggingFace model card). Note, though, that that CER 9.2 and ~6.3x figure come from CommonVoice 8, a “read-aloud” benchmark — single speaker, low noise. In natural meetings with overlapping speech and noise, general-purpose models (medium / large-v3) can be more stable.
• Runs on faster-whisper (CPU): weights are available for the high-speed inference library faster-whisper, so it transcribes at usable speeds on a Mac. Note that Kotoba has no mlx build, so even on Apple Silicon it runs on the CPU rather than the GPU (if you want GPU speed, medium / large-v3 — which have mlx builds — fit better).

In short, Kotoba Whisper sits as “Japanese-specialized, light and fast, and accurate on read-aloud audio.” If meeting accuracy is your top priority, general-purpose models are also worth considering — so being able to try both is reassuring.

Tips to improve accuracy

Even a good model depends on how you use it. To raise transcription quality for Japanese meetings, these three help most.

1. Pick the model that fits your need: the best choice differs depending on whether you prioritize speed and lightness or meeting accuracy. OffReco lets you choose the model in the setup screen, so try Kotoba for light/Japanese-specialized use, or (on Apple Silicon) the mlx medium / large-v3 for meeting accuracy, and switch to compare (how to choose a model).
2. Record in a quiet environment: background noise and people talking over each other cause errors with any model. Just positioning the mic well and capturing clean audio changes the outcome.
3. Use speaker separation: splitting the transcript by who said what makes the record far easier to read and to check or edit later. OffReco supports speaker separation (enabled by setting an optional Hugging Face token).

Where OffReco fits

What sets OffReco apart is that it runs both Kotoba Whisper and the general-purpose medium / large-v3 fully on-device, with instant switching.

• All processing on your Mac: recording, transcription, and speaker separation happen on the machine, so neither the audio nor the transcript leaves it. Transcription works in airplane mode (only the first-run model download needs a connection).
• Fully automatic: it auto-detects meetings, and transcription starts on its own when you end the recording. You don’t have to ask for it each time.
• Low barrier to entry: the first month is free, then ¥220/month or ¥2,200/year. It runs on macOS 14.2 or later.

You can generate minutes (a summary) with your own ChatGPT or Claude — OffReco never calls an AI itself, and only the transcript is shared (never the audio). With Claude Desktop it can be a single tap; with the web apps, two.

Wrapping up

If you want better transcription accuracy for Japanese meetings, the shortcut is to match a Japanese-specialized model and a general-purpose one to the job. Kotoba Whisper is a Japanese-specialized model distilled from large-v3 — lightweight, runnable on many Macs, and practical via faster-whisper (CPU). For top meeting accuracy, the mlx medium / large-v3 can be more stable on Apple Silicon. OffReco lets you use either one fully on-device and fully automatically, with instant switching, which suits anyone who wants to automate Japanese meeting notes without uploading audio to the cloud. Download it and check the accuracy on your own meetings. Related reading: How to transcribe meetings on a Mac without sending audio to the cloud and running Whisper locally with no setup.