HN Explorer

Ask HN: How to boost Gemini transcription accuracy for company names?

42 points by bingwu1995 ↗ HN
I’m using Gemini for speech-to-text and it often misrecognizes company names and acronyms.

Is there any way to use a custom lexicon or vocabulary with Gemini to improve recognition accuracy? If not directly supported, what are practical workarounds people use — e.g. preprocessing prompts, fine-tuning, or combining Gemini with another ASR that supports phrase boosting?

21 comments

[ 0.90 ms ] story [ 41.6 ms ] thread
gallexme
Adding it to the instructions worked well for me with specific terms
semessier
adding to the question, ruling out fine-tuning for practicality, what about injecting names towards the embedding but not into the context?
lysecret
I generally found 4o-transcribe to be more performant than gemini fyi.
Reubend
Any company names or special acronyms should be added to your prompt.
tifa2up
Don't solve it on the STT level. Get the raw transcription from Gemini then pass the output to an LLM to fix company names and other modifications.

Happy to share more details if helpful.

simonw
Have you tried feeding it a system prompt with a list of custom vocabulary? I would expect that to work really well.

"Transcribe this audio. Be careful to spell the following names and acronyms right: list-goes-here"

gearhart
We use openwhisper for transcription which accepts a list of "words to look out for" which we populate with a short list of the names of all the people and companies most likely to be mentioned in the text, and then we do a spell checking pass at the end using Gemini with a much longer list, telling it to look out for anything that might be a misspelling.

It's not perfect, but it's taken it from being an issue that made all our transcripts look terrible, to an issue I no longer think about.

I imagine just using the second spellchecking pass with Gemini would be almost as effective.

vayup
Something along these lines, as part of the prompt, has worked for me.

               # User-Defined Dictionary
                Always use the following exact terms if they sound similar in the audio:

                ```json
                {{jsonDictionary}}
                ```
another_twist
Use any proper ASR service that supports custom vocabulary ? Transcribe and Deepgram definitely support it and if you want to go fancy Nemo with custom vocabulary.

Are there constraints where you have to use Gemini ?

rancar2
The business edition of Wispr Flow does this well, and includes sharing among teams so you can make sure that the company wide vocabulary is consistent and well recognized.

https://wisprflow.ai/business

alex-skobe
We have used markdown and list of vocabulary at the end like

Return company name only from dictionary

#dictionary 1:Apple 2:..

And than Vercel AI sdk + Zod Schema + Gemini 2.5 pro and it pretty accurate

meerab
I use a two-pass approach - first pass with ASR (OpenAI Whisper) and second pass with an LLM. I ask users to provide context upfront and use that as the "initial_prompt" parameter in Whisper: https://github.com/openai/whisper/discussions/963#discussion...

Gemini might have similar capabilities for custom vocabulary, though I'm not certain about their specific implementation. The two-pass ASR+LLM approach could work with Gemini's output as well.

b112
Give it a database backend with lots and lots of facts. Things verified by humans. There, AI 'fixed'.
[deleted]
(comment deleted)
mediaman
We do this simply by injecting a company-defined list of proper names/terms into the prompt, within <special_terms>, and telling it to use that information to assist with spelling. It works pretty well.
gawi
If you are able to isolate the text portion corresponding to the company name, you can compute the similarity (based on the character edit distance - Levenshtein) against every item of a predefined list of companies (and their aliases) and pick the best match.