Multilingual STT for hermes-gateway (Russian + English)
Symptom
Russian voice notes sent to the agent on Telegram decode poorly: Whisper returns English hallucinations, transliteration, or short junk tokens. English voice notes work fine.
Root cause
/opt/data/config.yaml currently has:
stt:enabled:truelanguage:enlocal:model:base
stt.language: en is the global language hint. Per tools/transcription_tools.py:1557-1559, this is forwarded into WhisperModel.transcribe(language=...) as a forced decoder hint — telling the model "decode this audio as English". With Russian speech and language=en, faster-whisper (base, multilingual) hallucinates English phrases or returns transliterated Russian. Other providers aren't affected because we only ship local here.
The repo (admin/ansible/hermes_squid/) does not override stt.language — so the runtime value comes straight from the mounted hermes_gateway_data:/opt/data/config.yaml.
Goal
Have Russian AND English voice notes both transcribe well, without the user manually toggling language per message.
Chosen approach
Auto-detect by removing the global stt.language hint, keeping the HERMES_LOCAL_STT_LANGUAGE env var available as an escape hatch for when auto-detect is wrong on a particular clip.
Why this and not "force Russian" / "force English":
Both languages needed (user explicit).
base model auto-detects reasonably well on clips ≥ 2s. On shorter clips it sometimes defaults to English; the env var lets us override per-deployment without touching config.yaml.
What gets changed (single PR in admin/ansible/hermes_squid/)
Edit /opt/data/config.yaml (the one mounted into the container) — set stt.language: to empty / remove it. Done by brencher deploy, not by editing the live file in place.
No compose change needed — HERMES_LOCAL_STT_LANGUAGE is already read from env (transcription_tools.py:190) and remains unset.
Verify after redeploy: send a Russian voice note → transcribed text should be Russian. Send an English voice note → English. No regression on the existing EN case.
What this PR does NOT do
Does not change the STT model (base stays). If quality on Russian is insufficient, that's a separate ticket (likely upgrading to small or medium).
English voice note → English transcript (no regression).
Mixed / very short clips fall back to auto-detect best-effort; env var HERMES_LOCAL_STT_LANGUAGE=ru|en overrides correctly when set on the container.
Single-purpose PR. brencher deployment verification step in PR body.
Verification plan (manual, after merge)
Send Russian voice note via Telegram to the bot.
Confirm transcription is Russian text.
Send English voice note.
Confirm English transcription.
Optional: docker exec hermes-gateway printenv | grep STT to confirm no HERMES_LOCAL_STT_LANGUAGE is set in the default deployment.
## Multilingual STT for hermes-gateway (Russian + English)
### Symptom
Russian voice notes sent to the agent on Telegram decode poorly: Whisper returns English hallucinations, transliteration, or short junk tokens. English voice notes work fine.
### Root cause
`/opt/data/config.yaml` currently has:
```yaml
stt:
enabled: true
language: en
local:
model: base
```
`stt.language: en` is the **global** language hint. Per `tools/transcription_tools.py:1557-1559`, this is forwarded into `WhisperModel.transcribe(language=...)` as a forced decoder hint — telling the model "decode this audio as English". With Russian speech and `language=en`, faster-whisper (`base`, multilingual) hallucinates English phrases or returns transliterated Russian. Other providers aren't affected because we only ship `local` here.
The repo (`admin/ansible/hermes_squid/`) does not override `stt.language` — so the runtime value comes straight from the mounted `hermes_gateway_data:/opt/data/config.yaml`.
### Goal
Have Russian AND English voice notes both transcribe well, without the user manually toggling language per message.
### Chosen approach
**Auto-detect** by removing the global `stt.language` hint, keeping the `HERMES_LOCAL_STT_LANGUAGE` env var available as an escape hatch for when auto-detect is wrong on a particular clip.
Why this and not "force Russian" / "force English":
- Both languages needed (user explicit).
- `base` model auto-detects reasonably well on clips ≥ 2s. On shorter clips it sometimes defaults to English; the env var lets us override per-deployment without touching `config.yaml`.
### What gets changed (single PR in `admin/ansible/hermes_squid/`)
1. **Edit `/opt/data/config.yaml`** (the one mounted into the container) — set `stt.language:` to empty / remove it. Done by brencher deploy, not by editing the live file in place.
2. **No compose change needed** — `HERMES_LOCAL_STT_LANGUAGE` is already read from env (transcription_tools.py:190) and remains unset.
3. **Verify after redeploy**: send a Russian voice note → transcribed text should be Russian. Send an English voice note → English. No regression on the existing EN case.
### What this PR does NOT do
- Does not change the STT model (`base` stays). If quality on Russian is insufficient, that's a separate ticket (likely upgrading to `small` or `medium`).
- Does not touch squid / brencher / other services.
- Does not add any provider other than `local`.
### Acceptance criteria
- Russian voice note → Russian transcript (WER roughly comparable to English, ±10%).
- English voice note → English transcript (no regression).
- Mixed / very short clips fall back to auto-detect best-effort; env var `HERMES_LOCAL_STT_LANGUAGE=ru|en` overrides correctly when set on the container.
- Single-purpose PR. brencher deployment verification step in PR body.
### Verification plan (manual, after merge)
1. Send Russian voice note via Telegram to the bot.
2. Confirm transcription is Russian text.
3. Send English voice note.
4. Confirm English transcription.
5. Optional: `docker exec hermes-gateway printenv | grep STT` to confirm no `HERMES_LOCAL_STT_LANGUAGE` is set in the default deployment.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Multilingual STT for hermes-gateway (Russian + English)
Symptom
Russian voice notes sent to the agent on Telegram decode poorly: Whisper returns English hallucinations, transliteration, or short junk tokens. English voice notes work fine.
Root cause
/opt/data/config.yamlcurrently has:stt.language: enis the global language hint. Pertools/transcription_tools.py:1557-1559, this is forwarded intoWhisperModel.transcribe(language=...)as a forced decoder hint — telling the model "decode this audio as English". With Russian speech andlanguage=en, faster-whisper (base, multilingual) hallucinates English phrases or returns transliterated Russian. Other providers aren't affected because we only shiplocalhere.The repo (
admin/ansible/hermes_squid/) does not overridestt.language— so the runtime value comes straight from the mountedhermes_gateway_data:/opt/data/config.yaml.Goal
Have Russian AND English voice notes both transcribe well, without the user manually toggling language per message.
Chosen approach
Auto-detect by removing the global
stt.languagehint, keeping theHERMES_LOCAL_STT_LANGUAGEenv var available as an escape hatch for when auto-detect is wrong on a particular clip.Why this and not "force Russian" / "force English":
basemodel auto-detects reasonably well on clips ≥ 2s. On shorter clips it sometimes defaults to English; the env var lets us override per-deployment without touchingconfig.yaml.What gets changed (single PR in
admin/ansible/hermes_squid/)/opt/data/config.yaml(the one mounted into the container) — setstt.language:to empty / remove it. Done by brencher deploy, not by editing the live file in place.HERMES_LOCAL_STT_LANGUAGEis already read from env (transcription_tools.py:190) and remains unset.What this PR does NOT do
basestays). If quality on Russian is insufficient, that's a separate ticket (likely upgrading tosmallormedium).local.Acceptance criteria
HERMES_LOCAL_STT_LANGUAGE=ru|enoverrides correctly when set on the container.Verification plan (manual, after merge)
docker exec hermes-gateway printenv | grep STTto confirm noHERMES_LOCAL_STT_LANGUAGEis set in the default deployment.