# MiniMax Voice — System Overview

## Purpose

Move the Jervis home voice assistant speech-to-text (STT) and text-to-speech
(TTS) to MiniMax cloud APIs. Keep a local fallback. The house must keep working
when the internet is down.

Wake-word detection runs locally in `voice_bridge`. Only the spoken command
reaches the cloud. Cost scales with command duration, not listening time.

## Architecture

```
ESP32-S3 (wake word, mic)
  -> MQTT voice/audio_stream -> voice_bridge (OpenWakeWord, local)
  -> POST :5001/transcribe -> voice_stt_shim
        |- primary  -> cloud STT (MiniMax asr-1.0; OpenRouter = rollback)
        \- fallback -> voice_whisper_fallback :5002 (faster-whisper base.en)
  -> transcript -> MQTT homeassistant/voice/text -> voice-agent :8501
        |- tool calls -> Home Assistant :8123
        |- Home Assistant announcements -> MQTT homeassistant/voice/announce
        |     (announce mode: one LLM call, no tools)
        \- reply -> speak_direct.sh -> tts_router
              |- MiniMax T2A v2 (voice id)
              \- piper_tts (fallback)
  -> Snapcast :4953 (48 kHz stereo) -> house speakers
```

## Services

| Service | Host | Port | Role |
| :--- | :--- | :--- | :--- |
| `voice_bridge` | `.13` | docker | Wake word and audio buffer. |
| `voice_stt_shim` | `.13` | `5001` | Cloud STT with a fallback chain. |
| `voice_whisper_fallback` | `.13` | `5002` | Local `base.en` fallback. |
| `voice-agent` (Jervis) | `.13` | `8501` | LLM tool-calling brain. System unit. |
| `tts_router` | `.13` | — | Routes TTS to MiniMax or piper. |
| `piper_tts` | `.13` | docker | Local TTS fallback. |
| Snapcast | `.13` | `4953` | Announcements stream. |
| Home Assistant | `.30` | `8123` | House control. |

## The two integration surfaces

Surface 1 (STT): `POST :5001/transcribe`, multipart field `audio`, WAV file.
The service returns `{"transcript": "..."}`. Only `voice_bridge` calls it.

Surface 2 (TTS): `speak_direct.sh "<text>" "<voice>" "<speed>"`. Two callers use
it: `voice-agent` and Home Assistant on `.30` through SSH. A three-argument call
does not change.

## Status

| Phase | Scope | State |
| :--- | :--- | :--- |
| 0 | Baseline, corpus, benchmark harness. | Done. |
| 1 | MiniMax STT shim on `5001`. | Live. |
| 2 | Local fallback chain and `WHISPER_URL` flip. | Live. |
| 3 | MiniMax TTS router behind `speak_direct.sh`. | Live with the funded key. |
| 4 | Voice personality and speech plan. | Live. |
| 5 | Hardening, spend guard, documentation. | In progress. |

The live STT primary is **MiniMax `asr-1.0`** on the funded subscription key
(AUD-31). OpenRouter is a one-line rollback. The local `base.en` fallback on
`5002` is unchanged.

## Home Assistant routing and old-piper status

Home Assistant reaches the voice stack in one of two ways:

| HA entity | Action today |
| :--- | :--- |
| `script.jervis_say` | `mqtt.publish` to `homeassistant/voice/announce`. |
| `automation.speak_calendar_event_reminder_30_min_before` | `script.jervis_say` in the config store. |
| `automation.voice_reminder_speak` | `script.jervis_say` (verified on the new path). |
| `script.piper_tts_announcement` | `script.jervis_say`. `voice` and `speed` are inert. |
| `script.piper_doorbell` | `shell_command.tts_doorbell_piper` (still old piper). |

The agent runs MQTT announcements in **announce mode**: one LLM call and **no
tools**. An announcement can never act.

AUD-33 found **no HA to piper hang**. All 23 SSH sessions from HA (`.30`) to
`.13` in the 6 days to 2026-10-08 played on Snapcast. The only HA
`shell_command` timeout was `post_webhook_call` (an n8n `curl`), not piper.

One caller is still **broken, not hanging**: `script.piper_doorbell` runs
`doorbell_tts_to_wav.sh`, which uses the dead native piper binary and the
missing `sox`. It exits `127` in under one second and never rewrites the
front-door WAV. The calendar and `piper_tts_announcement` store already says
`script.jervis_say`, but no run happened after the AUD-21 reload, so the loaded
action is unproven. Evidence: `notes/aud32-ha-piper-hang-investigation.md`.

## Test results (2026-10-08, AUD-34)

| Suite | Tests | Result |
| :--- | :--- | :--- |
| `tts_router/test_tts_router` | 74 | OK |
| `voice_stt_shim/test_contract` | 7 | OK |
| `voice_bridge` `test_transcribe_and_send` + `test_corpus_capture` | 13 | OK |
| `benchmark/test_bench_normalise` | 7 | OK |

The live smoke test passed: STT primary, local fallback, and the fallback
chain all returned HTTP 200. The TTS router succeeded, fell back to piper, and
failed loudly (`rc=2`) as designed. Raw logs: `benchmark/results/aud32/`.

## Measured results

- Cloud STT uses no local RAM. The local `base.en` fallback uses about 185 MiB.
- Net resident saving after the STT flip: about 239 MiB.
- Cloud STT accuracy on the real 22-command corpus: word error rate 0.0151.
- Local `base.en` fallback: word error rate 0.0114, about 514 ms per call.

## Related documents

- `AGENTS.md` — the project brief and full history.
- `README.md` — the technical repository README.
- `notes/aud32-ha-piper-hang-investigation.md` — the HA piper investigation.
- `notes/aud34-voice-stack-tests.md` — the test run.
- `notes/aud32-verification-summary.md` — the combined summary.
- `notes/` — per-phase engineering notes and raw measurements.
- `docs/minimax-voice-map.html` — the interactive architecture map.