Voice

Voice mode is a local loop around the normal chat controller. It records audio, transcribes locally, sends the transcript to chat, and speaks the streamed assistant reply.

Controller

VoiceController owns the voice session. It uses:

• record for WAV recording and live amplitude.
• whisper_flutter_coreml for local transcription.
• flutter_tts for speech output.
• ChatController for actual assistant turns.

The voice page observes VoiceState for mode, transcript, amplitude, and status messages.

Entering voice mode

enter performs the setup:

1. Marks the session active.
2. Clears queued speech state.
3. Creates a Whisper instance with the base model.
4. Checks that Whisper can load.
5. Checks microphone permission.
6. Configures TTS completion and speech rate.
7. Starts recording.

If Whisper fails or the microphone permission is missing, the state moves to unavailable with a user-facing message.

Recording

The recorder writes a temporary WAV file at 16 kHz, mono. Amplitude updates drive the voice orb and silence detection.

When audio level stays below the threshold for roughly 2.5 seconds, recording finalizes if the model is not busy.

Transcription

Finalization stops recording and runs Whisper transcription against the temporary WAV file. The file is deleted after transcription. Empty transcripts return to listening mode with a short message.

Non-empty transcripts are sent to ChatController.send.

Speaking responses

The voice controller listens to ChatState. When the latest assistant message starts streaming, voice mode changes to speaking.

The controller does not wait for the whole response. It reads sentence-sized segments as they become available:

• Track how many characters have been spoken.
• Find the last sentence boundary in new text.
• Queue complete segments for TTS.
• Speak queued segments in order.
• Return to listening when the model finishes and the speech queue is empty.

This makes voice responses feel continuous while preserving the normal chat pipeline.

Interruptions

interrupt has two behaviors:

• While recording, it finalizes the current recording.
• While speaking, it stops TTS, clears the queue, resets spoken position, and starts listening again.

Cleanup

exit stops recording, cancels streams, stops TTS, clears queued speech, deletes the current temporary audio file when present, and resets state.

ref.onDispose performs the same defensive cleanup if the provider is destroyed.

Native voice entry

The app can open voice mode from the Android assist gesture. Wake-word settings are managed in Settings and delegated to native device actions. The wake-word service is controlled through DeviceActions.setWakeWordEnabled.