squawk

Push-to-talk voice assistant for a local llama.cpp model. Hold a key, speak, release — your speech goes to the model (as raw audio to a multimodal LLM, or transcribed locally first — your pick) and the reply streams back into a terminal UI, spoken aloud in a cloned voice with instant barge-in.

Everything runs on your machine: the LLM (llama.cpp), the voice (CrispASR qwen3-tts clone on the GPU, or Kokoro on CPU), and the optional local transcription. No cloud, no Python at runtime.

🎙  PUSH-TO-TALK — hold [SPACE] to talk, release to send
● REC  2.3s
🤖 The capital of France is Paris.
[SPACE] talk    [m] swap mode    [d] device    [q] quit

Quick start (text-only)

Requirements:

• A llama.cpp router (or server) at http://localhost:8080 serving an audio-capable model (defaults target unsloth/gemma-4-12b-it-GGUF:Q4_K_M with its mmproj loaded) — or any text model if you use the local-STT input mode.
• A terminal that speaks the Kitty keyboard protocol for true hold-to-talk (ghostty, kitty, foot…). Otherwise squawk auto-falls back to toggle mode.
• System libs: ALSA (Linux) / the platform audio backend cpal targets.

cargo run --bin squawk                    # launch the TUI (text-only)
cargo run --bin squawk -- --list-devices  # pick a mic
cargo run --bin squawk -- --init-config   # write a commented config to edit

The default build is pure Rust; the neural voices are opt-in cargo features (see below). Talk with SPACE, quit with q.

Going further — the docs

doc	what's in it
docs/crispasr.md	The main event: download & compile CrispASR, get the qwen3-tts models, clone your own voice, wire up local STT — and the gotchas (LD_LIBRARY_PATH, ABI drift)
docs/tts.md	Spoken replies in depth: the three engines (qwen3 / kokoro / http), streaming & seam-minimal dispatch, barge-in, karaoke, moods, filler
docs/configuration.md	The layered config system, every section at a glance, input modes, assistant personas
ARCHITECTURE.md	The system map and the non-obvious contracts (heard-boundary interruption, epoch barge-in, streaming karaoke, the FFI)

The typical full setup is: build CrispASR once, download the models, make a voice clone, then

cargo run --features qwen3,stt     # cloned voice out + local transcription in

cargo build --features qwen3,stt still compiles zero C++/CUDA — CrispASR's libcrispasr.so is loaded at runtime.

Features & input modes

• TTS engines (cargo feature = config tts.engine): qwen3 — the production voice, an in-process GPU voice clone streaming PCM as it decodes; kokoro — self-contained CPU neural voice (ONNX); http — offload to any OpenAI /v1/audio/speech server. An engine whose feature isn't built disables speech with a warning.
• Input modes (input.mode): multimodal sends your audio to the model (it hears tone and emotion); stt transcribes locally via CrispASR and sends text (works with any text LLM, cheapest context); hybrid-multimodal does both.
• Barge-in with honest context: interrupting a spoken reply records how much you actually heard — the next turn tells the model what did and didn't reach your ears, so it repairs instead of assuming.
• Personas: [assistants.<name>] profiles compose the whole system prompt from declarative fields and bind a voice per engine.

Controls

key	action
SPACE	hold to talk, release to send (or press/press in toggle mode); pressed over a streaming/spoken reply, it interrupts and starts recording
p	replay the last recording — with processing applied (validate by ear)
t	toggle discussion on/off (draft mode — see below)
r	cycle the model's thinking effort (off → low → … → max, next reply)
h	show/hide the model's reasoning stream
1–5	toggle an audio-processing stage live (see below)
- / +	speak the reply slower / faster (engines with rate control)
m	swap capture mode (hold ↔ toggle) — needs key-release support
↑/↓ or j/k, PgUp/PgDn	scroll / select a turn (auto-follows the latest reply)
f	fork: truncate the conversation after the selected turn and continue from there
x	clear the conversation
d	input-device picker (j/k move, Enter select, d/Esc cancel)
q / Ctrl-C	quit

Assistant replies render markdown (code fences, headings, bold, inline code). While recording, the header shows a live speaking-rate meter (syllables/sec) — very fast speech is harder for the audio encoder.

Audio processing (1–5)

Each recording runs through a pipeline of Stages before being sent (and before replay, so p plays exactly what the model receives). The header's fx line shows what's on (green) or off (dim):

key	stage	what
1	dc	DC-offset removal
2	hp	80 Hz high-pass (rumble / handling noise)
3	denoise	RNNoise denoiser (nnnoiseless); off by default — heavier
4	trim	drop leading/trailing silence
5	norm	normalize toward −20 dBFS RMS, gain-capped, peak-limited to −3 dBFS

Adding a treatment is a new file in processing/ implementing Stage plus one line in Pipeline::default_voice.

Draft mode (t)

Discussion off = draft mode: recordings are held instead of sent. Record, replay with p, re-record until you like it; turning discussion back on sends the held draft.

Workspace layout

A two-crate Cargo workspace, split so the core logic can be reused by future frontends (GUI, web):

squawk/
├── crates/
│   ├── squawk-core/   # UI-agnostic: config, conversation, audio, TTS/STT, streaming client
│   └── squawk-tui/    # terminal frontend (clap + crossterm + ratatui)
└── docs/              # setup & usage guides (see above)

The module-by-module map lives in ARCHITECTURE.md — highlights: assistant (persona engine), model_client (cancellable SSE streaming), responder (reply interpreter: moods, speech chunking, heard mapping), speech_engine/speech (off-thread synthesis, epoch barge-in, karaoke position), tts/ + stt/ (the pluggable backends), processing (the DSP pipeline).

Notes / roadmap

• Resampling applies a windowed-sinc low-pass before decimation, then linear interpolation — good enough for the encoder.
• Groundwork is in place (dormant) for a multi-call agent loop: tool calls, think-budget gating, and filler-masked continuations — see the design notes at the end of ARCHITECTURE.md.
• Not yet: voice-activity detection, wake-word activation, adaptive speaking-rate time-stretch (the live rate meter is the first piece).