Benchmark + samples for the 4-bit AWQ build
(lemuriandezapada/MOSS-TTS-v1.5-awq-int4, 6.3 GB) against the BF16 and HQ4
baselines. Same 33 prompts, seeds (1234) and sampling throughout; Marlin/bf16
kernels, SDPA, batch 1, one RTX 3090.
Code: groxaxo/moss-tts-hq4-quant/awq-3090
(full harness, env pins, reproduction commands).
Headline
- 33/33 pass, 0 silent — pooled RTF 0.43 (BF16 0.59, HQ4 0.88), 35.8 tok/s (BF16 25.5, HQ4 17.0), peak VRAM 13.05 GiB combined.
- Spanish: all 8 prompts healthy, no truncation/silence (see samples).
- Register control (instruction-guided female, F0 ≥195 Hz): 3/15 (HQ4 1/21, BF16 1/4) — usable control restored, both languages pass.
- Caveats: Marlin bf16 is run-to-run nondeterministic on sm86 (treat rates as single-draw estimates); a gptqmodel unload-retention quirk keeps decode peak at ~13 GiB (12GB deployment blocked until resolved).
Contents
docs/—comparison.json(BF16/HQ4/AWQ),benchmark_awq.json(per-prompt records), register reports,single_run.json,FINDINGS.mdsamples/— smoke (Spanish), 3 Spanish suite prompts, 3 register passes, plus the 33-promptprompts.txt
| Sample | What |
|---|---|
awq3060-single.wav |
Smoke test, Spanish |
es-conv-1.wav / es-long-1.wav / longform-es-1.wav |
Spanish short → 58 s longform |
register-en-3.wav (203.4 Hz) / register-es-3.wav (198.3 Hz) |
Seeds-1235 female passes |
register-es-3-seeds501.wav (219.2 Hz) |
Seeds-501 female pass (same seed HQ4 barely passed at 196.7) |