Staging the speech stack under a 12GB envelope

The question

The speech stack exceeded the target memory envelope before staging. This hardening experiment verified protected layers and quantized projections, then measured staged memory on the hardware actually available.

From the original notebook

Date: 2026-09-18. Repo: groxaxo/moss-tts-hq4-quant, branch fix/release-reproducibility-hardening-v2 at 7982ede (7982edee4f10b293db8a7e4d7293661a69679692), base main at bfd27b1. PR #1 (draft): “Harden HQ4 release provenance, validation, and 12GB staging claims”. Workstation checkout: WORKSTATION/moss-tts-hq4-quant.

Question answered: do the PR’s “local GPU gates before merge” pass on the Ubuntu workstation — py_compile, bash -n, verify_quantization_map.py, and the staged 12 GB inference path?

Measured result: all listed gates pass. Verifier: 252/252 INT4 projections, 211 protected BF16 params, bits=4 / g=128 / sym=true. Staged text-only run under a 0.5 memory fraction on one RTX 3090 peaks at 6.63 GiB (vs 13.01 GiB pre-staging), with the generator fully parked on CPU (0.01 GiB) during codec decode. No physical RTX 3060 exists on this host, so the true 3060 acceptance run remains open by design.

Workflow

flowchart LR
    A[Locate repo +<br/>fetch PR branch] --> B[Checkout 7982ede<br/>review 12-file diff]
    B --> C[Gate 1: py_compile<br/>4 tools]
    C --> D[Gate 2: bash -n<br/>4 scripts]
    D --> E[Gate 3: verifier<br/>GPU 2, real artifact]
    E --> F[Gate 4: staged run<br/>memfrac 0.5, GPU 1]
    F --> G[Phase-VRAM probe<br/>run_single + rec]
    G --> H[Quantizer probes<br/>refusal + recipe]
    H --> I[This record<br/>+ PR update]

Step outcomes (observed, in order):

What was tested

Environment: 3× RTX 3090 24 GB (driver 595.58.03), Python 3.12.3, torch 2.9.1+cu128, transformers 5.0.0, auto-round 0.15.1 (see versions). Runtime venv, weights, and artifact live under WORKSTATION/MOSS-TTS — moss-tts-hq4-quant ships code only, so the GPU gates ran the PR-branch scripts with the MOSS-TTS venv and absolute model paths.

Verifier detail: 36 layers → 252/252 quantized projections, 252/252 qweight tensors, 211 protected BF16 parameters, quant_method=auto-round.

Phase VRAM (GiB, allocated / peak), second run:

Overall peak 6.634 GiB — the pre-staging benchmark peaked at 13.01 GiB with generator+codec coexisting at decode (see benchmark_results_hq4.json in the artifact dir). The CPU-parking fix recovers ~6.4 GiB at decode.

Quantizer probes (CPU-only, throwaway dirs under /tmp, no GPU, no quant run): namespace remap, missing-calibration FileNotFoundError pointing at tools/build_moss_calibration.py, untracked-export refusal, stale-export fingerprint-mismatch refusal, and recipe contents (cal SHA256, W4A16/sym settings, dep versions) — all pass.

Runs and timing

Limits and reproduction boundary

• No RTX 3060 on this host. The 12 GB result is the documented --memory-fraction 0.5 surrogate on a 3090, not a physical 3060 run. The PR docs already state this correctly; this record does not upgrade the claim to “verified”.
• The tested artifact predates quant_recipe.json (expected — only new quant runs emit it). A full re-quant was not run (1 h+ GPU job); the recipe/fingerprint logic was validated via the refusal-path probes above.
• ops/hq4-quant/*.sh still cd WORKSTATION/MOSS-TTS — pre-existing, outside this PR’s scope, noted for a follow-up.
• The private Hugging Face model-card sync was reported outstanding in the handoff and was not attempted here.
• Rerun: scripts/rerun-gates.sh (RUN_GPU=1 for the GPU gates; REPO/VENV/MODEL/CODEC overridable). Machine-readable results: data/validation.json.

No merge or deployment performed; PR #1 left draft and open.

Keep exploring

Original experiment record. Workstation paths have been generalized. Detailed measurements below retain their original workload and validation boundaries.