ModelsLab/MiniMax-H3-svdquant-int4_r32

MiniMax-H3 · SVDQuant W4A4 (int4, rank 32, GPTQ)

4-bit weights and activations for the MiniMax-H3 31B video+audio transformer — true SVDQuant (ICLR 2025 Spotlight): activation outliers absorbed into a 16-bit rank-32 low-rank branch, the residual GPTQ-rounded to int4, activations quantized to int4 per-token at runtime, executed on fused CUTLASS tensor-core kernels. This is not weight-only quantization.

The text conditioner (Qwen3-VL 31B) ships W4A16+GPTQ in the same release: every stored weight is int4; only norms, embeddings, and the visual tower stay bf16, matching MiniMax's own quantization recipe.

Measured (NVIDIA A100 80GB, 124 frames @ 24fps, 960x544)

	BF16	this release	factor
DiT checkpoint	61.7 GB	19.6 GB	3.15x
generation wall-clock	484 s	369 s	1.31x faster
vs unfused reference dequant	2427 s	369 s	6.6x
quantized GEMM (layer level)	—	—	1.37-1.38x

All-resident configuration (this release's DiT + TE together, no CPU offload — unreachable for BF16 on one 80GB card):

	BF16 (offloaded)	all-int4 resident	factor
generation wall-clock	484 s	318 s	1.52x faster
pipeline VRAM	65 GB peak, offload churn	48.9 GB steady, 54.3 peak	fits
DiT + TE weights on disk	123.8 GB	37.6 GB	3.3x

Conversion cost: 44 min for the DiT (18 calib + 26 GPTQ) on one A100. Kernel outputs agree with the fp32 reference oracle to 1.5-2.1% (the bf16-vs-fp32 activation-rounding delta) at every layer shape.

Quality: same-seed renders are visually indistinguishable from BF16 (samples in this repo). On the Z-Image anchor, the same pipeline's GPTQ checkpoint scores better LPIPS than the officially published nunchaku checkpoint (0.288 vs 0.334).

Before / after (same seed, sound on)

BF16 original	this release (all-int4)

Prompts: a red fox in a snowy pine forest; a saxophonist under neon in light rain. Video and soundtrack are generated jointly by the model.

Use

pip install git+https://github.com/ModelsLab/svdquant git+https://github.com/rootonchair/nunchaku-lite

import svdquant
transformer = svdquant.load_model("minimax-h3-packed.safetensors")  # this repo's file
# drop into the diffusers ModularPipeline in place of the BF16 transformer

Requires an int4-tensor-core GPU (sm_75-89: RTX 20/30/40, A100) and torch >= 2.11. An NVFP4 sibling for RTX 50-series (sm_120 has no int4 path) is planned from a fresh BF16 pass — int4 and fp4 grids do not nest, so transcoding is never used.

Honest notes

1. Attention stays bf16 — at video sequence lengths it bounds the end-to-end speedup (Amdahl); the 1.31x reflects that.
2. The token_refiner (2 blocks, ~2% of params) stays bf16, following MiniMax's own int8 recipe.
3. The AWQ repack of the 50 modulation layers re-derives scales; groups GPTQ pushed to -8 take one extra bounded rounding.

Independently verified (cold download)

Reproduced on a separate, fresh A100 80GB by downloading only this repo's files plus the base model — the exact user path:

	BF16 (offloaded)	this release (resident)
generation	496.7 s	350.9 s (1.42x)
peak VRAM	65.2 GB	54.6 GB (49.2 steady)

Raw log: verified.json in this repo. First-build A100 measured 1.52x; host-to-host variance puts the honest claim at 1.4-1.5x.

Credits and license

Weights derive from MiniMaxAI/MiniMax-H3 and inherit its license. Method: SVDQuant (Li et al., MIT HAN Lab). Kernels and packed layout: nunchaku, nunchaku-lite and diffuse-compressor by rootonchair (Apache-2.0, vendored with attribution). Quantized with svdquant by ModelsLab.