Hugging Face added native support for Nunchaku 4-bit diffusion inference to its Diffusers library on July 23, 2026, enabling large diffusion transformers to run with lower memory requirements and faster speeds without requiring local CUDA compilation.
Text-to-image diffusion models operating in BF16 precision typically require 20 to 30 gigabytes of VRAM. Existing backends in Diffusers, including bitsandbytes, GGUF, torchao, and Quanto, primarily use weight-only quantization. These backends reduce memory by dequantizing low-precision weights back to high precision during compute, but they generally fail to improve inference speed and can introduce minor latency overhead.
SVDQuant, the method behind Nunchaku, processes main transformer layers using 4-bit weights and 4-bit activations (W4A4). Standard 4-bit quantization struggles with diffusion transformers due to outlier values in both weights and activations. SVDQuant isolates activation outliers into weights, represents difficult weight matrix portions using a 16-bit low-rank branch, and quantizes the remaining residual to 4 bits. Nunchaku Lite patches standard linear modules in Diffusers with svdq_w4a4 layers for attention and MLP projections, along with awq_w4a16 layers for adaptive normalization and modulation projections.
Hardware support and performance
NVIDIA Blackwell GPUs, including the RTX 50 series, RTX PRO 6000, and B200, run the NVFP4 kernel variants. Older architecture generations—including Turing, Ampere, and Ada GPUs such as the RTX 30 and 40 series, A100, and L40S—use INT4 variants instead. Hugging Face stated that Volta and Hopper GPUs are currently not supported by the 4-bit kernels, and the quantizer raises an error at load time on unsupported hardware.
Benchmarks recorded on an NVIDIA RTX PRO 6000 using ERNIE-Image-Turbo showed that Nunchaku Lite NVFP4 reduced peak VRAM usage from 31.1 GB in the BF16 baseline to 20.6 GB. End-to-end latency dropped from 3.00 seconds to 2.27 seconds. Applying torch.compile reduced pipeline latency further to 1.68 seconds, achieving a 1.8x overall speedup. Combining NVFP4 with a bitsandbytes NF4 text encoder lowered peak VRAM to 16.0 GB. On an RTX 5090 GPU, a pre-quantized ERNIE-Image-Turbo pipeline generated a 1024x1024 image in roughly 1.7 seconds with peak memory usage of 12 GB.
Model compression toolkit
Hugging Face also released the diffuse-compressor toolkit, allowing users to calibrate, quantize, package, and publish custom Diffusers models. For architectures requiring fused operations, such as FLUX.1-dev, Nunchaku Lite uses target configurations and runtime adapters to rewrite module layouts. For example, FLUX.1-dev fuses separate query, key, and value linear projections into a single to_qkv module to execute projection, normalization, and rotary embeddings inside a single fused operator.
