Yuang Ai*, Jiaming Han*, Shaobin Zhuang*, Weijia Mao, Xuefeng Hu, Ziyan Yang, Zhenheng Yang, Huaibo Huang†, Xiangyu Yue†, Hao Chen*†‡

^* Equal Contribution  ^† Corresponding Author  ^‡ Project Lead

For visual generation, discrete autoregressive models often struggle with poor tokenizer reconstruction, difficulties in sampling from large vocabularies, and slow token-by-token generation speeds. We present BitDance, which addresses these challenges via a large-vocabulary binary tokenizer, a binary diffusion head for sampling in large discrete space, and a next-patch diffusion paradigm that enables efficient multitoken prediction. BitDance is an open-source discrete autoregressive foundation model with 14B parameters, trained on large-scale multimodal tokens. While maintaining the standard language modeling paradigm for text tokens, BitDance employs a next-patch diffusion paradigm for visual tokens to predict multiple tokens in parallel—up to 64 per step. This unified multimodal framework is simple, efficient, scalable, and capable of efficiently generating high-resolution, photorealistic images.

• 2026.2.18: Special thanks to @Bili-Sakura for providing the diffusers version for all BitDance models!
• 2026.2.17: We release UniWeTok, An Unified Binary Tokenizer with Codebook Size $\mathit{2^{128}}$ for Unified Multimodal Large Language Model. Checkout the UniWeTok README for more details!
• 2026.2.17: We release the website, demo, model, and paper for BitDance.

BitDance is a purely autoregressive multimodal generative model. It adopts a decoder-only architecture with three key components: a large-vocabulary binary tokenizer, a binary diffusion head, and a next-patch diffusion paradigm for efficient multi-token prediction. This allows BitDance to predict up to 64 visual tokens in parallel, significantly improving speed. After large-scale training, it surpasses open-source AR models on text-to-image benchmarks and achieves a speedup of over 30x compared to standard next-token prediction AR models.

1️⃣ Binary Visual Tokenizers

We release three binary tokenizers with different downsampling ratios and vocabulary sizes. All model weights and configs can be found at BitDance-Tokenizer.

Vocabulary Size	Down Ratio	IN-256 PSNR	IN-256 SSIM	Weight	Config
$2^{32}$	16	24.90	0.72	ae_d16c32	ae_d16c32
$2^{128}$	32	23.26	0.67	ae_d32c128	ae_d32c128
$2^{256}$	32	25.29	0.74	ae_d32c256	ae_d32c256

2️⃣ T2I Models

We offer two models, BitDance-14B-64x and BitDance-14B-16x, which can predict 64 and 16 tokens in parallel at each step, respectively.

Model	#Token per Step	Step-1024px	Supported Size	Huggingface
BitDance-14B-64x	64	64	1024px	BitDance-14B-64x
BitDance-14B-16x	16	256	512&1024px	BitDance-14B-16x

3️⃣ ImageNet-trained Models

To reproduce the experiments on ImageNet, check here for details.

Model	Resolution	Params	Step-256px	FID	Huggingface
BitDance-B-1x	256x256	242M	256	1.68	BitDance_B_1x.pt
BitDance-B-4x	256x256	260M	64	1.69	BitDance_B_4x.pt
BitDance-B-16x	256x256	260M	16	1.91	BitDance_B_16x.pt
BitDance-L-1x	256x256	527M	256	1.31	BitDance_L_1x.pt
BitDance-H-1x	256x256	1.0B	256	1.24	BitDance_H_1x.pt

Thanks to @Bili-Sakura, all diffusers version models can be found at here.

1️⃣ Create Conda Environment and Install Package

git clone https://github.com/shallowdream204/BitDance.git
cd BitDance
conda create -n bitdance python=3.11 -y
conda activate bitdance
pip install -r requirements.txt
pip install flash_attn==2.8.2 --no-build-isolation

2️⃣ Download Model Weights

Run the following scripts to download all T2I models.

hf download shallowdream204/BitDance-14B-64x --local-dir models/BitDance-14B-64x --max-workers=16
hf download shallowdream204/BitDance-14B-16x --local-dir models/BitDance-14B-16x --max-workers=16

3️⃣ T2I Inference (check here for the supported image resolution)

# example_t2i.py
from modeling.t2i_pipeline import BitDanceT2IPipeline

model_path = 'models/BitDance-14B-64x'
# model_path = 'models/BitDance-14B-16x'
device = 'cuda'

pipe = BitDanceT2IPipeline(model_path=model_path, device=device)

prompt = "A close-up portrait in a cinematic photography style, capturing a girl-next-door look on a sunny daytime urban street. She wears a khaki sweater, with long, flowing hair gently draped over her shoulders. Her head is turned slightly, revealing soft facial features illuminated by realistic, delicate sunlight coming from the left. The sunlight subtly highlights individual strands of her hair. The image has a Canon film-like color tone, evoking a warm nostalgic atmosphere."

image = pipe.generate(
    prompt=prompt,
    height=1024,
    width=1024,
    num_sampling_steps=50, # adjust to 25 steps for faster inference, but may slightly reduce quality
    guidance_scale=7.5,
    num_images=1,
    seed=42
)[0]

image.save("example.png")

Check here for detailed instructions of diffusers version.

Example inference script for BitDance-14B-64x:

hf download BiliSakura/BitDance-14B-64x-diffusers --local-dir BitDance-14B-64x-diffusers --max-workers=16
cd BitDance-14B-64x-diffusers
python test_bitdance.py

🔥 Try the Huggingface Space demo to start playing with BitDance: BitDance-Demo

You can also run the demo locally:

python app.py

We provide the scripts for evaluation on DPG Bench and GenEval. More benchmark evaluation scripts are coming soon.

Evaluation of BitDance-14B-64x Model

bash scripts/eval/eval_bitdance_14b_64x.sh

Evaluation of BitDance-14B-16x Model

bash scripts/eval/eval_bitdance_14b_16x.sh

Note you still need to follow the instructions in DPG Bench and GenEval to evaluate the results.

We are organizing the code related to data loading. The training instruction of BitDance is coming soon.

BitDance-14B-64x is distilled from BitDance-14B-16x using a small amount of high-quality data. BitDance-14B-64x achieves approximately a 3x inference speedup while maintaining the same excellent generation quality. Here we present the side-by-side comparison between BitDance-14B-64x and BitDance-14B-16x.

Text Prompt	BitDance-14B-64x	BitDance-14B-16x
A close-up portrait in a cinematic photography style, capturing a girl-next-door look on a sunny daytime urban street. She wears a khaki sweater, with long, flowing hair gently draped over her shoulders. Her head is turned slightly, revealing soft facial features illuminated by realistic, delicate sunlight coming from the left. The sunlight subtly highlights individual strands of her hair. The image has a Canon film-like color tone, evoking a warm, nostalgic atmosphere.
A Roman-style Athena sculpture lounges elegantly on a deck chair aboard a luxury cruise ship, wearing lake-blue sunglasses and golden headphones, holding a bubble wand that blows purple bubbles. The sculpture gazes thoughtfully toward distant mountains, her expression contemplative. Crafted from finely polished marble, she is draped in a flowing, pleated robe, with skin rendered in smooth, delicate detail. The scene features only the sculpture, set against a backdrop of the deep blue sea visible through the deck railing. The entire image is bathed in warm natural sunlight, with no other figures to distract from the view.
A gritty, noir-style comic book panel. A detective in a trench coat stands in a dark alleyway, lighting a cigarette. The only light source is the flame of the lighter, illuminating his rugged face and the rain falling around him. The shadows are deep blacks (ink style). Speech bubble in the corner says 'It was a long night.' The lines are bold and expressive, cross-hatching shading, monochromatic with a splash of red for the lighter flame.
A medieval setting with a humanoid robot crafted from wood, iron, and leather, resembling a knight or a craftsman, standing amidst cobblestone streets and rustic buildings. The robot features intricate mechanical components, such as gears and exposed metal joints, blending seamlessly into the historical aesthetic. The scene is illuminated by soft, warm sunlight, casting gentle shadows and highlighting the textures of the materials. The atmosphere is serene and timeless, evoking a sense of curiosity and wonder, with a touch of fantasy in the mechanical design. Ultra-realistic details, cinematic lighting, and a harmonious balance between the medieval and futuristic elements.

BitDance is licensed under the Apache 2.0 license.

If you find our work useful for your research, please consider citing our paper:

@article{ai2026bitdance,
  title   = {BitDance: Scaling Autoregressive Generative Models with Binary Tokens},
  author  = {Ai, Yuang and Han, Jiaming and Zhuang, Shaobin and Hu, Xuefeng and Yang, Ziyan and Yang, Zhenheng and Huang, Huaibo and Yue, Xiangyu and Chen, Hao},
  journal = {arXiv preprint arXiv:2602.14041},
  year    = {2026}
}