sayakpaul/kv-caching-flow-models-benchmarks
394 MB
Sayak Paul PRO
sayakpaul
AI & ML interests
Diffusion models, representation learning
Recent Activity
updated a bucket about 2 hours ago
sayakpaul/kv-caching-flow-models-benchmarks updated a dataset about 4 hours ago
huggingface/diffusers-metadata updated a Space about 12 hours ago
sayakpaul/diffusers-dashboardOrganizations
Posts 24
Post
3487
Fast LoRA inference for Flux with Diffusers and PEFT 🚨
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
torch.compile2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
Provides a list of papers focusing on optimizing T2I diffusion models, targeting fewer timesteps, architecture optimization, and more.
-
Progressive Distillation for Fast Sampling of Diffusion Models
Paper • 2202.00512 • Published • 2 -
On Distillation of Guided Diffusion Models
Paper • 2210.03142 • Published -
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Paper • 2309.06380 • Published • 34 -
Consistency Models
Paper • 2303.01469 • Published • 9
Contains Flux Fill, Canny, and Dev checkpoints in NF4.
Provides a list of papers focusing on optimizing T2I diffusion models, targeting fewer timesteps, architecture optimization, and more.
-
Progressive Distillation for Fast Sampling of Diffusion Models
Paper • 2202.00512 • Published • 2 -
On Distillation of Guided Diffusion Models
Paper • 2210.03142 • Published -
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Paper • 2309.06380 • Published • 34 -
Consistency Models
Paper • 2303.01469 • Published • 9
spaces 18
Running
Agents
Diffusers Project Pulse
🎨
Launch an interactive web app for model interaction
Running
Batch image captioning on TPU
🔥
vLLM TPU, Qwen2.5-VL, and the Torchax fallback path
Sleeping
Stem Separator
🎵
Separate audio tracks into individual stems
Runtime error
Agents
Stem Separator Inference
🏆
Separate audio into individual stems (vocals, drums, etc.)
Running
NB Fashion Moodboard
💻
Nano Banana powered fashion moodboard
Running
Agents
6
Civitai To Hub
🐠
Upload checkpoints from CivitAI to Hugging Face Hub.
models 27
sayakpaul/FluxTransformer2DModel-sm120-cu130-r54
Updated
sayakpaul/different-lora-from-civitai
12B • Updated • 46 • 1
sayakpaul/flux-diffusers-gguf
12B • Updated • 63 • 1
sayakpaul/trained-lumina2-lora-yarn
Text-to-Image • Updated • 23 • 3
sayakpaul/FLUX.1-dev-edit-v0
Text-to-Image • 12B • Updated • 45 • 46
sayakpaul/cartoon-control-lr_1e-4-wd_1e-4-gs_10.0-cd_0.1
Text-to-Image • 12B • Updated • 76 • 6
sayakpaul/q8-ltx-video
2B • Updated • 41 • 7
sayakpaul/bnb-single-file-checkpoint-from-civitai
6B • Updated • 43
sayakpaul/mochi-lora-dissolve
Text-to-Video • Updated • 34 • 2
sayakpaul/flux.1-dev-int8-aot-compiled
Updated • 4
datasets 26
sayakpaul/sample-datasets
Viewer • Updated • 6 • 10.9k • 1
sayakpaul/ltx2-optims-artifacts
Viewer • Updated • 7 • 123
sayakpaul/torch-profiling-trace-diffusers
Viewer • Updated • 18 • 212
sayakpaul/diffusers-blackwell-quants
Updated • 525 • 1
sayakpaul/ShapeNet-55
Viewer • Updated • 52.5k • 1.23k
sayakpaul/llm-bias-mt-bench
Viewer • Updated • 4.76k • 66
sayakpaul/butteflies_with_classes
Preview • Updated • 1.98k
sayakpaul/OmniEdit-mini
Viewer • Updated • 21.1k • 218 • 6
sayakpaul/video-dataset-disney-organized
Viewer • Updated • 69 • 564 • 6
sayakpaul/pick-a-pic-v2-unique-prompts
Viewer • Updated • 59k • 193 • 2