Describe the bug
AuraFlowPipeline.__call__ raises a dtype-mismatch RuntimeError in the VAE
decode when the same pipeline instance is called a second time. A single
call succeeds; the error only appears on reuse (e.g. a warmup call followed by a
real call).
Root cause is in the decode block:
needs_upcasting = self.vae.dtype == torch.float16 and self.vae.config.force_upcast
if needs_upcasting:
self.upcast_vae() # casts the ENTIRE vae to float32 in place
latents = latents.to(next(iter(self.vae.post_quant_conv.parameters())).dtype)
image = self.vae.decode(latents / self.vae.config.scaling_factor, return_dict=False)[0]
upcast_vae() upcasts the whole VAE to float32 in place, so after the first
call self.vae.dtype == torch.float32 permanently. On the second call
needs_upcasting is therefore False, the latents.to(...) line is skipped,
and float16 latents are fed to the now-float32 VAE:
RuntimeError: Input type (c10::Half) and bias type (float) should be the same
This is the same class of bug fixed for pipeline_pixart_sigma.py in #8391 —
that fix cast the latents to the VAE dtype unconditionally at the decode call.
The equivalent line in pipeline_aura_flow.py still guards the cast behind
if needs_upcasting:.
Reproduction
import torch
from diffusers import AuraFlowPipeline
pipe = AuraFlowPipeline.from_pretrained("fal/AuraFlow", torch_dtype=torch.float16).to("cuda")
prompt = "A cat holding a sign that says hello world"
gen = lambda: pipe(prompt=prompt, height=512, width=512, num_inference_steps=50,
guidance_scale=3.5, generator=torch.Generator("cuda").manual_seed(42))
gen() # 1st call: OK (needs_upcasting True, latents cast runs, vae left in fp32)
gen() # 2nd call: RuntimeError (needs_upcasting now False, latents stay fp16)
Suggested fix
Apply the same pattern as #8391 — cast the latents to the VAE dtype inline at the
decode call so it always runs regardless of needs_upcasting:
if needs_upcasting:
self.upcast_vae()
image = self.vae.decode(latents.to(self.vae.dtype) / self.vae.config.scaling_factor, return_dict=False)[0]
Verification
Applied the suggested fix to a checkout at commit 33becabe, with no other
patches, and ran warmup + 1 run (the 2nd call is what triggers the bug). Both
produce a coherent image and no dtype error:
| Environment |
Warmup |
2nd call |
Result |
| NVIDIA RTX 5090, CUDA (torch 2.13 dev) |
18.51s |
7.31s |
Pass |
| AMD Radeon (gfx1151), ROCm (torch 2.11) |
64.23s |
63.25s |
Pass |
The offending line is unchanged on current main.
Describe the bug
AuraFlowPipeline.__call__raises a dtype-mismatchRuntimeErrorin the VAEdecode when the same pipeline instance is called a second time. A single
call succeeds; the error only appears on reuse (e.g. a warmup call followed by a
real call).
Root cause is in the decode block:
upcast_vae()upcasts the whole VAE to float32 in place, so after the firstcall
self.vae.dtype == torch.float32permanently. On the second callneeds_upcastingis thereforeFalse, thelatents.to(...)line is skipped,and float16 latents are fed to the now-float32 VAE:
This is the same class of bug fixed for
pipeline_pixart_sigma.pyin #8391 —that fix cast the latents to the VAE dtype unconditionally at the decode call.
The equivalent line in
pipeline_aura_flow.pystill guards the cast behindif needs_upcasting:.Reproduction
Suggested fix
Apply the same pattern as #8391 — cast the latents to the VAE dtype inline at the
decode call so it always runs regardless of
needs_upcasting:Verification
Applied the suggested fix to a checkout at commit
33becabe, with no otherpatches, and ran warmup + 1 run (the 2nd call is what triggers the bug). Both
produce a coherent image and no dtype error:
The offending line is unchanged on current
main.