Skip to content

AuraFlowPipeline: VAE dtype mismatch on pipeline reuse (latents cast skipped on 2nd call) #14183

Description

@IvenHsu01

Describe the bug

AuraFlowPipeline.__call__ raises a dtype-mismatch RuntimeError in the VAE
decode when the same pipeline instance is called a second time. A single
call succeeds; the error only appears on reuse (e.g. a warmup call followed by a
real call).

Root cause is in the decode block:

needs_upcasting = self.vae.dtype == torch.float16 and self.vae.config.force_upcast
if needs_upcasting:
    self.upcast_vae()                     # casts the ENTIRE vae to float32 in place
    latents = latents.to(next(iter(self.vae.post_quant_conv.parameters())).dtype)
image = self.vae.decode(latents / self.vae.config.scaling_factor, return_dict=False)[0]

upcast_vae() upcasts the whole VAE to float32 in place, so after the first
call self.vae.dtype == torch.float32 permanently. On the second call
needs_upcasting is therefore False, the latents.to(...) line is skipped,
and float16 latents are fed to the now-float32 VAE:

RuntimeError: Input type (c10::Half) and bias type (float) should be the same

This is the same class of bug fixed for pipeline_pixart_sigma.py in #8391
that fix cast the latents to the VAE dtype unconditionally at the decode call.
The equivalent line in pipeline_aura_flow.py still guards the cast behind
if needs_upcasting:.

Reproduction

import torch
from diffusers import AuraFlowPipeline

pipe = AuraFlowPipeline.from_pretrained("fal/AuraFlow", torch_dtype=torch.float16).to("cuda")
prompt = "A cat holding a sign that says hello world"
gen = lambda: pipe(prompt=prompt, height=512, width=512, num_inference_steps=50,
                   guidance_scale=3.5, generator=torch.Generator("cuda").manual_seed(42))

gen()   # 1st call: OK (needs_upcasting True, latents cast runs, vae left in fp32)
gen()   # 2nd call: RuntimeError (needs_upcasting now False, latents stay fp16)

Suggested fix

Apply the same pattern as #8391 — cast the latents to the VAE dtype inline at the
decode call so it always runs regardless of needs_upcasting:

if needs_upcasting:
    self.upcast_vae()
image = self.vae.decode(latents.to(self.vae.dtype) / self.vae.config.scaling_factor, return_dict=False)[0]

Verification

Applied the suggested fix to a checkout at commit 33becabe, with no other
patches, and ran warmup + 1 run (the 2nd call is what triggers the bug). Both
produce a coherent image and no dtype error:

Environment Warmup 2nd call Result
NVIDIA RTX 5090, CUDA (torch 2.13 dev) 18.51s 7.31s Pass
AMD Radeon (gfx1151), ROCm (torch 2.11) 64.23s 63.25s Pass

The offending line is unchanged on current main.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions