Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"does not have a parameter or a buffer named input_scale" in quanto FluxTransformer2DModel #359

Open
LianShuaiLong opened this issue Dec 6, 2024 · 0 comments

Comments

@LianShuaiLong
Copy link

LianShuaiLong commented Dec 6, 2024

@sayakpaul
here is my quanto code:

from optimum.quanto.models import QuantizedDiffusersModel
class QuantizedFluxTransformer2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
model = FluxTransformer2DModel.from_pretrained("./FLUX.1-schnell",subfolder="transformer")
qmodel = QuantizedFluxTransformer2DModel.quantize(model,weights=qfloat8)
qmodel.save_pretrained("./FLUX.1-schnell-fp8")

no errors in quanto process
but valueError raise when infer with the quanto model:
ValueError: Linear(in_features=4096, out_features=3072, bias=True) does not have a parameter or a buffer named input_scale
here is my inference code:

pipeline = FluxPipeline.from_pretrained(
"./FLUX.1-schnell",
transformer=None,
torch_dtype=torch.float16
).to("cuda")
transformer = FluxTransformer2DModel.from_pretrained("./FLUX.1-schnell-fp8")
transformer.to("cuda",dtype=torch.float16)
pipeline.transformer = transformer
prompt = "a cat"
image = pipeline(
prompt=prompt,
num_inference_steps=4,
num_images_per_prompt=1,
width=512,
height=512,
generator = torch.Generator.manual_seed(42)
).images[0]
image.save('cat.png')

@LianShuaiLong LianShuaiLong changed the title Some bugs in quanto FluxTransformer2DModel "does not have a parameter or a buffer named input_scale" in quanto FluxTransformer2DModel Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant