You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
from optimum.quanto.models import QuantizedDiffusersModel
class QuantizedFluxTransformer2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
model = FluxTransformer2DModel.from_pretrained("./FLUX.1-schnell",subfolder="transformer")
qmodel = QuantizedFluxTransformer2DModel.quantize(model,weights=qfloat8)
qmodel.save_pretrained("./FLUX.1-schnell-fp8")
no errors in quanto process
but valueError raise when infer with the quanto model:
ValueError: Linear(in_features=4096, out_features=3072, bias=True) does not have a parameter or a buffer named input_scale
here is my inference code:
The text was updated successfully, but these errors were encountered:
LianShuaiLong
changed the title
Some bugs in quanto FluxTransformer2DModel
"does not have a parameter or a buffer named input_scale" in quanto FluxTransformer2DModel
Dec 6, 2024
@sayakpaul
here is my quanto code:
from optimum.quanto.models import QuantizedDiffusersModel
class QuantizedFluxTransformer2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
model = FluxTransformer2DModel.from_pretrained("./FLUX.1-schnell",subfolder="transformer")
qmodel = QuantizedFluxTransformer2DModel.quantize(model,weights=qfloat8)
qmodel.save_pretrained("./FLUX.1-schnell-fp8")
no errors in quanto process
but valueError raise when infer with the quanto model:
ValueError: Linear(in_features=4096, out_features=3072, bias=True) does not have a parameter or a buffer named input_scale
here is my inference code:
pipeline = FluxPipeline.from_pretrained(
"./FLUX.1-schnell",
transformer=None,
torch_dtype=torch.float16
).to("cuda")
transformer = FluxTransformer2DModel.from_pretrained("./FLUX.1-schnell-fp8")
transformer.to("cuda",dtype=torch.float16)
pipeline.transformer = transformer
prompt = "a cat"
image = pipeline(
prompt=prompt,
num_inference_steps=4,
num_images_per_prompt=1,
width=512,
height=512,
generator = torch.Generator.manual_seed(42)
).images[0]
image.save('cat.png')
The text was updated successfully, but these errors were encountered: