Hugging Face Model Load Error: How to Fix 'Unauthorized' and 'Repository Not Found' Issues

[OMANNTYOSU]

ControlNet preprocessor location: /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2024-10-19 01:22:12,801 - ControlNet - INFO - ControlNet v1.1.455
01:22:32 - ReActor - STATUS - Running v0.7.1-a1 on Device: CUDA
Loading weights [31e35c80fc] from /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors
Creating model from config: /content/gdrive/MyDrive/sd/stablediffusion/generative-models/configs/inference/sd_xl_base.yaml
creating model quickly: OSError
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/None/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 403, in cached_file
resolved_file = hf_hub_download(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 862, in hf_hub_download
return _hf_hub_download_to_cache_dir(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 969, in _hf_hub_download_to_cache_dir
_raise_on_head_call_error(head_call_error, force_download, local_files_only)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1484, in _raise_on_head_call_error
raise head_call_error
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1376, in _get_metadata_or_catch_error
metadata = get_hf_file_metadata(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1296, in get_hf_file_metadata
r = _request_wrapper(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 277, in _request_wrapper
response = _request_wrapper(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 301, in _request_wrapper
hf_raise_for_status(response)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_http.py", line 454, in hf_raise_for_status
raise _format(RepositoryNotFoundError, message, response) from e
huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-67130a8f-4d63981f66483f4c09d87cd9;c9ceee25-45be-44b8-9a9d-dac62c5c723d)

Repository Not Found for url: https://huggingface.co/None/resolve/main/config.json.
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/initialize.py", line 149, in load_model
shared.sd_model # noqa: B018
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/shared_items.py", line 175, in sd_model
return modules.sd_models.model_data.get_sd_model()
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_models.py", line 693, in get_sd_model
load_model()
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_models.py", line 820, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_models.py", line 775, in instantiate_from_config
return constructor(**params)
File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/models/diffusion.py", line 61, in init
self.conditioner = instantiate_from_config(
File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/encoders/modules.py", line 88, in init
embedder = instantiate_from_config(embconfig)
File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/encoders/modules.py", line 361, in init
self.transformer = CLIPTextModel.from_pretrained(version)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_disable_initialization.py", line 68, in CLIPTextModel_from_pretrained
res = self.CLIPTextModel_from_pretrained(None, *model_args, config=pretrained_model_name_or_path, state_dict={}, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 3301, in from_pretrained
resolved_config_file = cached_file(
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 426, in cached_file
raise EnvironmentError(
OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=<your_token>

Failed to create model quickly; will retry using slow method.
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
How can this problem be solved?

[M-Masry4122]

I'm new to stable diffusion and I'm having the same problem.

[cosmicoxytocin]

Make sure you're using the latest notebook Found Here

---

Here's the cell I use to handle incompatible dependency errors:

#@markdown # Update Dependencies
import os
import sys
import subprocess
import time
from tqdm.auto import tqdm

def update_dependencies():
  print('\033[0;33mUpdating Dependencies..')
  for package in tqdm(['jax', 'jaxlib', 'scikit-image', 'huggingface_hub',
                       'diffusers', 'accelerate'], desc='Uninstalling', unit='package'):
    subprocess.run(f'pip uninstall -y -q {package}', shell=True)

  for package in tqdm(['jax==0.4.23', 'jaxlib==0.4.23', 'scikit-image', 'hyperactive', 'huggingface_hub==0.25.2',
                       'diffusers==0.29.2', 'accelerate==0.27.2'], desc='Installing', unit='package'):
    subprocess.run(f'pip install -q {package}', shell=True)

  clear_output(wait=True)
  inf('\u2714 Done', 'success', '50px')


update_dependencies()

IMPORTANT: Make sure you add this cell right above the final cell in the notebook (above "Start Stable-Diffusion").

Note: Not all packages may be required. Some of them are due to errors I've encountered with certain extensions.
This may or may not help with your particular traceback. But give it a try and let me know how it fixed the issue.

[Guroexe]

How to deal with the new error: Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/webui.py", line 13, in
initialize.imports()
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/initialize.py", line 16, in imports
import pytorch_lightning # noqa: F401
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/init.py", line 34, in
from pytorch_lightning.callbacks import Callback # noqa: E402
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/init.py", line 26, in
from pytorch_lightning.callbacks.pruning import ModelPruning
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/pruning.py", line 30, in
from pytorch_lightning.core.module import LightningModule
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/core/init.py", line 16, in
from pytorch_lightning.core.module import LightningModule
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/core/module.py", line 41, in
from pytorch_lightning.loggers import Logger, LoggerCollection
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loggers/init.py", line 23, in
from pytorch_lightning.loggers.tensorboard import TensorBoardLogger
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loggers/tensorboard.py", line 26, in
from torch.utils.tensorboard import SummaryWriter
File "/usr/local/lib/python3.10/dist-packages/torch/utils/tensorboard/init.py", line 12, in
from .writer import FileWriter, SummaryWriter # noqa: F401
File "/usr/local/lib/python3.10/dist-packages/torch/utils/tensorboard/writer.py", line 13, in
from tensorboard.compat.proto import event_pb2
File "/usr/local/lib/python3.10/dist-packages/tensorboard/compat/proto/event_pb2.py", line 17, in
from tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2
File "/usr/local/lib/python3.10/dist-packages/tensorboard/compat/proto/summary_pb2.py", line 17, in
from tensorboard.compat.proto import tensor_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__pb2
File "/usr/local/lib/python3.10/dist-packages/tensorboard/compat/proto/tensor_pb2.py", line 16, in
from tensorboard.compat.proto import resource_handle_pb2 as tensorboard_dot_compat_dot_proto_dot_resource__handle__pb2
File "/usr/local/lib/python3.10/dist-packages/tensorboard/compat/proto/resource_handle_pb2.py", line 16, in
from tensorboard.compat.proto import tensor_shape_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__shape__pb2
File "/usr/local/lib/python3.10/dist-packages/tensorboard/compat/proto/tensor_shape_pb2.py", line 36, in
_descriptor.FieldDescriptor(
File "/usr/local/lib/python3.10/dist-packages/google/protobuf/descriptor.py", line 553, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

[rickvalstar]

***** EDIT ***** I did a little more looking at my initial workaround and revised it as I continue to seek the root cause

Everyone (including me) has been looking at this problem incorrectly thinking it was a huggingface, etc. issue.

As much as I'd love to debug where None comes from in:

OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'

I found the "clue" lies here:

creating model quickly: OSError

and here:

Failed to create model quickly; will retry using slow method

It's a stable-diffusion-webui "feature" in ../modules/sd_models.py

***** EDIT begin *****
Doing some simple debugging, I found 'disable_clip' evaluates to (True or False) = True

disable_clip=clip_is_included_into_sd or shared.cmd_opts.do_not_download_clip

Further, 'clip_is_included_into_sd' evaluates to True because 'sd1_clip_weight' is found in 'state_dict'

sd1_clip_weight = 'cond_stage_model.transformer.text_model.embeddings.token_embedding.weight'

clip_is_included_into_sd = (
      any(x for x in [sd1_clip_weight, sd2_clip_weight, sdxl_clip_weight, sdxl_refiner_clip_weight]
          if x in state_dict)
)

Until I figure out why 'disable_clip' set to True errors out, I'm forcing it to False so I only need to override one line of the original code.

My [revised] simple workaround:

<snipped>
sd_model = None
try:
    # with sd_disable_initialization.DisableInitialization(disable_clip=clip_is_included_into_sd or shared.cmd_opts.do_not_download_clip):
    with sd_disable_initialization.DisableInitialization(disable_clip=False):
        with sd_disable_initialization.InitializeOnMeta():
            sd_model = instantiate_from_config(sd_config.model)

except Exception as e:
    errors.display(e, "creating model quickly", full_traceback=True)

if sd_model is None:
    print('Failed to create model quickly; will retry using slow method.', file=sys.stderr)
<snipped>

***** EDIT end *****

My [initial] simple workaround:

<snipped>
sd_model = None
# try:
#     with sd_disable_initialization.DisableInitialization(disable_clip=clip_is_included_into_sd or shared.cmd_opts.do_not_download_clip):
#         with sd_disable_initialization.InitializeOnMeta():
#             sd_model = instantiate_from_config(sd_config.model)
#
# except Exception as e:
#     errors.display(e, "creating model quickly", full_traceback=True)

if sd_model is None:
    # print('Failed to create model quickly; will retry using slow method.', file=sys.stderr)
    print('Creating model using slow method.', file=sys.stderr)
<snipped>

Enjoy

[Prurite]

Until I figure out why 'disable_clip' set to True errors out, I'm forcing it to False so I only need to override one line of the original code.

Forcing disable_clip to False will result in downloading some large clip models. My solution is to force it to True with the fix below:

AUTOMATIC1111/stable-diffusion-webui#13245 (comment)