Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error when run python crawler.py posts #127

Open
8thstar opened this issue Jul 23, 2022 · 1 comment
Open

error when run python crawler.py posts #127

8thstar opened this issue Jul 23, 2022 · 1 comment

Comments

@8thstar
Copy link

8thstar commented Jul 23, 2022

Error occurred during loading data. Trying to use cache server https://fake-useragent.herokuapp.com/browsers/0.1.11
Traceback (most recent call last):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 154, in load
for item in get_browsers(verify_ssl=verify_ssl):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 99, in get_browsers
html = html.split('

')[1]
IndexError: list index out of range
Traceback (most recent call last):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 154, in load
for item in get_browsers(verify_ssl=verify_ssl):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 99, in get_browsers
html = html.split('
')[1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1447, in connect
super().connect()
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 946, in connect
self.sock = self._create_connection(
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\socket.py", line 844, in create_connection
raise err
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\socket.py", line 832, in create_connection
sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 64, in get
with contextlib.closing(urlopen(
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 517, in open
response = self._open(req, data)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\insta crawler\crawler.py", line 82, in
get_posts_by_user(
File "E:\insta crawler\crawler.py", line 27, in get_posts_by_user
ins_crawler = InsCrawler(has_screen=debug)
File "E:\insta crawler\inscrawler\crawler.py", line 68, in init
self.browser = Browser(has_screen)
File "E:\insta crawler\inscrawler\browser.py", line 25, in init
chrome_options.add_argument("user-agent="+UserAgent().random)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\fake.py", line 69, in init
self.load()
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\fake.py", line 75, in load
self.data = load_cached(
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 250, in load_cached
update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 245, in update
write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 187, in load
ret = json.loads(get(
File "C:\Users\8thstar\AppData\Local\Programs\Python\Python39\lib\site-packages\fake_useragent\utils.py", line 84, in get
raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

@jin749
Copy link

jin749 commented May 6, 2023

You can do this: pip install --upgrade fake-useragent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants