Skip to content
This repository has been archived by the owner on Jul 8, 2024. It is now read-only.

How to handle a urlopen error? #67

Open
jkjan opened this issue Apr 22, 2020 · 1 comment · May be fixed by #70
Open

How to handle a urlopen error? #67

jkjan opened this issue Apr 22, 2020 · 1 comment · May be fixed by #70

Comments

@jkjan
Copy link

jkjan commented Apr 22, 2020

Sometimes I get this error and I don't know how to handle it.
This is what I get when I try to crawl tweets like 500 keyword, a keyword at a time.
It takes a rest for 16 minutes after it collects 10k more tweets.
This error is just random. My teammates has not reported this one yet.

"An error occured during an HTTP request: <urlopen error [WinError 10060]
A connection attempt failed because the
connected party did not properly respond after a period of time, or
established connection failed because connected host has failed to respond
Try to open in browser: ~~ twitter link ~~"

I know why the problem occurs and that is something I cannot fix.
This is very rare and I don't have time or capability of fixing that occasional connection error. And if it happens, all processes stop, so I should restart the whole program.

So I decided to avoid that error and go on to the next keyword using try and except.
Because I can save a keyword that made an error and search for that later.
I tried Exception, but it didn't catch that error.
Also HTTPError or URLError, didn't catch that either.
What kind of exception can catch that error?

Summary :
This is an example code.
image

How can I print that "Error!" ?

@marmistrz
Copy link

You can't. The library just calls sys.exit. Until #70 is merged, you can use my fork: #70. Warning: the fork will not be updated.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants