-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
EPG parser is broken because cfscrape was broken by urllib3 update #27
Comments
for now, the parser still seems to work and there is data in the data repo. Meanwhile, if somebody has experience with Cloudflare scraping I am open for pull requests |
Since the endpoint the scraper is hitting is an API endpoint (at least for teleboy), I wouldn't have thought this would have any cf anti-bot measures on it so cfscrape would likely not be required in any case. |
was removed by @rbkn and it seems to work after a small fix |
Thanks both! :D |
cfscrape
(https://github.com/Anorov/cloudflare-scrape) doesn't look like it's being developed any longer. Meanwhile,urllib3
has continued changing, and in version 2.0.0 the removed support forDEFAULT_CIPHERS
, whichcfscrape
relies on.The result is that trying to run the EPG parser fails with
The best solution would be to no longer depend on
cfscrape
. I am not familiar with it, but among the 118 open issues there seems to be quite a few mentioning that it's broken and doesn't really bypass Cloudflare anymore (like 455, 458 and 459).As a temporary measure I was able to make the parser work with a
pip3 install urllib3==1.26.16
(the last release before 2.0.0); this may be a good addition torequirements.txt
(urllib3
is installed byrequests
, but it doesn't seem to create a problem. Yet), but I expect this to break once and for all whenrequests
starts requiringurllib3>=2.0.0
:(Hope this helps!
The text was updated successfully, but these errors were encountered: