Fixed cf new challenge solution #373

Sraq-Zit · 2020-06-20T01:44:51Z

There was mainly 3 problems:

Value of jschl_vc token was taken from commented input element instead of the actual one
The value of the div element with its ID as the variable k is split into multiple divs and numbered at the end of the ID for each div
node: setInterval not defined. I removed that part from js code

Tested on:

EDIT
As of yesterday the tested websites are no more working, due to the new cf challenge.
The challenge lies within the script at the path that goes like example.com/cdn-cgi/challenge-platform/orchestrate/jsch/v1
Once the script is loaded and executed, the function _cf_chl_enter() is called.

I will try and see if I can automate the same process in python

mcronce · 2020-06-21T19:55:24Z

I'm getting a failure when I install from your fork:

ERROR:root:Error executing Cloudflare IUAM Javascript. Cloudflare may have changed their technique, or there may be a bug in the script.

Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."
ERROR:root:'https://www.iptorrents.com/login.php' returned an error. Could not collect tokens.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 386, in get_tokens
    resp = scraper.get(url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 129, in request
    resp = self.solve_cf_challenge(resp, **kwargs)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 204, in solve_cf_challenge
    answer, delay = self.solve_challenge(body, domain)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 331, in solve_challenge
    raise subprocess.CalledProcessError(node.returncode, "node -e ...", stderr)
subprocess.CalledProcessError: Command 'node -e ...' returned non-zero exit status 1

I'm going to try to manually run node and see if its output provides any clue as to why it's bombing out, will report back with results

Sraq-Zit · 2020-06-22T04:27:02Z

Alright !

By the way, I tried it and it worked fine with me

pilardo · 2020-06-24T11:46:43Z

@Sraq-Zit i'm newbie. how can i update my local file?

Sraq-Zit · 2020-06-24T13:55:59Z

@Sraq-Zit i'm newbie. how can i update my local file?

Reinstall it through this fork

pip install https://github.com/Sraq-Zit/cloudflare-scrape/archive/master.zip

chanind · 2020-07-01T22:45:15Z

cfscrape/__init__.py

 challenge, ms = re.search(
 r"setTimeout\(function\(\){\s*(var "
- r"s,t,o,p,b,r,e,a,k,i,n,g,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"
- r"(?:[^{<>]*},\s*(\d{4,}))?",
+ r"s,t,o,p, b,r,e,a,k,i,n,g,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"


This PR only works for me if I add ,cpo into this line, like:
r"s,t,o,p, b,r,e,a,k,i,n,g,cpo,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"

This PR only works for me if I add ,cpo into this line, like:
r"s,t,o,p, b,r,e,a,k,i,n,g,cpo,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"

It still don't work with me . Can you test this site ?
https://tubortaxdeal.com
https://www.aichoai.com

weird, the JS on those 2 sites looks really different, like it's not even minified or something. I wonder if they're using a different version of the cloudflare protection stuff?

This PR only works for me if I add ,cpo into this line, like:
r"s,t,o,p, b,r,e,a,k,i,n,g,cpo,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"

It was added just recently for the websites I tested.
Thanks !

This PR only works for me if I add ,cpo into this line, like:
r"s,t,o,p, b,r,e,a,k,i,n,g,cpo,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"

It was added just recently for the websites I tested.
Thanks !

don't work for https://localbitcoins.net

Also doesn't work for http://www.mangago.me/read-manga/hataraku_maousama/

andress134 · 2020-07-23T11:22:09Z

I'm getting a failure when I install from your fork:

ERROR:root:Error executing Cloudflare IUAM Javascript. Cloudflare may have changed their technique, or there may be a bug in the script.

Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."
ERROR:root:'https://www.iptorrents.com/login.php' returned an error. Could not collect tokens.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 386, in get_tokens
    resp = scraper.get(url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 129, in request
    resp = self.solve_cf_challenge(resp, **kwargs)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 204, in solve_cf_challenge
    answer, delay = self.solve_challenge(body, domain)
  File "/usr/lib/python2.7/site-packages/cfscrape/__init__.py", line 331, in solve_challenge
    raise subprocess.CalledProcessError(node.returncode, "node -e ...", stderr)
subprocess.CalledProcessError: Command 'node -e ...' returned non-zero exit status 1

I'm going to try to manually run node and see if its output provides any clue as to why it's bombing out, will report back with results

I see cloudflare has updated again the default challenge for uam

F170x · 2020-07-30T13:37:36Z

Doesn't work with: https://www.pexels.com/search/man/?format=js&seed=&page=2&type=
I don't know how to cloudflare detect between browser incognit mode and a python script, maybe with a difference in the headers or in the ssl protocol version?

jaeakle · 2020-08-12T02:40:54Z

Did Cloudflare change again?
Im getting the error with this line as usual:
r"s,t,o,p, b,r,e,a,k,i,n,g,cpo,f.+?\r?\n[\s\S]+?a\.value\s*=.+?)\r?\n"

rnyPlanet · 2020-11-01T20:31:57Z

works today?

andress134 · 2020-11-03T00:02:29Z

works today?

no

rnyPlanet · 2020-11-04T08:57:21Z

@chanind @Sraq-Zit can you help update?

hellcoder420 · 2020-12-06T06:52:35Z

can someone please continue the work of this pr

ngooty · 2020-12-28T07:15:39Z

Is there a solution for this problem. am stuck since a month to resolve this.

rnyPlanet · 2020-12-28T08:45:15Z

Is there a solution for this problem. am stuck since a month to resolve this.

Wow. and how are you getting on with the solution? I also tried but I have little knowledge ..

ngooty · 2020-12-28T09:21:30Z

I have not got any solution or workaround. still stuck with the problem thanks, Nag

…

On Mon, Dec 28, 2020 at 2:15 PM Stanislav ***@***.***> wrote: Is there a solution for this problem. am stuck since a month to resolve this. Wow. and how are you getting on with the solution? I also tried but I have little knowledge .. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#373 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANN2TRNDXX22OJGVOIWBMN3SXBASRANCNFSM4ODFEWRA> .

ngooty · 2020-12-28T12:44:05Z

Hi, I got a workaround for the problem. get signed up to "scrapingant.com" and use its API (it will produce unique key) to scape the websites behind cloud flare. Thanks, Nag

…

On Mon, Dec 28, 2020 at 2:51 PM Nagaraju Gooty ***@***.***> wrote: I have not got any solution or workaround. still stuck with the problem thanks, Nag On Mon, Dec 28, 2020 at 2:15 PM Stanislav ***@***.***> wrote: > Is there a solution for this problem. am stuck since a month to resolve > this. > > Wow. and how are you getting on with the solution? I also tried but I > have little knowledge .. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#373 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ANN2TRNDXX22OJGVOIWBMN3SXBASRANCNFSM4ODFEWRA> > . >

andress134 · 2021-06-28T01:27:14Z

@Sraq-Zit is posible to fix v1 challenge again?
thanks

hirios · 2022-01-09T07:16:04Z

+ 1

Fixed kissanime-cf new challenge

7ee8825

Sraq-Zit mentioned this pull request Jun 20, 2020

cloudflare #371

Open

4 tasks

Version change

3b924a7

Sraq-Zit mentioned this pull request Jun 27, 2020

ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique #376

Open

4 tasks

chanind reviewed Jul 1, 2020

View reviewed changes

Changed regexp for js to match that of cloudflare

9546707

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed cf new challenge solution #373

Fixed cf new challenge solution #373

Sraq-Zit commented Jun 20, 2020 •

edited

mcronce commented Jun 21, 2020

Sraq-Zit commented Jun 22, 2020

pilardo commented Jun 24, 2020

Sraq-Zit commented Jun 24, 2020

chanind Jul 1, 2020

caipenpen Jul 2, 2020

chanind Jul 2, 2020

Sraq-Zit Jul 5, 2020

dentakoy Jul 6, 2020

AAccount Jul 15, 2020

andress134 commented Jul 23, 2020

F170x commented Jul 30, 2020

jaeakle commented Aug 12, 2020 •

edited

rnyPlanet commented Nov 1, 2020

andress134 commented Nov 3, 2020

rnyPlanet commented Nov 4, 2020 •

edited

hellcoder420 commented Dec 6, 2020

ngooty commented Dec 28, 2020

rnyPlanet commented Dec 28, 2020

ngooty commented Dec 28, 2020 via email

ngooty commented Dec 28, 2020 via email

andress134 commented Jun 28, 2021

hirios commented Jan 9, 2022 •

edited

Fixed cf new challenge solution #373

Are you sure you want to change the base?

Fixed cf new challenge solution #373

Conversation

Sraq-Zit commented Jun 20, 2020 • edited

mcronce commented Jun 21, 2020

Sraq-Zit commented Jun 22, 2020

pilardo commented Jun 24, 2020

Sraq-Zit commented Jun 24, 2020

chanind Jul 1, 2020

Choose a reason for hiding this comment

caipenpen Jul 2, 2020

Choose a reason for hiding this comment

chanind Jul 2, 2020

Choose a reason for hiding this comment

Sraq-Zit Jul 5, 2020

Choose a reason for hiding this comment

dentakoy Jul 6, 2020

Choose a reason for hiding this comment

AAccount Jul 15, 2020

Choose a reason for hiding this comment

andress134 commented Jul 23, 2020

F170x commented Jul 30, 2020

jaeakle commented Aug 12, 2020 • edited

rnyPlanet commented Nov 1, 2020

andress134 commented Nov 3, 2020

rnyPlanet commented Nov 4, 2020 • edited

hellcoder420 commented Dec 6, 2020

ngooty commented Dec 28, 2020

rnyPlanet commented Dec 28, 2020

ngooty commented Dec 28, 2020 via email

ngooty commented Dec 28, 2020 via email

andress134 commented Jun 28, 2021

hirios commented Jan 9, 2022 • edited

Sraq-Zit commented Jun 20, 2020 •

edited

jaeakle commented Aug 12, 2020 •

edited

rnyPlanet commented Nov 4, 2020 •

edited

hirios commented Jan 9, 2022 •

edited