We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hello,
could anybody help with this issue?
Thx a lot Marcus
docker-compose logs -f:
web-app | [2021-01-27 11:25:13 +0000] [10] [DEBUG] POST /new/ crawler-worker | 11:25:13 high: crawler.run('http://p6o7m73ujalhgkiv.onion') (276dba5f-d77f-4352-91de-6fb1b728c2d2) web-app | [2021-01-27 11:25:13 +0000] [10] [DEBUG] GET /favicon.ico crawler-worker | 11:25:21 Traceback (most recent call last): crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job crawler-worker | rv = job.perform() crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 696, in perform crawler-worker | self._result = self._execute() crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 719, in _execute crawler-worker | return self.func(*self.args, **self.kwargs) crawler-worker | File "/application/crawler/init.py", line 10, in run crawler-worker | spider.proccess() crawler-worker | File "/application/crawler/spider.py", line 73, in proccess crawler-worker | 'links': data.get_links(), crawler-worker | File "/application/crawler/html_extractors.py", line 20, in get_links crawler-worker | is_onion = True if '.onion' in parsed_href.netloc else False crawler-worker | TypeError: a bytes-like object is required, not 'str' crawler-worker | Traceback (most recent call last): crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job crawler-worker | rv = job.perform() crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 696, in perform crawler-worker | self._result = self._execute() crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 719, in _execute crawler-worker | return self.func(*self.args, **self.kwargs) crawler-worker | File "/application/crawler/init.py", line 10, in run crawler-worker | spider.proccess() crawler-worker | File "/application/crawler/spider.py", line 73, in proccess crawler-worker | 'links': data.get_links(), crawler-worker | File "/application/crawler/html_extractors.py", line 20, in get_links crawler-worker | is_onion = True if '.onion' in parsed_href.netloc else False crawler-worker | TypeError: a bytes-like object is required, not 'str'
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Hello,
could anybody help with this issue?
Thx a lot
Marcus
docker-compose logs -f:
web-app | [2021-01-27 11:25:13 +0000] [10] [DEBUG] POST /new/
crawler-worker | 11:25:13 high: crawler.run('http://p6o7m73ujalhgkiv.onion') (276dba5f-d77f-4352-91de-6fb1b728c2d2)
web-app | [2021-01-27 11:25:13 +0000] [10] [DEBUG] GET /favicon.ico
crawler-worker | 11:25:21 Traceback (most recent call last):
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job
crawler-worker | rv = job.perform()
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 696, in perform
crawler-worker | self._result = self._execute()
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 719, in _execute
crawler-worker | return self.func(*self.args, **self.kwargs)
crawler-worker | File "/application/crawler/init.py", line 10, in run
crawler-worker | spider.proccess()
crawler-worker | File "/application/crawler/spider.py", line 73, in proccess
crawler-worker | 'links': data.get_links(),
crawler-worker | File "/application/crawler/html_extractors.py", line 20, in get_links
crawler-worker | is_onion = True if '.onion' in parsed_href.netloc else False
crawler-worker | TypeError: a bytes-like object is required, not 'str'
crawler-worker | Traceback (most recent call last):
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job
crawler-worker | rv = job.perform()
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 696, in perform
crawler-worker | self._result = self._execute()
crawler-worker | File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 719, in _execute
crawler-worker | return self.func(*self.args, **self.kwargs)
crawler-worker | File "/application/crawler/init.py", line 10, in run
crawler-worker | spider.proccess()
crawler-worker | File "/application/crawler/spider.py", line 73, in proccess
crawler-worker | 'links': data.get_links(),
crawler-worker | File "/application/crawler/html_extractors.py", line 20, in get_links
crawler-worker | is_onion = True if '.onion' in parsed_href.netloc else False
crawler-worker | TypeError: a bytes-like object is required, not 'str'
The text was updated successfully, but these errors were encountered: