Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why omnivore fails to fetch content in some webpages #3925

Open
natarslan opened this issue May 9, 2024 · 6 comments
Open

Why omnivore fails to fetch content in some webpages #3925

natarslan opened this issue May 9, 2024 · 6 comments
Labels

Comments

@natarslan
Copy link

I realise that once in a while omnivore cant fetch the article content both in Android app and in pc web browser. I was wondering if there are specific reasons behind this or is it just a random thing?

The current fail is in this page:
https://climeworks.com/press-release/climeworks-switches-on-worlds-largest-direct-air-capture-plant-mammoth

@natarslan natarslan added the QA label May 9, 2024
@jacksonh
Copy link
Contributor

jacksonh commented May 9, 2024

Hey, that particular one seemed to fetch OK for me. Do you mind posting the error you see?

Typically when we can't fetch a URL its some sort of blocking enabled by the site. Often it is CloudFlare "bot protection" which can be quite aggressive.

@natarslan
Copy link
Author

natarslan commented May 9, 2024

Hey,

I double checked the web page. It actually works in PC (web browser now). But it still doesnt work on Android (version 14). Here comes the screenshot.

What I also checked:

  • Made sure that NetGuard (vpn) is not blocking internet access to omnivore -> fetch fails
  • Totally disabled NetGuard in case it was somehow interfering with Omnivore -> fetch still fails

omnivore|100

@jacksonh
Copy link
Contributor

When you say works on PC, you saved it with the browser extension there?

@natarslan
Copy link
Author

Yes in desktop browser it works. But in android app the page didnt work (see the screenshot).

@jacksonh
Copy link
Contributor

Yes in desktop browser it works. But in android app the page didnt work (see the screenshot).

but you only saved the page once? Or you saved on Android also?

@natarslan
Copy link
Author

natarslan commented May 13, 2024

I tried saving the page

  • first on android (failed to fetch - so i removed the item)
  • and then on my desktop browser (failed first, then worked).
  • Lastly, I deleted the page from browser and tried it again on Android (failed).

I can confirm that it now works on Android also.

The question is, is there anything I can do to "help" omnivore fetch the pages (can disabling adblockers etc help)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants