Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I can’t run #1

Open
fehmi opened this issue Jan 5, 2024 · 2 comments
Open

I can’t run #1

fehmi opened this issue Jan 5, 2024 · 2 comments

Comments

@fehmi
Copy link

fehmi commented Jan 5, 2024

Where is dist/index.js?

@kashif-ghafoor
Copy link
Owner

Thank you for using scraper.
After writing readme I made a lot of changes. I updated the overall structure of scraper.

linksScraper first scrapes sales navigator and write profile links to database.
profileScraper then takes unscraped links from database and then scrape them.
I don't have sales navigator account right now to test the scraper.

Actually, I faild at this project as I was not able to bypass linkedin detection. After 200 or 300 scrapes my account gets banned. It was 2 years before. Now I have better ideas to improve the scraper. I will update readme as well as scraper as soon as I find time.

@fehmi
Copy link
Author

fehmi commented Jan 6, 2024

Thanks for your response. When you try to scrap leads page by page from search results, after 100-200 pages LinkedIn hangs the account. To bypass this, I realized we need to save search and loop over the saved search URL, not actual search URL with params.

I don't know how to find a workaround for scrap profile URLs. I'm sure there is a limit for those too.

Do you know a way to find emails for extracted profiles from search results. There are services but they are too expensive. Because I need it for 1.000.000 profiles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants