Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to crawl only pages whose url match certain prefix? #89

Open
kanihal opened this issue Aug 20, 2017 · 0 comments
Open

How to crawl only pages whose url match certain prefix? #89

kanihal opened this issue Aug 20, 2017 · 0 comments
Labels
enhancement Hacktoberfest help wanted

Comments

@kanihal
Copy link

kanihal commented Aug 20, 2017 •
edited
Loading

How to crawl and save only pages for site say : https://www.cse.iitb.ac.in/~soumen/
i.e. I dont want to crawl the domain cse.iitb.ac.in but rather just the pages that appear with prefix "https://www.cse.iitb.ac.in/~soumen/" i.e. https://www.cse.iitb.ac.in/~soumen/*

If "prefix match" crawl feature is not there, could you please add it?
something like
selection_003
@kanihal kanihal changed the title How crawl only pages whose url match certain prefix? How to crawl only pages whose url match certain prefix? Aug 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Hacktoberfest help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kanihal @N0taN3rd