Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the scraper(s) 100% free of manual workarounds #93

Open
BruceJohnJennerLawso opened this issue Jan 6, 2017 · 3 comments
Open

Make the scraper(s) 100% free of manual workarounds #93

BruceJohnJennerLawso opened this issue Jan 6, 2017 · 3 comments

Comments

@BruceJohnJennerLawso
Copy link
Owner

This is a big one, almost a mini-project in its own right.

Too many csvs have been hand edited to fix errors or mistakes where the scraper was lacking (ie the manual header row fixes for season, playoffs, the entire csv for the 1918 wanderers, etc.)

Really need to fix this, potentially by doing a full rewrite of both scrapers in beautiful soup, so as to eliminate the hacky workarounds required to deal with empty spaces in tables, etc, etc.

@BruceJohnJennerLawso
Copy link
Owner Author

Also the manual header fixes for WHA guest teams

@BruceJohnJennerLawso
Copy link
Owner Author

Simplest way of doing this is probably to get master as stable as possible, make a branch, and repeatedly scrape the data and test until everything is working in one scrape->run step

@BruceJohnJennerLawso
Copy link
Owner Author

#109 just made this a hell of a lot harder

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant