Code

[WIP] 2023 updates#76

Code

Open

simonwoerpel wants to merge 30 commits intoopenspending:masteropenspending/farmsubsidy-scrapers:masterfrom investigativedata:masterinvestigativedata/farmsubsidy-scrapers:masterCopy head branch name to clipboard

+17,514-2,743

Commits

Commits on Nov 2, 2023

• Update requirements

  simonwoerpel

  committed
  179a962

• Ignore scraper data directories

  simonwoerpel

  committed
  86c5052

• Update scraper user agent

  simonwoerpel

  committed
  fd28600

• BE: 2023 tweaks

  simonwoerpel

  committed
  6f3d0ca

Commits on Nov 3, 2023

• ee spider: fix year parameter to take effect

  Tilana

  committed
  8e32816

• es_scraper: add 2022 url and apply minor fixes

  Tilana

  committed
  104c68d

• fr_scraper: adjust for 2022 data

  Tilana

  committed
  ddf652b

• gb_scraper: adjust to 2022

  Tilana

  committed
  2eb0b7b

Commits on Nov 5, 2023

• LU: scrapy working 2023

  simonwoerpel

  committed
  8eaccee

• AT: scrapy working 2023

  simonwoerpel

  committed
  497d221

• DE: Add scrapy scraper

  simonwoerpel

  committed
  a1b0ebc

• LV: add scrapy scraper

  simonwoerpel

  committed
  a90d424

• SK: Fix scrapy scraper for 2022

  simonwoerpel

  committed
  e04b65b

• HU: Update converter for 2023

  simonwoerpel

  committed
  ec23cd1

• EU: Cool URIs always change

  simonwoerpel

  committed
  e9634bf

• here as well -.-

  simonwoerpel

  committed
  045ecad

Commits on Nov 7, 2023

• Merge pull request #1 from investigativedata/dev

  Show description for 580e1fa
  simonwoerpel

  authored
  580e1fa

Commits on Nov 8, 2023

• BE: slugify

  simonwoerpel

  committed
  6840e9b

Commits on Nov 9, 2023

• MT: add a bad notebook

  simonwoerpel

  committed
  d15ec4c

• LT: add some code

  simonwoerpel

  committed
  e1088e7

• CY: adjust data source to 2022 / flag old scraper

  Tilana

  committed
  2ba064a

Commits on Nov 22, 2024

• Improve folder structure

  jfilter

  committed
  e7017d3

• Improve README

  jfilter

  committed
  fb13a5e

• Update requirements to get it to run again

  jfilter

  committed
  27419ca

Commits on Dec 8, 2024

• Add basic cli structure

  jfilter

  committed
  2639dc1

Commits on Dec 9, 2024

• Add file download

  jfilter

  committed
  6a4d5b8

Commits on Dec 10, 2024

• Add file processing

  jfilter

  committed
  e5c892c

Commits on Dec 12, 2024

• Add description for hard-to-automate exports

  jfilter

  committed
  12117e2

• Add new RO scraper

  jfilter

  committed
  b7ac44f

Commits on Dec 13, 2024

• Add option to scrape sequentially

  jfilter

  committed
  0e71615