Model training notes

[emmamendelsohn]

Cross validation:

• Spatial blocking or clustering (kmeans) in predictor space
• Likely to drop certain proportion of negative pixels (that have never had events)
• Likely to merge blocks/clusters to ensure positive events are represented in training splits

Addressing rarity of first outbreaks:

1. Need to code first outbreak in a thread versus subsequent outbreaks
2. Stratify train/test and blocking based on first and subsequent events. Evaluate model performance for each.
3. To tune performance for first events: upweight new events in the data and/or write custom evaluation function (weighted logistic error)

[emmamendelsohn]

Relevant papers:

spatialsample package for tidymodel-friendly spatial CV: https://arxiv.org/pdf/2303.07334.pdf
waywiser package for measuring spatial error: https://github.com/ropensci/waywiser
methods of evaluating spatial transferability of model predictions: https://onlinelibrary.wiley.com/doi/10.1111/geb.13635

[emmamendelsohn]

Address now in #95