Data preparation and transformation exercise

The objective of this exercise is to practice various steps of data preprocessing and feature engineering.

The scenario is the preparation of data for a ML multilinear regressions.

The dataset used is the "Climate Weather Surface of Brazil - Hourly", wich is available at Kaggle.

It contains hourly climate data taken from weather stations in Brasil, taken between 2000 and 2021.

This exercise is broken down as follows:

Part I

1. Load data
2. Inspect data

Part II

3. Format features
4. Clean messy data
5. Remove duplicate values

Part III

6. Treat missing values
7. Imputation

Part IV

8. Remove strongly correlated features
9. Remove outliers

Part V

10. Aggregate features
11. Encode categorical features
12. Feature scaling
13. Dimensionality reduction and feature decomposition

Part VI

14. Sample and balance