Hyperparameter Optimization with Ray Tune

Full Tech Talk on Youtube

Getting Started

This repository provides an example implementation of a Ray Tune hyperparameter search with a PyTorch training pipeline and MLflow logging. Below is an overview of the repository's contents:

• requirements.txt: Python packages you can pip install that are required to run the code in this repository
• hyperparam_search.py: Python script that loads the CIFAR dataset, defines the training pipeline, and kicks off a hyperparameter search with Ray Tune

Environment Setup

This section provides instructions for setting up your environment and installing dependencies you will need to execute the code in this repository.

Start by cloning this project into your directory and changing the directory:

git clone https://github.com/modzy/ray-tune-hyperparameter-search.git
cd ray-tune-hyperparameter-search

Next, in your Python environment (must be v3.7 or greater), create and activate a virtual environment with your preferred virtual environment tool (conda, pyenv, venv, etc.) These instructions will leverage Python's native venv module.

python -m venv venv

Activate environment.

For Linux or MacOS:

source venv/bin/activate

For Windows:

.\venv\Scripts\activate

Finally, use pip to install the python packages required to run the API:

pip install -r requirements.txt

You are all set! Continue following along to test out the hyperparameter search yourself.

Run Hyperparameter Search

If you'd like modify the code to add or remove parameters to tune, but to run the code as-is, simply execute the basic parameter search:

python hyperparam_search.py cifar basic_search