Attend and Discriminate: Beyond the State-of-the-Art for Human Activity Recognition Using Wearable Sensors

Introduction

Wearable sensors are fundamental to improving our understanding of human activities, especially for an increasing number of healthcare applications from rehabilitation to fine-grained gait analysis. Although our collective know-how to solve Human Activity Recognition (HAR) problems with wearables has progressed immensely with end-to-end deep learning paradigms, several fundamental opportunities remain overlooked.

The repository provides the codebase and pre-trained network parameters corresponding to our publication (to appear in IMWUT 2021), where we rigorously explore new opportunities to learn enriched and highly discriminating activity representations.

Citation

If you find our work useful in your research, please consider citing:

@article{10.1145/3448083,
  title={Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors},
  author={Abedin, Alireza and Ehsanpour, Mahsa and Shi, Qinfeng and Rezatofighi, Hamid and Ranasinghe, Damith C},
  journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},
  volume = {5},
  number = {1},
  doi={10.1145/3448083},
  year={2021},
}

Quick Start

Downloads

Firstly, clone this repository

git clone https://github.com/AdelaideAuto-IDLab/Attend-And-Discriminate.git

Subsequently, download the raw HAR datasets, and place the zip file inside the cloned directory. Similarly, download the HAR architecture weights, and place the zip file inside the cloned directory. Next, run the bash script within the cloned directory to uncompress the zip files and cleanup the downloads:

bash clean_download.sh

Training

In order to train a model with the corresponding HAR dataset, simply run:

python main.py --dataset [opportunity,pamap2,skoda,hospital] --train_mode

The executed script generates the corresponding HAR dataset in data/{dataset}, and stores the experiment artefacts in models/{experiment} directory.

As training progresses, three sets of outputs are stored in models/{experiment}/:

• checkpoints: The model weights.
• logs: The tensorboard logged loss and evaluation metrics.
• visuals: The generated confusion matrices.

We can monitor the training progress using TensorBoard:

tensorboard --logdir ./models/{experiment}

By default the experiment name is determined by the current system time. To customize training parameters, arguments can be set in settings.py (e.g. dataset, batch_size, lr,...). Further, to view the argument helper documentation:

python main.py --help

usage: main.py [-h] [--experiment EXPERIMENT] [--train_mode]
               [--dataset {opportunity,skoda,pamap2,hospital}]
               [--window WINDOW] [--stride STRIDE] [--stride_test STRIDE_TEST]
               [--model MODEL] [--epochs EPOCHS] [--load_epoch LOAD_EPOCH]
               [--print_freq PRINT_FREQ]

HAR dataset, model and optimization arguments

optional arguments:
  -h, --help            show this help message and exit
  --experiment EXPERIMENT
                        experiment name (default: None)
  --train_mode          execute code in training mode (default: False)
  --dataset {opportunity,skoda,pamap2,hospital}
                        HAR dataset (default: opportunity)
  --window WINDOW       sliding window size (default: 24)
  --stride STRIDE       sliding window stride (default: 12)
  --stride_test STRIDE_TEST
                        set to 1 for sample-wise prediction (default: 1)
  --model MODEL         HAR architecture (default: AttendDiscriminate)
  --epochs EPOCHS       number of training epochs (default: 300)
  --load_epoch LOAD_EPOCH
                        epoch to resume training (default: 0)
  --print_freq PRINT_FREQ

Inference

In order to create a pre-trained HAR architecture and run inference on the corresponding hold-out test splits, simply run:

python main.py --dataset [opportunity,pamap2,skoda,hospital]

The executed script loads the pre-trained weights from weights/checkpoint_{dataset}.pth, generates the test dataset and reports the recognition performance metrics.

Core Modules

Datasets Module

The datasets.py module extends the PyTorch Dataset class for multi-channel time-series data captured by wearable sensors. The module functionality can be tested by simply running

python datasets.py --dataset [opportunity,pamap2,skoda,hospital]

The executed script creates the corresponding HAR dataset in data directory. This loads the corresponding dataset files, runs sliding window segmentation to partition the raw samples and displays additional information on the created dataset and individual data segments.

Models Module

The models.py module manages the HAR architecture according to the configured settings. The module functionality can be tested by simply running

python models.py --dataset [opportunity,pamap2,skoda,hospital]

The executed script creates the corresponding HAR architecture for the dataset. It also displays a summary of trainable network parameters and layer information, and runs a synthetic data batch through the network to verify the correctness of input/output tensor dimensionalities.