A Set of Control Points Conditioned Pedestrian Trajectory Prediction

Inhwan Bae · Hae-Gon Jeon
AAAI 2023

Project Page AAAI Paper Source Code Related Works

An illustration of our Graph-TERN model

This repository contains the code for control point conditioned prediction and the initial trajectory refinement network for trajectory prediction.

🚩 Graph-TERN Model 🚩

• Control points divide each pedestrian's future path into sections and are inferred by each stochastic goal.
• Gaussian mixture model pruning scheme to effectively cutting-off abnormal behaviors of pedestrians.
• Initial trajectory refinement by adding the correction vector field to the linearly interpolated path.

News

• The latest version of Graph-TERN is now available, offering significantly improved results!
• You can find the exact AAAI'23 version here.

Model Training

Setup

Environment
All models were trained and tested on Ubuntu 18.04 with Python 3.7 and PyTorch 1.6.0 with CUDA 10.1.

Dataset
Preprocessed ETH and UCY datasets are included in this repository, under ./dataset/. The train/validation/test splits are the same as those fond in Social-GAN.

Train Graph-TERN

To train our Graph-TERN on the ETH and UCY datasets at once, we provide a bash script train.sh for a simplified execution.

./scripts/train.sh

We provide additional arguments for experiments:

./scripts/train.sh -p <experiment_tag_prefix> -s <experiment_tag_suffix> -d <space_seperated_dataset_string> -i <space_seperated_gpu_id_string>

# Examples
./scripts/train.sh -d "hotel" -i "1"
./scripts/train.sh -p graph-tern_ -s _experiment -d "zara2" -i "2"
./scripts/train.sh -d "eth hotel univ zara1 zara2" -i "0 0 0 0 0"

If you want to train the model with custom hyper-parameters, use train.py instead of the script file.

python train.py --input_size <input_coordinate_dimension> --output_size <output_gaussian_dimension> \
--n_epgcn <number_of_control_point_gcn_layers> --n_epcnn <number_of_control_point_cnn_layers> \
--n_trgcn <number_of_refinement_gcn_layers> --n_trcnn <number_of_refinement_cnn_layers> \
--n_ways <number_of_control_points>  --n_smpl <number_of_samples_for_refine> --kernel_size <kernel_size>\
--obs_seq_len <observation_length> --pred_seq_len  <prediction_length> --dataset <dataset_name> \
--batch_size <minibatch_size> --num_epochs <number_of_epochs> --clip_grad <gradient_clipping> \
--lr <learning_rate> --lr_sh_rate <number_of_steps_to_drop_lr> --use_lrschd <use_lr_scheduler> \
--tag <experiment_tag>

Model Evaluation

Pretrained Models

We have included pretrained models in the ./checkpoint/ folder.

Evaluate Graph-TERN

To evaluate our Graph-TERN at once, we provide a bash script test.sh for a simplified execution.

./scripts/test.sh -p <experiment_tag_prefix> -s <experiment_tag_suffix> -d <space_seperated_dataset_string> -i <space_seperated_gpu_id_string>

# Examples
./scripts/test.sh
./scripts/test.sh -d "hotel" -i "1"
./scripts/test.sh -p graph-tern_ -s _experiment -d "zara2" -i "2"
./scripts/test.sh -d "eth hotel univ zara1 zara2" -i "0 0 0 0 0"

If you want to evaluate the model individually, you can use test.py with custom hyper-parameters.

python test.py --tag <experiment_tag> --n_samples <number_of_multimodal_samples>

# Examples
python test.py --tag graph-tern_eth_experiment
python test.py --tag graph-tern_hotel_experiment
python test.py --tag graph-tern_univ_experiment
python test.py --tag graph-tern_zara1_experiment
python test.py --tag graph-tern_zara2_experiment

📖 Citation

If you find this code useful for your research, please cite our trajectory prediction papers :)

💬 LMTrajectory (CVPR'24) 🗨️ | 1️⃣ SingularTrajectory (CVPR'24) 1️⃣ | 🌌 EigenTrajectory (ICCV'23) 🌌 | 🚩 Graph‑TERN (AAAI'23) 🚩 | 🧑‍🤝‍🧑 GP‑Graph (ECCV'22) 🧑‍🤝‍🧑 | 🎲 NPSN (CVPR'22) 🎲 | 🧶 DMRGCN (AAAI'21) 🧶

@article{bae2023graphtern,
  title={A Set of Control Points Conditioned Pedestrian Trajectory Prediction},
  author={Bae, Inhwan and Jeon, Hae-Gon},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2023}
}

More Information (Click to expand)

@inproceedings{bae2024lmtrajectory,
  title={Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction},
  author={Bae, Inhwan and Lee, Junoh and Jeon, Hae-Gon},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}

@inproceedings{bae2024singulartrajectory,
  title={SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model},
  author={Bae, Inhwan and Park, Young-Jae and Jeon, Hae-Gon},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}

@inproceedings{bae2023eigentrajectory,
  title={EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting},
  author={Bae, Inhwan and Oh, Jean and Jeon, Hae-Gon},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}

@inproceedings{bae2022gpgraph,
  title={Learning Pedestrian Group Representations for Multi-modal Trajectory Prediction},
  author={Bae, Inhwan and Park, Jin-Hwi and Jeon, Hae-Gon},
  booktitle={Proceedings of the European Conference on Computer Vision},
  year={2022}
}

@inproceedings{bae2022npsn,
  title={Non-Probability Sampling Network for Stochastic Human Trajectory Prediction},
  author={Bae, Inhwan and Park, Jin-Hwi and Jeon, Hae-Gon},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}

@article{bae2021dmrgcn,
  title={Disentangled Multi-Relational Graph Convolutional Network for Pedestrian Trajectory Prediction},
  author={Bae, Inhwan and Jeon, Hae-Gon},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2021}
}