Skip to content

Code for the paper "Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language", RAL with ICRA 2022

License

Notifications You must be signed in to change notification settings

0aqz0/neural-retargeting

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural Retargeting

Code for the paper "Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language"

arXiv YouTube Bilibili

Prerequisite

  • PyTorch Tensors and Dynamic neural networks in Python with strong GPU acceleration
  • pytorch_geometric Geometric Deep Learning Extension Library for PyTorch
  • Kornia a differentiable computer vision library for PyTorch.
  • HDF5 for Python The h5py package is a Pythonic interface to the HDF5 binary data format.

Dataset

The Chinese sign language dataset can be downloaded here.

Model

The pretrained model can be downloaded here.

Get Started

Training

CUDA_VISIBLE_DEVICES=0 python main.py --cfg './configs/train/yumi.yaml'

Inference

CUDA_VISIBLE_DEVICES=0 python inference.py --cfg './configs/inference/yumi.yaml'

Simulation Experiment

We build the simulation environment using pybullet, and the code is in this repository.

After inference is done, the motion retargeting results are stored in a h5 file. Then run the sample code here.

Real-World Experiment

Real-world experiments could be conducted on ABB's YuMi dual-arm collaborative robot equipped with Inspire-Robotics' dexterous hands.

We release the code in this repository, please follow the instructions.

Citation

If you find this project useful in your research, please cite this paper.

@article{zhang2022kinematic,
  title={Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language},
  author={Zhang, Haodong and Li, Weijie and Liu, Jiangpin and Chen, Zexi and Cui, Yuxiang and Wang, Yue and Xiong, Rong},
  journal={IEEE Robotics and Automation Letters},
  year={2022},
  publisher={IEEE}
}

About

Code for the paper "Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language", RAL with ICRA 2022

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages