This is the demo code for the MuVS (Multi-view SMPLify) method presented in the paper:
You need to install the the following packages via pip like shown below:
pip install --upgrade numpy scipy tensorflow-gpu matplotlib
pip install git+https://github.com/mattloper/chumpy.git@refs/pull/18/head # required since the main branch is broken since pip 10
pip install https://github.com/MPI-IS/meshlite/raw/master/download/psbody_meshlite-0.1-cp27-cp27mu-linux_x86_64.whl # meshlite
Additionally, smpl needs to be downloaed from http://smpl.is.tue.mpg.de/downloads and symlinked
ln -s <smpl dir>/smpl_webuser TF_Code/smpl
Deepcut and one human-specific segmentation method are also used (Not used now, since no differentialbe render in Tensorflow availabe). You need to install them from the project links.
This is where the code resides. A brief summary of what each file does in the following:
camera.py: perspective pinhole camera model, which projects 3D points into image space.
smpl_batch.py: Tensorflow verion of SMPL model, which takes in SMPL paramters and yields 3D SMPL joints.
util.py: experimental settings and data loading methods.
Image data, DCT basis, SMPL model and pose prior is stored here. If you want to try your own data, you need to prepare it in the way shown in folder HEVA_Validate:
1, One folder for each view.
2, For each view, images are in *Image* subfolder.
3, 2D pose information in *Pose_2D subfolder*.
4, Create subfolders *Res_1* and *Res_2* to save the results.
Firstly you have to organize your data like described in the previous setction. Then you need to run Deepercut for all the images, and place them in the right folder. After that you can use these commands to run MuVS from the main folder:
./gen_run_jobs_1.sh # This runs per-frame fitting for all the frames
./gen_run_jobs_2.sh # This runs temporal fitting for all the temporal units
The resultant files (3D SMPL mesh, SMPL shape, pose and translation parameters) will be saved in ply and pkl in Res_1/Res_2 respectively.
If you find this code useful for your research, please consider citing:
@inproceedings{MuVS:3DV:2017,
title = {Towards Accurate Marker-less Human Shape and Pose Estimation over Time},
author = {Huang, Yinghao and Bogo, Federica and Lassner, Christoph and Kanazawa, Angjoo and Gehler, Peter V. and Romero, Javier and Akhter, Ijaz and Black, Michael J.},
booktitle = {International Conference on 3D Vision (3DV)},
year = {2017}
}