HFNet

This repository contains the implementation of the HFNet from the paper:

HFNet: High-Precision Robotic Grasp Detection in Unstructured Environments Using Hierarchical RGB-D Feature Fusion and Fine-Grained Pose Alignment

Robot grasping experiment video: video

Requirements

• numpy
• opencv-python
• matplotlib
• scikit-image
• imageio
• torch
• torchvision
• torchsummary
• tensorboardX
• pyrealsense2
• Pillow

Datasets

This repository supports both the Cornell Grasping Dataset and Jacquard Dataset.

Cornell Grasping Dataset

1. Download the and extract Cornell Grasping Dataset.
2. Convert the PCD files to depth images by running python -m utils.dataset_processing.generate_cornell_depth <Path To Dataset>

Jacquard Dataset

1. Download and extract the Jacquard Dataset.

Model Training

A model can be trained using the train_network.py script.

Example for Cornell dataset:

python train.py --dataset cornell --dataset-path <Path To Dataset> --description training_cornell

Example for Jacquard dataset:

python train.py --dataset jacquard --dataset-path <Path To Dataset> --description training_jacquard

Model Evaluation

The trained network can be evaluated using the evaluate.py script.

Example for Cornell dataset:

python evaluate.py --network <Path to Trained Network> --dataset cornell --dataset-path <Path to Dataset> --iou-eval

Grasp Visualization

python evaluate.py --network <Path to Trained Network> --dataset cornell --dataset-path <Path to Dataset> --iou-eval --vis