Dataset and Model Traning

[anhtuduong]

1. Data collection and annotation:

The first step in object detection is to collect a dataset of images containing blocks in various poses and lighting conditions. Use Blender to generate synthetic data and ZED camera to capture real images.

• (Synthetic) Data collection and annotation Blender_lego_dataset_generator#1
• (ZED camera) Data collection and annotation #5

2. Dataset splitting:

After the dataset is annotated, that will be split it into training, validation, and testing sets. The training set will be used to train the YOLOv5 model, the validation set will be used to monitor the training progress and adjust the hyperparameters, and the testing set will be used to evaluate the performance of the model on new data.

3. Data preprocessing:

Before training the model, preprocess the images and annotations. This may involve resizing the images, normalizing the pixel values, and converting the annotations to the YOLOv5 format.

4. Model training:

With the preprocessed dataset, train the YOLOv5 model using a deep learning framework like PyTorch. Can start with a pre-trained model and fine-tune it on the LEGO block dataset using transfer learning. Will need to adjust the hyperparameters of the model, such as the learning rate and batch size, to achieve the best performance.

5. Model evaluation:

Once the model is trained, evaluate its performance on the testing set. Can use metrics such as precision, recall, and F1 score to measure how well the model can detect LEGO blocks. If the performance is not satisfactory, go back to the previous steps to adjust the data collection, preprocessing, or model training.

6. Model deployment:

After the model is trained and evaluated, deploy it in the UR5BlokVision ROS package. The package will use the YOLOv5 model to detect LEGO blocks in the images acquired by the ZED camera, and output the bounding boxes and class probabilities for each detected block.

[anhtuduong]

Training log

18/10/23: yolov8L 300 epochs

yolov8L 600 epochs goal:

14/11/23:

• anhtu.duong: 3
• anhtu471996: 13

01/12/23:

• totodatrento: 600