How to train a custom dataset with YOLOv7 for instance segmentation?

 

Another post starts with you beautiful people! It is overwhelming for me to see massive interest in my last three posts about the YOLOv7 series💓. Your response keeps me motivated to share my learning with you all 💝. If you have not checked my previous posts about YOLOv7, then I am sharing here links to read those once and then proceed with this post-

1. Train a custom dataset with YOLOv7
   
2. Export custom YOLOv7 model to ONNX
   
3. Export custom YOLOv7 model to TensorRT
   

Till now we have learned about object detection with YOLOv7. In this post, we are going to learn how can we train a custom dataset for instance segmentation task with YOLOv7 👌. For your information instance segmentation is the task of detecting and delineating each distinct object of interest appearing in an image. For our hands-on we need a dataset having images and their annotations in polygon format and of course in YOLO format. So I have found and downloaded the American Sign Language dataset in the required format from this link.

This dataset has all letters A through Z in American Sign Language labeled with polygon labels. To our knowledge, the alphabet and its correspondence signs look like below-

For this post also I am using the google colab environment. Let's first install its required library and restart the colab notebook runtime once it is installed-

Next, we will download the code repo of the YOLOv7's segmentation branch since while writing this post author has not officially declared the main branch for the segmentation code. So I have downloaded the u7 branch and uploaded it to my gdrive so that I can access it in my colab notebook 😀. After uploading the branch code to drive, I unzipped it like as below-

The unzipped folder structure of the dataset will look like as below-

Next, we will prepare the required YAML files for our instance segmentation training. First, we will create our configuration YAML file where will define the path of the training and validation images, the number of the target classes, and the names of the target classes as below-

Here I have provided the train and validation images path, and the number of classes as 26 since our classes are from the alphabet A-Z. Next, we will create a second YAML file having the content of it's default yolov7-seg.yaml but with one difference- the value of 'nc' parameter from 80 to 26-

Next, we will create the hyperparameter YAML file having the same content as its official hyp.scratch-high.yaml as below-

Next, we will download the pre-trained weight file for the training as below-

Now we are ready to start our instance segmentation training with the below command-

Here, change the .yaml files paths and the weight file path as per yours. In my colab environment for 100 epochs it has taken 1 hour to complete. for better accuracy, you should train it for 300 epochs for this dataset. Once the training is completed, you will see the following output in the console-

After completion of the training, let's check the trained model on a test image as below-

You will see the following output in the console after successfully running the above command-

Let's open the processed image and see what it looks like-

See our trained model is able to do the instance segmentation and is able to predict the target class of a sign 💥. Let me share a few more output images-

Indeed we have a strong sign language detection model. Let's run the inference on a video-

The output in the console will look like the below after a successful run of the above command-

Let's open this video in our colab notebook-

And it will show the processed video file as below-

That's it for today's guys! We have learned another useful technique with YOLOv7. Here I am sharing my colab notebook for this tutorial. So no need to rest, just copy the notebook into your colab environment and try it with your own dataset. In my next post I will share something useful again till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇