YOLObile- a new state of the art Real-Time Object Detection model for Mobile Devices

 

Another post starts with you beautiful people!

Thanks for giving so many views on my previous post 👍.

I am glad to see my previous posts are helping people to use state of the art object detection and recognition deep learning model in their projects. If you are new to my blog, I recommend seeing once my previous posts, and you will not be disappointed if your goal is to learn applied computer vision free of cost. Continuing my journey of sharing knowledge in this post I am going to share with you a new state of the art framework for object detection on mobile devices- YOLObile 📱

There has been a trade-off between speed and the accuracy of object detections. For example, the state of the art, YOLOv4 model gives us a very accurate detection but its speed is slow if we want to use it on a mobile device. On the other hand, its lighter version YOLOv4-tiny works very fast on a mobile device but its accuracy reduces. For a detailed comparison of FPS vs mAP you can refer to the above diagram. Nowadays computer vision techniques are improving day by day due to hardware advancement and so using AI in mobile devices is a hot choice for every mobile app.

However, the current state-of-the-art object detection works are either accuracy-oriented using a large model but leading to high latency or speed-oriented using a lightweight model but sacrificing accuracy. To solve this problem respected authors [Yuxuan Cai, Hongjia Li, Geng Yuan, Wei Niu, Yanyu Li, Xulong Tang, Bin Ren, Yanzhi Wang] of this paper have introduced the YOLObile framework, a real-time object detection on mobile devices via compression-compilation co-design. 💥

As per the authors- A novel block-punched pruning scheme is proposed for any kernel size. To improve computational efficiency on mobile devices, a GPU-CPU collaborative scheme is adopted along with advanced compiler-assisted optimizations. Experimental results indicate that our pruning scheme achieves a 14x compression rate of YOLOv4 with 49.0 mAP. Under our YOLObile framework, we achieve 17 FPS inference speed using GPU on Samsung Galaxy S20. By incorporating our proposed GPU-CPU collaborative scheme, the inference speed is increased to 19.1 FPS and outperforms the original YOLOv4 by 5x speedup.

That's quite a solid approach right 💪. If I summarize their approach then I would say that their approach is basically unstructured weights pruning, with a bit of structure on top, where they group the weights in blocks of 4 channels and 8 filters (x32). This structure allows them to improve parallelism. One important point to know here is that unstructured sparsity is very difficult to parallelize compare to structured sparsity also it is hardware-specific.On the other hand, up to 93% of reduction in weight size without a real loss in accuracy is really tempting with this framework. 💝

I know after reading this all of you want to see the result of this new approach on any benchmark dataset💥 and the good news is that authors have recently released the weights (in PyTorch format) which you can download from this link 

From the download link, you will get two weights files of 245 MB size: best8x-514.pt and best14x-49.pt. These two weights are the output of the 8x and 14x pruning models. According to the authors- these models are trained on a server with eight NVIDIA RTX 2080Ti GPUs. The training methods are implemented using PyTorch API. The details of both weights are as below:-

I tested the MAP on coco 2017 dataset (80 classes) with image-size of 320, batch-size of 8, best8x-514 weights in my Windows 10 laptop having 6GB of NVIDIA GTX 1060 and results are as below-

For evaluating this framework, the authors used an off-the-shelf Samsung Galaxy S20 smart-phone, which has a Qualcomm Snapdragon 865 Octa-core CPU  and  a  Qualcomm  Adreno  650  GPU.  Each test ran on  50  different input frames  (images),  with the average speed results reported. The YOLObile is derived based onYOLOv4, with 320×320 input size, and train on the MS COCO dataset. They denote mAP as the Average Precision under IoU 0.5 threshold and AP@[.5:.95] as the Aver-age Precision under IoU from 0.5 to 0.95-

The above evaluation result is quite amazing right💥. A real-time object detection framework on mobile devices via compression-compilation co-design will definitely going to be a game-changer. I have uploaded the inference code in my github repository You can follow the same to test the new framework in your own system. Since this framework is quite new there are no open-source code is available for the training on a custom dataset but as soon as I found any one of the Darknet or TensorFlow implementation, I will share with you all. Till then Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇