Video Analytics Dashboard for YoloV5 and DeepSort
This blog is about a small scale video analytics tool which I built recently using Yolov5, DeepSort and Streamlit. In this blog, I will walk you through the features of the dashboard, show some demos and also discuss future scope.
Motive
As AI engineers, we love data and we love to see graphs and numbers! So why not project the inference data on some platform to understand the inference better? When a model is deployed on the edge for some kind of monitoring, it takes up rigorous amount of frontend and backend developement apart from deep learning efforts — from getting the live data to displaying the correct output. So, I wanted to replicate a small scale video analytics tool and understand what all feature would be useful for such a tool and what could be the limitations?
The best part is, we can plug in any custom trained model, be it trained to detect animals, urban city objects, chimenys in factories or different space junks! This dashboard is not domain specific and hence can also be used for testing your custom trained models! So instead of scrolling through the logs in the terminal, you use this dashboard for the same! Further on, this dashboard can be modifed to suit any given use case, be it traffic monitoring, model testing tool or monitoring of aquatic animals!
Project Components
- Inference Configuration
- Inference
- Inference stats
- System stats
- Inference overview
Inference Configuration
This component deals with the entire settings of the tool, giving you several options for input source(can be local video, rtsp or webcam), class confidence value, threshold for drift detection, threhold for fps drop warning,saving output videos and saving frames on which the model is not performing well.
Inference
This inference which you see is output of YoloV5s version 6 with DeepSort to track the objects. We have label and then class confidence value. In the box we have the id for the object. For example, here we have a car with id 2.
Check references for links.
Inference stats
Inference stats shows the FPS, detected objects in current frame and total objects detected so far (not tracked).
These stats can help in multiple way — such as,
The tool will alert when FPS falls below given threshold. This can help keep an eye on the performance of system to decide whether or not to upgrade the hardware or optimise model. It may happen that with time, number of objects are slowly increasing and the model is becoming slow in detections and hence would lead to drop in FPS. It may also happen that, during certain time of the day when there is usual surge in number of objects the FPS reduces indicating optimizing the model or upgrading the hardware being used.
As as add-on, we can store number of vehicles being driven on certain road for given time duration. For eg, from 6AM to 8AM there is heavy traffic and hence requires the person monitoring the traffic to be vigilant. Then after 8AM, the traffic reduces. This data can also be sent to local body to help them schedule road repairs.
Total detected objects data can help when we are collecting data for retraining the model as well. This count can help us prevent class imbalance. For eg, we want to retrain our model due to data drift (Government has added new electric buses for which model is getting confused with small vans hence leading to change in data being infered on), so this count can help us plan our training process to prevent overfitting on one single class. Also, for eg, if during the entire day there were 100 buses on the road being driven along side the cars, then the transport body can think of providing a seperate lane for buses to help regulate traffic.
System stats
System stats help monitor CPU, RAM and GPU usage. It helps keep an eye on the hardware if it’s being under or over utilized. One can decide on upgrading the GPU if GPU utilization peaks frequently when there is a sudden surge of objects in the camera. Similarly this is applicable to memory and CPU as well.
Inference overview
In this section, we get summary of the inference, for example -> classes on which the model is not performing well, number of frames for which atleast one object was detected with confidence less than threshold value, minimum and maximum FPS.
Demo
The following video is a demo of the dashboard using a video from local device.
Epilogue
This was a small walk through of the tool and this can be modified to something outstanding. We can add different types of graphs and stats from the device or inference. We can also use optimized model which will inturn give higher FPS.
One can do futher research to understand model and data drift. Here, I just tried to implement the definition of data drift. Drift detection in production can help in prior upgradation of the model.
Thankyou for reading my blog :). Hope you learnt something new!
For more such blogs, do follow me on Medium. I post blogs every month. Lets connect on LinkedIn as well :)
Code
Here is the code. If you like the project do give it a star on GitHub. :).
GitHub — Link
(Also, I am running this project locally, having 4Gb Nvidia 1650 and 16Gb ram. I’ll optimize the code going forward :) )
References
[1] YoloV5 by Ultralytics —Link
[2] DeepSort — Link
[3] Streamlit — Documentation
[4] Input video source— Video
