The Role of detection Algorithm in Tracking Techniques with the help of Deep Learning for an Enhanced user Experience

Article Sidebar

Published: Mar 28, 2024
DOI: https://doi.org/10.52783/jes.844
Keywords:
OpenCV, YOLOv5, CNN, gesture recognition, object detection

Main Article Content

Pallavi Parlewar, Sheetal Dhande, Smita Rathod, Chetan Laddha, Ishant Kohad, Harsh Salunke

Abstract

The drum set is a prominent instrument in a variety of music genres including pop, rock, and jazz. However, it costs money and space to get from nothing to purchasing a kit. The objective of these studies is to enable drummers to hone their skills, at least blithely, without a full set of drums and to expedite the initiation of drummers to their drumming experiences without incurring any costs. Drums do not fit inside a single bag, making them more difficult to travel than other instruments. We propose a virtual drum prototype that enables air-drumming using just two sticks, a computer equipped with a camera, and readily available markers that resemble the drumstick tips, such colourful papers. The detection algorithm combines tracking techniques with deep learning technology for an enhanced user experience. This was done using the Python-based OpenCV, as well as the concept of color-based blob recognition was used to find the markers. The capacity to function as a USB controller and MIDI controllers as well as the possibility for further development as a released program have all been shown by this prototype. Results from experiments show how useful and successful this method is for producing a lifelike and immersive virtual drumming experience.

Article Details

Issue
Vol. 20 No. 1s (2024)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

Author Biography

Pallavi Parlewar, Sheetal Dhande, Smita Rathod, Chetan Laddha, Ishant Kohad, Harsh Salunke

[1]Pallavi Parlewar

2Sheetal Dhande

3Smita Rathod

4Chetan Laddha

5Ishant Kohad

6Harsh Salunke

 

[1] 1Associate Professor, Department of Electronics and Communication, Shri Ramdeobaaba College of Engineering and Management, Nagpur, India

2Professor, Department of Computer Science and Engineering, Sipna College of Engineering and Technology, Amravati, India,

3 Assistant Professor, Computer science & Engineering, Shri Guru Gobind Singhji Institute of Engineering and Technology (SGGSIE&T) Nanded, India

4,5,6Student, Department of Electronics and Communication, Shri Ramdeobaaba College of Engineering and Management, Nagpur, India  

Corresponding author E mail : parlewarpk@rknec.edu

 

References

Carl Timothy Tolentino, Agatha Uy, Prospero Naval. "Air Drums: Playing Drums Using Computer Vision", 2019 International Symposium on Multimedia and Communication Technology (ISMAC), 2019

Sumathi. S, D. Vijila, M. Shastika. "Air xylophone Using OpenCV", 2022 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), 2022

K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904–1916, 2015.

Harel Yadid, Almog Algranti, Mark Levin, Ayal Taitler. "A2D: Anywhere Anytime Drumming", 2023 IEEE Region 10 Symposium (TENSYMP), 2023

S. R. Fidellaga Bering and S. M. W. Famador, "Virtual Drum Simulator Using Computer Vision," in The 4th IIAE International Conference on Intelligent Systems and Image Processing 2016, January 2016.

Huairong CHEN, Xi WANG, Haonan WANG, Nannan GU, Meiyin ZHU, Shubo YANG. "Inverted decoupling and LMI-based controller design for a turboprop engine with actuator dynamics", Chinese Journal of Aeronautics, 2020

S. R. F. Bering and S. M. W. Famador, "Virtual Drum Simulator Using Computer Vision," pp. 370–375, 2017.

Bizhe Bai, Jie Tian, Sicong Luo, Tao Wang, Sisuo LYU. "YUSEG: Yolo and Unet is all you need for cell instance segmentation", 2022 Conference on Neural Information Processing Systems, 2022

Liguo Zhou, Guang Chen, Lian Liu, Ruining Wang, Alois Knoll. "Real-time semantic segmentation in traffic scene using Cross Stage Partial-based encoder–decoder network", Engineering Applications of Artificial Intelligence, 2023

F. A. Fiolana, F. Yumono, and M. Y. A. Anggara, "VIRTUAL DRUM BERBASIS SISTEM PENGOLAH CITRA DIGITAL," JTECS : Jurnal Sistem Telekomunikasi Elektronika Sistem Kontrol Power Sistem & Komputer, vol. 1, no. 1, pp. 59, January 2021

G. Jocher, "ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation," Nov 2020.

J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.

C. Y. Wang, A. Bochkovskiy, and H. Y. M. Liao, "Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," arXiv preprint arXiv:2207.02696, 2022.

C. T. Tolentino, A. Uy, and P. Naval, "Air drums: Playing drums using computer vision," in 2019 International Symposium on Multimedia and Communication Technology (ISMAC), IEEE, 2019, pp. 1–6.

V. U. Rathod and S. V. Gumaste, “Role of Deep Learning in Mobile Ad-hoc Networks”, IJRITCC, vol. 10, no. 2s, pp. 237–246, Dec. 2022.

P. K. Parlewar et.al, “A Smart Ai Based Personal Assistant To Support Covid-19 Patient”, International Journal of Mechanical Engineering, vol. 6, Iss. 3, pp 2834-2836, ISSN: 0974-5823

Xin Wang, Ning He, Chen Hong, Qi Wang, Ming Chen. "Improved YOLOX-X based UAV aerial photography object detection algorithm", Image and Vision Computing, 2023

N. P. Sable, V. U. Rathod, P. N. Mahalle, and P. N. Railkar, “An Efficient and Reliable Data Transmission Service using Network Coding Algorithms in Peer-to-Peer Network”, IJRITCC, vol. 10, no. 1s, pp. 144–154, Dec. 2022.

M. Tan, R. Pang, and Q. V. Le, "Efficientdet: Scalable and efficient object detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10781–10790.

N. P. Sable, R. Sonkamble, V. U. Rathod, S. Shirke, J. Y. Deshmukh, and G. T. Chavan, “Web3 Chain Authentication and Authorization Security Standard (CAA)”, IJRITCC, vol. 11, no. 5, pp. 70–76, May 2023.

Vijay U. Rathod* & Shyamrao V. Gumaste, “Effect Of Deep Channel To Improve Performance On Mobile Ad-Hoc Networks”, J. Optoelectron. Laser, vol. 41, no. 7, pp. 754–756, Jul. 2022.

S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path aggregation network for instance segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8759–8768.

"Pocketdrum 2 pro : Play and practice the drums anywhere you go," [Online]. Available: https://www.aeroband.net/, 2016.

Vijay U. Rathod, Yogesh Mali, Nilesh Sable, Deepika Ajalkar, M. Bharathi, and N. Padmaja,” A Network-Centred Optimization Technique for Operative Target Selection”, Journal of Electrical Systems (JEs), vol. 19, no. 2s, pp. 87–96, 2023.

N. P. Sable, V. U. Rathod, M. D. Salunke, H. B. Jadhav, R. S. Tambe, and S. R. Kothavle, “Enhancing Routing Performance in Software-Defined Wireless Sensor Networks through Reinforcement Learning”, International Journal of Intelligent Systems and Applications in Engineering (IJISAE), vol. 11, no. 10s, pp. 73–83, Aug. 2023.

Rathod, V.U. and Gumaste, S.V., 2022. Role of Neural Network in Mobile Ad Hoc Networks for Mobility Prediction. International Journal of Communication Networks and Information Security, 14(1s), pp.153-166.

M. F. Kholid et al., "Human Movement Detection Dengan Accumulative Differences Image," Jurnal TEKNIMEDIA, vol. 1, no. 1, pp. [page range], May 2020.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.

Kumar, S. ., Parlewar, P. ., & Jagtap, V. . (2023). Design of an Ensemble learning Model to Recognize Multilingual Hand Gestures via Interlinked Key Point analysis. International Journal of Intelligent Systems and Applications in Engineering, 11(10s), 103–112. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/3238

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017

Nilesh P. Sable, Vijay U. Rathod, Parikshit N. Mahalle, Jayashri Bagade, Rajesh Phursule ; Internet of Things-based Smart Sensing Mechanism for Mining Applications, Industry 4.0 Convergence with AI, IoT, Big Data and Cloud Computing: Fundamentals, Challenges and Applications IoT and Big Data Analytics (2023) 4: 132. https://doi.org/10.2174/9789815179187123040012.

Y. Mali, Vijay U. Rathod “A Comparative Analysis of Machine Learning Models for Soil Health Prediction and Crop Selection”, International Journal of Intelligent Systems and Applications in Engineering (IJISAE), vol. 11, no. 10s, pp. 811–828, Aug. 2023.

Parlewar, P. ., Jagtap, V. ., Pujeri, U. ., Kulkarni, M. M. S. ., Shirkande, S. T. ., & Tripathi, A. . (2023). An Efficient Low-Loss Data Transmission Model for Noisy Networks. International Journal of Intelligent Systems and Applications in Engineering, 11(9s), 267–276.Retrieved from https://www.ijisae.org/index.php/IJISAE/article/view/3117

Kamble, V. M., Parlewar, P., Keskar, A. G., & Bhurchandi, K. M. (2016). Performance evaluation of wavelet, ridgelet, curvelet and contourlet transforms based techniques for digital image denoising. Artificial Intelligence Review, 45, 509-533

Parlewar, P.K., Bhurchandi, K.M. A 4-quadrant Curvelet Transform for Denoising Digital Images. Int. J. Autom. Comput. 10, 217–226 (2013). https://doi.org/10.1007/s11633-013-0715-z

Sable, N.P., Rathod, V.U. (2023). Rethinking Blockchain and Machine Learning for Resource-Constrained WSN. In: Neustein, A., Mahalle, P.N., Joshi, P., Shinde, G.R. (eds) AI, IoT, Big Data and Cloud Computing for Industry 4.0. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-031-29713-7_17.