Enhancing Recognition of Indian Sign Language via Fusion of SURF-based SVM and CNN Models for Seamless Integration into a Regional Language Translation System

Abstract

Efficient communication is fundamental for human interaction however, people with speech and hearing impairments frequently find it difficult to communicate with others without the help of a translation. For the deaf and mute people, sign language is an essential form of nonverbal communication, highlighting the necessity of having a system that can understand and recognize sign language.  In order to enable seamless communication for those with hearing and speech impairments, this research presents a novel method for the automatic identification of finger typing in Indian sign language. The suggested framework processes input signs, employs skin colour-based segmentation, and utilizes various image processing techniques to detect and transform sign shapes and there is a voice module which will convert the sign language into speech in respective regional language. After that, the discovered region is converted into a binary image. The binary picture that is produced is then subjected to the Euclidean distance transformation. On the image that has been modified by distance, row and column projection is used. Central moments are utilized in conjunction with HU's moments for feature extraction. Neural networks and SVM are used for classification.  As for India is considered there are so many languages which are spoken all over the country. When the same sign language is used the interpreter finds it difficult to understand so we provide a system when ever the sign language is used it can take up any type of languages which are being used. Revolutionizing Communication for Speech-Impaired Individuals: A Comprehensive Framework Recognizing Indian Sign Language with BOVW, SURF, SVM, and CNN. Integration with Real-time Video Stream Analysis and Multimodal Output is being generated as voice.