Arabic Speech Emotion Recognition using Convolutional Neural Networks

Abstract

Emotions are considered an essential and fundamental aspect of human conversations. It serves as a means for opinion expression and for enlightening others about their psychological and physical well-being. Therefore, extracting the speaker's emotional state has become an active research topic lately due to the demand for more human interactive applications. This field of research has noted significant advancement, especially in the English language, owing to the availability of massive speech-labeled corpora. However, the progress of analogous methodologies in the Arabic language is still in its infancy stages. In this paper, we present an effective Arabic Speech Recognition model, proficient in discerning both the emotional state and gender of the speaker through voice analysis. The model is trained to recognize six primary emotions: tiredness, sadness, anger, neutrality, happiness, and joy. For dataset preprocessing and feature extraction, various spectral features, such as the Mel-frequency Cepstral coefficient (MFCC), were extracted and tested to determine the optimal feature combination. For Classifier selection, Three Machine Learning models (SVM, KNN, and HMM) and two Deep Learning models (LSTM and CNN) were evaluated for training. The experimental results were analyzed and compared across the five models using various performance measures. This evaluation aimed to select the optimal model capable of performing well under different conditions, including noisy environments. The best results are achieved by the Combination of MFCC, root-mean-square (RMS), mel-scaled spectrogram, Spectral feature, and zero-crossing rate as spectral features, and the CNN as a classification model. This selection yielded significant results outperforming the models in the State of the Art, with a Recognition accuracy of 93% for emotion recognition and 99% for gender recognition.