Eye Gaze for Monitoring Attention Through Hybrid Ensemble Learning
Main Article Content
Abstract
One of the countless tasks that call attention to monitoring is necessary for comprising healthcare, education, transportation safety, and human-computer interaction. This research describes novel work done in attention monitoring by fusing a hybrid eye gaze model with deep learning to monitor a driver's attention level. The hybrid eye gaze model proposed is described and its results are produced in this paper. The proposed model uses an augmented dataset where data augmentation techniques like rotation, shifting, shearing, and flipping are applied together with adjustments like changing the fill mode in terms of zooming into the image and rescaling. These are all crucial aspects in reliable and consistent training of the model. Our model is built on modern pre-trained architectures which include VGG16, VGG19, InceptionV3, EfficientNetB0, EfficientNetB7, and InceptionResNetV2. To aid in capturing very minute attention dynamics, we modify these architectures and then incorporate more layers. Later, we used a model ensemble to increase the accuracy and efficiency of the model. Later, the XGBoost model is integrated with all other models used before in the hybrid model technique to obtain better accuracy and efficiency of the model. The model performance is adequately evaluated using various evaluation measures like accuracy, precision, recall, F1 Score, and support. These metrics provide a holistic understanding of the model's capability to detect and predict attention patterns in different contexts. After using the models, we could get the best accuracy from VGG19 and InceptionResNetV2, i.e., 84.6% and 83.6% respectively. VGG16 hybrid models recorded 82% in the accuracy test. With deep learning and pre-trained architectures, the Hybrid Eye Gaze Model shows a strong and flexible attention monitoring solution for varying types of applications.
Article Details
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
References
Jegham and others, “Deep learning-based hard spatial attention for driver in-vehicle action monitoring,” Expert Syst Appl, vol. 219, p. 119629, 2023.
Z. Trabelsi and others, “Real-Time Attention Monitoring System for Classroom: A Deep Learning Approach for Student’s Behavior Recognition,” Big Data and Cognitive Computing, vol. 7, no. 1, p. 48, 2023.
X. Lei and others, “Mutual information based anomaly detection of monitoring data with attention mechanism and residual learning,” Mech Syst Signal Process, vol. 182, p. 109607, 2023.
R. V. Bidwe, S. Mishra, and S. Bajaj, “Performance evaluation of Transfer Learning models for ASD prediction using non-clinical analysis,” in Proceedings of the 2023 Fifteenth International Conference on Contemporary Computing, New York, NY, USA: ACM, Aug. 2023, pp. 474–483. doi: 10.1145/3607947.3608050.
M. Cheng and others, “Intelligent tool wear monitoring and multi-step prediction based on deep learning model,” J Manuf Syst, vol. 62, pp. 286–300, 2022.
G. Wang and F. Zhang, “A sequence-to-sequence model with attention and monotonicity loss for tool wear monitoring and prediction,” IEEE Trans Instrum Meas, vol. 70, pp. 1–11, 2021.
L. Li and others, “Monitoring and prediction of dust concentration in an open-pit mine using a deep-learning algorithm,” J Environ Health Sci Eng, vol. 19, pp. 401–414, 2021.
S. A. R. ˙I. Meriem, A. Moussaoui, and A. Hadid, “Automated facial expression recognition using deep learning techniques: an overview,” International Journal of Informatics and Applied Mathematics, vol. 3, no. 1, pp. 39–53, 2020.
K. Kim and J. Jeong, “Real-time monitoring for hydraulic states based on convolutional bidirectional LSTM with attention mechanism,” Sensors, vol. 20, no. 24, p. 7099, 2020.
B. Brousseau, J. Rose, and M. Eizenman, “Hybrid eyetracking on a smartphone with CNN feature extraction and an infrared 3D model,” Sensors, vol. 20, no. 2, p. 543, 2020.
S. Bursic and others, “Improving the accuracy of automatic facial expression recognition in speaking subjects with deep learning,” Applied Sciences, vol. 10, no. 11, p. 4002, 2020.
S. S. Roy, M. Ahmed, and M. A. H. Akhand, “Noisy image classification using hybrid deep learning methods,” Journal of Information and Communication Technology, vol. 17, no. 2, pp. 233–269, 2018.
Fathallah, L. Abdi, and A. Douik, “Facial expression recognition via deep learning,” in 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), 2017.
S. Kim, “PDE-based image restoration: A hybrid model and color image denoising,” IEEE Transactions on Image Processing, vol. 15, no. 5, pp. 1163–1170, 2006.
M. S. Bartlett and others, “Recognizing facial expression: machine learning and application to spontaneous behavior,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2005.
R. V Bidwe and others, “Deep Learning Approaches for Video Compression: A Bibliometric Analysis,” Big Data and Cognitive Computing, vol. 6, no. 2, p. 44, Apr. 2022, doi: 10.3390/bdcc6020044.
D. Mane, K. Shah, R. Solapure, R. Bidwe, and S. Shah, “Image-Based Plant Seedling Classification Using Ensemble Learning,” 2023, pp. 433–447. doi: 10.1007/978-981-19-2225-1_39.
S. Nalwar et al., “EffResUNet: Encoder Decoder Architecture for Cloud-Type Segmentation,” Big Data and Cognitive Computing, vol. 6, no. 4, p. 150, Dec. 2022, doi: 10.3390/bdcc6040150.
D. Mane, R. Bidwe, B. Zope, and N. Ranjan, “Traffic Density Classification for Multiclass Vehicles Using Customized Convolutional Neural Network for Smart City,” 2022, pp. 1015–1030. doi: 10.1007/978-981-19-2130-8_78.
G. Agrawal, U. Jha, and R. Bidwe, “Automatic Facial Expression Recognition using Advanced Transfer Learning,” in Proceedings of the 2023 Fifteenth International Conference on Contemporary Computing, New York, NY, USA: ACM, Aug. 2023, pp. 450–458. doi: 10.1145/3607947.3608047.