ARCHIVES

Emotion recognition through facial expressions has been a key parameter in the development of human-computer interaction as a technology with some useful results in terms of emotional states in real time. This study presents a hybrid approach for emotion recognition by combining deep learning methods and traditional feature extracting methods to achieve high accuracy and computational efficiency for emotion recognition. The proposed system makes use of the MobileNetV2 architecture for deep feature extraction, which is known to be lightweight, hence suitable for real-time applications. Moreover, fine-grained texture features are abstracted from facial regions through Local Binary Pattern (LBP) for capturing texture details at the fine grain-level, which are complementary to the deep representations and improve the model's ability to distinguish the subtle emotional states. These two sets of features are concatenated and fed to a Random Forest (RF) classifier, that is known to be robust and efficient in the context of moderately sized datasets, in order to classify the emotions using categories such as happy, sad, anger, surprise and neutral. The system is trained and tested with the help of a benchmark facial emotion dataset, incorporating pre-processing techniques of face detection, alignment, normalization and data augmentation for better generalization. The performance of the system is assessed using various metrics such as accuracy, precision, recall, F1-score, confusion matrices with measurements of latency to be able to provide real-time feasibility. Experimental results show that the performance of MobileNetV2 + LBP + RF is higher than that of MobileNetV2 + SoftMax head in the case of distinguishing subtle emotions. The system is implemented in the form of a Flask-based web application that is combined with the OpenCV library for live webcam streaming with minimal latency and user privacy through on-device inference.