Sensing Human Emotion using Emerging Machine Learning Techniques

Dileep Kumar Gupta; Prof. (Dr.) Devendra Agarwal; Dr. Yusuf Perwej; Opinder Vishwakarma; Priya Mishra; Nitya

doi:10.32628/IJSRSET24114104

Authors

Dileep Kumar Gupta Assistant Professor, Department of Computer Science & Engineering, Goel Institute of Technology & Management, Lucknow, U.P, India Author
Prof. (Dr.) Devendra Agarwal Dean (Academics), Goel Institute of Technology & Management, Lucknow, U.P, India Author
Dr. Yusuf Perwej Professor, Department of Computer Science & Engineering, Goel Institute of Technology & Management, Lucknow, U.P, India Author
Opinder Vishwakarma Scholar (B.Tech Final Year) Computer Science & Engineering, Goel Institute of Technology & Management, Lucknow, U.P, India Author
Priya Mishra Scholar (B.Tech Final Year) Computer Science & Engineering, Goel Institute of Technology & Management, Lucknow, U.P, India Author
Nitya School of Studies in Mathematics Vikram University, Ujjain, Maharashtra, India Author

DOI:

https://doi.org/10.32628/IJSRSET24114104

Keywords:

Emotion Detection, Convolution Neural Network, Face Expression, Pre-processing, Classification, Machine Learning

Abstract

Human emotion recognition using machine learning is a new field that has the potential to improve user experience, lower crime, and target advertising. The ability of today's emotion detection systems to identify human emotions is essential. Applications ranging from security cameras to emotion detection are readily accessible. Machine learning-based emotion detection recognises and deciphers human emotions from text and visual data. In this study, we use convolutional neural networks and natural language processing approaches to create and assess models for emotion detection. Instead of speaking clearly, these human face expressions visually communicate a lot of information. Recognising facial expressions is important for human-machine interaction. Applications for automatic facial expression recognition systems are numerous and include, but are not limited to, comprehending human conduct, identifying mental health issues, and creating artificial human emotions. It is still difficult for computers to recognise facial expressions with a high recognition rate. Geometry and appearance-based methods are two widely used approaches for automatic FER systems in the literature. Pre-processing, face detection, feature extraction, and expression classification are the four steps that typically make up facial expression recognition. The goal of this research is to recognise the seven main human emotions anger, disgust, fear, happiness, sadness, surprise, and neutrality using a variety of deep learning techniques (convolutional neural networks).

Downloads

Download data is not yet available.

References

Huang, D.; Guan, C.; Ang, K.K.; Zhang, H.; Pan, Y. Asymmetric spatial pattern for EEG-based emotion detection. In Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia, 10–15 June 2012; pp. 1–7. 2 DOI: https://doi.org/10.1109/IJCNN.2012.6252390

Y. Perwej, “Unsupervised Feature Learning for Text Pattern Analysis with Emotional Data Collection: A Novel System for Big Data Analytics”, IEEE International Conference on Advanced computing Technologies & Applications (ICACTA'22), SCOPUS, IEEE No: #54488 ISBN No Xplore: 978-1-6654-9515-8, Coimbatore, India, 4-5 March 2022, DOI:10.1109/ICACTA54488.2022.9753501 DOI: https://doi.org/10.1109/ICACTA54488.2022.9753501

Cui, Y.; Wang, S.; Zhao, R. Machine learning-based student emotion recognition for business English class. Int. J. Emerg. Technol. Learn. 2021, 16, 94–107. DOI: https://doi.org/10.3991/ijet.v16i12.23313

K. Tai, "The application of digital image processing technology in glass bottle crack detection system[J]", Acta Technica CSAV (Ceskoslovensk Akademie Ved), vol. 62, no. 1, pp. 381-390, 2017

Saurabh Sahu, Km Divya, Dr. Neeta Rastogi, Puneet Kumar Yadav, Y. Perwej, “Sentimental Analysis on Web Scraping Using Machine Learning Method” , Journal of Information and Computational Science (JOICS), ISSN: 1548-7741, Volume 12, Issue 8, Pages 24-29, 2022, DOI: 10.12733/JICS.2022/V12I08.535569.67004

J Chen, X Yao, Huang Fen∗ et al., "N status monitoring model in winter wheat based on image processing[J]", Transactions of the Chinese Society of Agricultural Engineering, vol. 32, no. 4, pp. 163-170, 2016

Dawar Husain, Y. Perwej, Satendra Kumar Vishwakarma, Prof. (Dr.) Shishir Rastogi, Vaishali Singh, N. Akhtar, “Implementation and Statistical Analysis of De-noising Techniques for Standard Image”, International Journal of Multidisciplinary Education Research (IJMER), ISSN:2277-7881, Volume 11, Issue10 (4), Pages 69-78, 2022, DOI: 10.IJMER/2022/11.10.72

Schoneveld, L.; Othmani, A.; Abdelkawy, H. Leveraging recent advances in deep learning for audio-visual emotion recognition. Pattern Recognit. Lett. 2021, 146, 1–7 DOI: https://doi.org/10.1016/j.patrec.2021.03.007

Sun, Q.; Liang, L.; Dang, X.; Chen, Y. Deep learning-based dimensional emotion recognition combining the attention mechanism and global second-order feature representations. Comput. Electr. Eng. 2022, 104, 108469

Y. Perwej, F. Parwej, A. Perwej, “Copyright Protection of Digital Images Using Robust Watermarking Based on Joint DLT and DWT”, International Journal of Scientific & Engineering Research (IJSER), France, ISSN 2229-5518, Volume 3, Issue 6, Pages 1- 9, June 2012

Y. Perwej, “An Evaluation of Deep Learning Miniature Concerning in Soft Computing”, International Journal of Advanced Research in Computer and Communication Engineering (IJARCCE), ISSN (Online): 2278-1021, Volume 4, Issue 2, Pages 10 - 16, 2015, DOI: 10.17148/IJARCCE.2015.4203 DOI: https://doi.org/10.17148/IJARCCE.2015.4203

Kajal, Neha Singh, N. Akhtar, Ms. Sana Rabbani, Y.f Perwej, Susheel Kumar, “Using Emerging Deep Convolutional Neural Networks (DCNN) Learning Techniques for Detecting Phony News”, International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN: 2456-3307, Volume 10, Issue 1, Pages 122-137, 2024, DOI: 10.32628/CSEIT2410113 DOI: https://doi.org/10.32628/CSEIT2410113

Q. Fan, L. Brown and J. Smith, "A closer look at Faster R-CNN for vehicle detection", 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 124-129, 2016 DOI: https://doi.org/10.1109/IVS.2016.7535375

Cao, Y. Ma, X. Meng, Y. Gao and M. Meng, "Emotion Recognition Based On CNN," 2019 Chinese Control Conference (CCC), Guangzhou, China, 2019, pp.8627-8630. doi:10.23919/ChiCC.2019.8866540. DOI: https://doi.org/10.23919/ChiCC.2019.8866540

Shrey modi et al. " Facial Emotion Recognition using Convolution Neural Network Proceedings of the Fifth International Conference on Intelligent Computing and Control Systems (ICICCS 2021) IEEE Xplore Part Number: CFP21K74- ART; ISBN: 978-0-7381-1327

Kamble,K.S.; Sengupta, J. Ensemble machine learning-based affective computing for emotion recognition using dual-decomposed EEGsignals. IEEE Sens. J. 2021, 22, 2496–2507 DOI: https://doi.org/10.1109/JSEN.2021.3135953

Y. Perwej, A. Perwej, F. Parwej, “An Adaptive Watermarking Technique for the copyright of digital images and Digital Image Protection”, International journal of Multimedia & Its Applications (IJMA), which is published by Academy & Industry Research Collaboration Center (AIRCC) , USA , Volume 4, No.2, Pages 21- 38, April 2012, DOI: 10.5121/ijma.2012.4202 DOI: https://doi.org/10.5121/ijma.2012.4202

Y. Perwej, N. Akhtar, F. Parwej, “The Kingdom of Saudi Arabia Vehicle License Plate Recognition using Learning Vector Quantization Artificial Neural Network”, International Journal of Computer Applications (IJCA), USA, ISSN 0975 – 8887, Volume 98, No.11, Pages 32 – 38, 2014, DOI: 10.5120/17230-7556 DOI: https://doi.org/10.5120/17230-7556

Sahoo, G.K.; Das, S.K.; Singh, P. Deep learning-based facial emotion recognition for driver healthcare. In Proceedings of the 2022 National Conference on Communications (NCC), Mumbai, India, 24–27 May 2022; pp. 154–159. DOI: https://doi.org/10.1109/NCC55593.2022.9806751

A. Krizhevsky and G. Hinton. “Learning multiple layers of features from tiny images”, 2009

K. Bouaziz, T Ramakrishnan, S. Raghavan, K. Grove, A.A.Omari, C Lakshminarayan, “ Character Recognition by Deep Learning: An Enterprise solution.”, 2018 IEEE Conference on Big Data DOI: https://doi.org/10.1109/BigData.2018.8622465

N. Akhtar, Y. Perwej, F. Parwej, Jai Pratap Dixit, “A Review of Solving Real Domain Problems in Engineering for Computational Intelligence Using Soft Computing” Proceedings of the 11th INDIACom; INDIACom-2017; SCOPUS, IEEE Conference ID: 40353, 2017 4th International Conference on “Computing for Sustainable Global Development”, ISSN 0973-7529; ISBN 978-93-80544-24-3, Pages 706–711, Bharati Vidyapeeth's Institute of Computer Applications and Management (BVICAM), New Delhi (INDIA), 01st - 03rd March, 2017

Zhang X-D (2020) A matrix algebra approach to artificial intelligence. Springer DOI: https://doi.org/10.1007/978-981-15-2770-8

Liu H, Lang B, ”Machine learning and deep learning methods for intrusion detection systems: a survey,” Appl Sci 9(20):4396, 2019 DOI: https://doi.org/10.3390/app9204396

Jung, H., Lee, K., & Yoon, C. (2019). Facial emotion recognition using deep neural networks with multimodal data. IEEE Transactions on Affective Computing, 10(4), 554-565.

Aparna Trivedi, Chandan Mani Tripathi, Y. Perwej, Ashish Kumar Srivastava, Neha Kulshrestha, “Face Recognition Based Automated Attendance Management System”, International Journal of Scientific Research in Science and Technology (IJSRST), Print ISSN : 2395-6011, Online ISSN : 2395-602X, Volume 9, Issue 1, Pages 261-268, January-February-2022, DOI: 10.32628/IJSRST229147 DOI: https://doi.org/10.32628/IJSRST229147

Kim, K., Bang, H., & Kim, J. (2020). Emotion recognition from facial expressions using 3D convolutional neural networks. IEEE Trans. on Affective Computing, 11(1), 50-60.

Goodfellow, I., Bengio, Y., and Courville, A. ,”Deep Learning: Adaptive Computation and Machine Learning Series”, The MIT Press Book, Cambridge, MA, USA; 80, 2016

Wang, Z., Ho, S. B., and Cambria, E. ,” A review of emotion sensing: categorization models and algorithms”, Multimedia. Tools Appl. 79, 35553–35582, 2020 DOI: https://doi.org/10.1007/s11042-019-08328-z

Neumann,M.; Vu, N.T. Attentive convolutional neural network based speech emotion recognition: A study on the impact of input features, signal length, and acted speech. arXiv 2017, arXiv:1706.00612. DOI: https://doi.org/10.21437/Interspeech.2017-917

Imani, M.; Montazer, G.A. A survey of emotion recognition methods with emphasis on E-Learning environments. J. Netw. Comput. Appl. 2019, 147, 102423 DOI: https://doi.org/10.1016/j.jnca.2019.102423

Bhavesh Kumar Jaisawal, Y. Perwej, Sanjay Kumar Singh, Susheel Kumar, Jai Pratap Dixit, Niraj Kumar Singh, “An Empirical Investigation of Human Identity Verification Methods” , International Journal of Scientific Research in Science, Engineering and Technology (IJSRSET), Print ISSN: 2395-1990 ,Online ISSN : 2394-4099, Volume 10, Issue 1, Pages 16-38, 2022, DOI: 10.32628/IJSRSET2310012 DOI: https://doi.org/10.32628/IJSRSET2310012

Sha, T.; Zhang, W.; Shen, T.; Li, Z.; Mei, T. Deep Person Generation: A Survey from the Perspective of Face, Pose, and Cloth Synthesis. ACM Comput. Surv. 2023, 55, 1–37 DOI: https://doi.org/10.1145/3575656

Apoorva Dwivedi, Dr. Basant Ballabh Dumka, N. Akhtar, Ms Farah Shan, Y. Perwej, “Tropical Convolutional Neural Networks (TCNNs) Based Methods for Breast Cancer Diagnosis”, International Journal of Scientific Research in Science and Technology (IJSRST), Print ISSN: 2395-6011, Online ISSN: 2395-602X, Volume 10, Issue 3, Pages 1100 -1116, 2023, DOI: 10.32628/IJSRST523103183 DOI: https://doi.org/10.32628/IJSRST523103183

Sachin Bhardwaj, Apoorva Dwivedi, Ashutosh Pandey, Y. Perwej, Pervez Rauf Khan, “Machine Learning-Based Crowd Behavior Analysis and Forecasting”, International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN: 2456-3307, Volume 9, Issue 3, Pages 418-429, 2023-2023, DOI: 10.32628/CSEIT23903104 DOI: https://doi.org/10.32628/CSEIT23903104

Sun, Q.; Liang, L.; Dang, X.; Chen, Y. Deep learning-based dimensional emotion recognition combining the attention mechanism and global second-order feature representations. Comput. Electr. Eng. 2022, 104, 108469 DOI: https://doi.org/10.1016/j.compeleceng.2022.108469

Shweta Pandey, Rohit Agarwal, Sachin Bhardwaj, Sanjay Kumar Singh, Y. Perwej, Niraj Kumar Singh, “A Review of Current Perspective and Propensity in Reinforcement Learning (RL) in an Orderly Manner” , International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 9, Issue 1, Pages 206-227, 2023, DOI: 10.32628/CSEIT2390147 DOI: https://doi.org/10.32628/CSEIT2390147

Y. Goldberg and M. Elhadad, "SVM: Fast Space-Efficient non-Heuristic Polynomial Kernel Computation for NLP Applications", Proc. ACL-08: HLT, 2008

Zhao H, Xiao Y, Zhang Z,”Robust semi supervised generative adversarial networks for speech emotion recognition via distribution smoo.”, IEEE Access 8:106889–106900, 2020 DOI: https://doi.org/10.1109/ACCESS.2020.3000751

N. Akhtar, Dr. Hemlata Pant, Apoorva Dwivedi, Vivek Jain, Y. Perwej, “A Breast Cancer Diagnosis Framework Based on Machine Learning”, International Journal of Scientific Research in Science, Engineering and Technology (IJSRSET), Print ISSN: 2395-1990, Volume 10, Issue 3, Pages 118-132, 2023, DOI: 10.32628/IJSRSET2310375 DOI: https://doi.org/10.32628/IJSRSET2310375

Zhang J, Yin Z, Chen P, Nichele S ,”Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review”, Inf Fusion 59:103–126, 2020 DOI: https://doi.org/10.1016/j.inffus.2020.01.011

Saurabh Sahu, Km Divya, Dr. Neeta Rastogi, Puneet Kumar Yadav, Y. Perwej, “Sentimental Analysis on Web Scraping Using Machine Learning Method” , Journal of Information and Computational Science (JOICS), ISSN: 1548-7741, Volume 12, Issue 8, Pages 24-29, 2022, DOI: 10.12733/JICS.2022/V12I08.535569.67004

Wan-Hui W, Yu-Hui Q, Guang-Yuan L.,”Electrocardio graphy recording, feature extraction and classification for emo tion recognition”, In: 2009 WRI World congress on computer science and information eng., vol 4, pp 168–172. IEEE, 2009 DOI: https://doi.org/10.1109/CSIE.2009.130

Mohsen S, Alharbi AG (2021) EEG-based human emotion prediction using an LSTM model. In: 2021 IEEE international midwest symposium on circuits and systems (MWSCAS), pp 458–461. IEEE DOI: https://doi.org/10.1109/MWSCAS47672.2021.9531707

Peng S, Cao L, Zhou Y, Ouyang Z, Yang A, Li X, Jia W, Shui Yu (2022) A survey on deep learning for textual emotion analysis in social networks. Dig Commun Netw 8(5):745–762 DOI: https://doi.org/10.1016/j.dcan.2021.10.003