Lip Movement Feature Detection and Classification Methods

Authors(5) :-Siddharth Mujumdar, Monika Borse, Jignasa Shah, Gunjan Soni, Dr. Sheshang Degadwala

Computerized lip reading has been one of the most actively researched areas of computer vision in recent past because of its crime fighting potential and invariance to acoustic environment. However, several factors like fast speech, bad pronunciation, and poor illumination, movement of face, moustaches and beards make lip reading difficult. In present work, we propose a solution for automatic lip contour tracking and recognizing numbers 1-10 of English language spoken by speakers using the information available from lip movements. In this method first, we are detecting Face then in that ROI Lip detection. Lip movements as the only input and without using any speech recognition system in parallel. The approach used in this work is found to significantly solve the purpose of lip reading when size of database is small. In this paper also compare LBP and HOG feature with Support Vector Machine Classification. The system is use for the modern sign in PASSWORD systems.

Authors and Affiliations

Siddharth Mujumdar
UG Student, Computer Engineering, Sigma Institute of Engineering, Vadodara, Gujarat, India
Monika Borse
UG Student, Computer Engineering, Sigma Institute of Engineering, Vadodara, Gujarat, India
Jignasa Shah
UG Student, Computer Engineering, Sigma Institute of Engineering, Vadodara, Gujarat, India
Gunjan Soni
UG Student, Computer Engineering, Sigma Institute of Engineering, Vadodara, Gujarat, India
Dr. Sheshang Degadwala
Head of Department, Computer Engineering, Sigma Institute of Engineering, Vadodara, Gujarat, India

Lip detection, Lip Movement, Contour Tracking, LBP, HOG, SVM

  1. Hueber T, Benaroya E-L, Chollet G, Denby B, Dreyfus G, Stone M. .” Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips. Speech Communication”. 52 288–300.,2010
  2.  Wang, J., Samal, A., & Green, J. R. . Preliminary test of a realtime, interactive silent speech interface based on electromagnetic articulograph, the 5th ACL/ISCA Workshop on Speech and Language Processing for Assistive Technologies, Baltimore, MD, 38-45.,2014
  3. Jorgensen C, Dusan S. . Speech interfaces based upon surface electromyography. Speech Communication.,2010
  4. Schultz T, Wand M. (2010). Modeling Coarticulation in EMGbased Continuous Speech Recognition. Speech Communication. 52: 341-353.
  5. P. Viola and M. Jones, “Robust real-time face detection,” Int. J. Comput.Vis., vol. 57, no.2, pp. 137–154, 2004.
  6. N. Eveno, A. Caplier, and P. Y. Coulon, 2002, “Key points based segmentation of lips,” in Proc. ICME, pp. 125–128
  7. Shreyansh Jain, Prem C. Pandey, and Rajbabu Velmurugan Department of Electrical Engineering, Lip Contour Detection for Estimation of Mouth Opening Area, 978-1-4673-8564-0/15/$31.00 ©2015 IEEE 
  8. Lalitha. S.D. , Dr. Thyagharajan K.K, A Study on Lip Localization Techniques used for Lip reading from a Video, International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 1 (2016) pp 611-615
  9. Ho Lip Chin,Marsyita Hanafi, Tanko Danial Salk, Integrated Face And Facial Components Detection0 2166-8523/15 $31.00 © 2015 IEEE
  10. Schultz T, Wand M. (2010).Modeling Coarticulation in EMGbased Continuous Speech Recognition. Speech Communication. 52: 341-353
  11. L. G. ves da Silveira, J. Facon, and D. L. Borges, “Visual speech recognition: A solution from feature extraction to words classi?cation,” Proc. XVI Brazilian Symposium on Computer Graphics and Image Processing, pp. 399-405, 2003.
  12. Y. Ogoshi, H. Ide, C. Araki, and H. Kimura, “Active Lip Contour Using Hue Characteristics Energy Model for A Lip Reading System,” Trans. IEEJ, Vol.128, No.5, pp. 811-812, 2008.
  13. S. Saha and P. C. Pandey, “Estimation of the area of mouth opening during speech production,” in Proc. 8th Indian Conference on Vision, Graphics and Image Processing, Mumbai, India, 2012
  14. A. Anchit and S. Mathur, “Comparative analysis of Haar and Skin color method for face detection,” Recent Advances and Innovations in Engineering (ICRAIE), pp. 15, 2014.
  15. Alan Wee-Chung Liewa, Shu Hung Leungb, Wing Hong Laua, Nov 2002, “Lip contour extraction from color images using a deformable model”, The Journal of Pattern Recognition Society

Publication Details

Published in : Volume 4 | Issue 5 | March-April 2018
Date of Publication : 2018-04-10
License:  This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 312-317
Manuscript Number : CI016
Publisher : Technoscience Academy

Print ISSN : 2395-1990, Online ISSN : 2394-4099

Cite This Article :

Siddharth Mujumdar, Monika Borse, Jignasa Shah, Gunjan Soni, Dr. Sheshang Degadwala, " Lip Movement Feature Detection and Classification Methods, International Journal of Scientific Research in Science, Engineering and Technology(IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 4, Issue 5, pp.312-317, March-April-2018. Available at doi : https://doi.org/10.32628/CI016      Citation Detection and Elimination     |     
Journal URL : https://ijsrset.com/CI016

Article Preview