AIR-Writing Word Recognition

Authors

  • Sushruthi Sheri  ECM Department, JB Institute of Engineering & Technology, Hyderabad, Telangana, India
  • Palle Karthik Reddy  ECM Department, JB Institute of Engineering & Technology, Hyderabad, Telangana, India
  • Mr. Narsappa  ECM Department, JB Institute of Engineering & Technology, Hyderabad, Telangana, India

Keywords:

Handwriting Recognition, Air-Writing, 6-DOF Motion, Usability Study

Abstract

Air-writing refers to writing of linguistic characters or words in a free space by hand or finger movements. Airwriting differs from conventional handwriting; the latter contains the pen-up-pen-down motion, while the former lacks such a delimited sequence of writing events. We address air-writing recognition problems in a pair of companion papers. In Part 1, recognition of characters or words is accomplished based on 6 degrees-of-freedom hand motion data. We address air-writing on two levels: motion characters and motion words. Isolated air-writing characters can be recognized similar to motion gestures although with increased sophistication and variability. For motion word recognition in which letters are connected and superimposed in the same virtual box in space, we build statistical models for words by concatenating clustered ligature models and individual letter models. Hidden Markov model is used for airwriting modeling and recognition. We show that motion data along dimensions beyond a 2D trajectory can be beneficially discriminative for air-writing recog-nition. We investigate the relative effectiveness of various feature dimensions of optical and inertial tracking signals, and report the attainable recognition performance correspondingly. The proposed system achieves a word error rate of 0.8% for word-based recognition and 1.9% for letter-based recognition. We also subjectively and objectively evaluate the effectiveness of airwriting and compare it to text input using a virtual keyboard. The words-per-minute of airwriting and virtual keyboard are 5.43 and 8.42, respectively.

References

  1. M. Chen, G. AlRegib, and B.-H. Juang, “Air-writing recognition, part 2: Detection and recognition of writing activity in continuous stream of motion data,” IEEE Trans. HumanMach. Syst, in press.
  2. “Leap motion,” http://www.leapmotion.com, 2015.
  3. M. Chen, G. AlRegib, and B.-H. Juang, “Feature processing and modeling for 6d motion gesture recognition,” IEEE Trans. Multimedia, vol. 15, no. 3, 2013, pp. 561–571.
  4. C. H. Blickenstorfer, “Graffiti: Wow!” Pen Computing Magazine, vol. 1, 1995, pp. 30–31.
  5. K. Perlin, “Quikwriting: continuous stylus-based text entry,” in Proc. of the 11th annual ACM symposium on User interface software and technology, 1998, pp. 215–216.
  6. P. Isokoski and R. Raisamo, “Quikwriting as a multi-device text entry method,” in Proc. of the third Nordic conference on Human-computer interaction, 2004, pp. 105–108.
  7. T. Koltringer,¨ P. Isokoski, and T. Grechenig, “Twostick: writing with a game controller,” in Proc. of Graphics Interface, 2007, pp. 103–110.
  8. “Swype — type fast, swype faster,” http://www.swype.com/, 2013.
  9. R. Plamondon and S. Srihari, “Online and off-line handwriting recogni-tion: a comprehensive survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, 2000, pp. 63 –84.
  10. J. Makhoul, T. Starner, R. Schwartz, and G. Chou, “On-line cursive handwriting recognition using hidden markov models and statistical grammars,” in Proc. of the workshop on Human Language Technology, 1994, pp. 432–436.
  11. J. Hu, M. Brown, and W. Turin, “Hmm based online handwriting recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 10, 1996, pp. 1039–1045.
  12. B. K. Sin and J. H. Kim, “Ligature modeling for online cursive script recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 6, 1997, pp. 623–633.
  13. J. Mantyj¨arvi,¨ J. Kela, P. Korpipa¨a,¨ and S. Kallio, “Enabling fast and effortless customisation in accelerometer based gesture interaction,” in Proc. of the 3rd Int’l Conf. on Mobile and ubiquitous multimedia, 2004, pp. 25–31.
  14. Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti, “American sign language recognition with the kinect,” in Proc. of the 13th intl. conf. on multimodal interfaces, 2011, pp. 279–286.
  15. N. Pugeault and R. Bowden, “Spelling it out: Real-time asl finger-spelling recognition,” in Computer Vision Workshops (ICCV Work-shops), 2011 IEEE International Conference on. IEEE, 2011, pp. 1114–1119.
  16. C. Amma, D. Gehrig, and T. Schultz, “Airwriting recognition using wearable motion sensors,” in Proc. of the 1st Augmented Human Intl. Conf., 2010, pp. 10:1–10:8.
  17. C. Amma, M. Georgi, and T. Schultz, “Airwriting: Hands-free mobile text input by spotting and continuous recognition of 3d-space handwrit-ing with inertial sensors,” in Proc. of the 16th International Symposium on Wearable Computers, 2012, pp. 52–59.
  18. L. Jin, D. Yang, L.-X. Zhen, and J.-C. Huang, “A novel vision based finger-writing character recognition system,” in Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, vol. 1, 2006, pp. 1104–1107.
  19. A. Schick, D. Morlock, C. Amma, T. Schultz, and R. Stiefelhagen, “Vision-based handwriting recognition for unrestricted text input in mid-air,” in Proc. of the 14th ACM intl. conf. on Multimodal interaction, 2012, pp. 217–220.
  20. . Zhang, Z. Ye, L. Jin, Z. Feng, and S. Xu, “A new writing experience: Finger writing in the air using a kinect sensor,” MultiMedia, IEEE, vol. 20, no. 4, 2013, pp. 85–93.
  21. M. Chen, G. AlRegib, and B.-H. Juang, “6DMG: A new 6D motion gesture database,” in Proc. of the third annual ACM conf. on Multimedia systems, 2012.
  22. “Worldviz ppt-x,” http://www.worldviz.com, 2015.
  23. P. O. Kristensson, T. Nicholson, and A. Quigley, “Continuous recogni-tion of one-handed and two-handed gestures using 3d full-body motion tracking sensors,” in Proc. of the ACM international conference on Intelligent User Interfaces, 2012, pp. 89–92.
  24. T. Brants and A. Franz, “Web 1T 5-gram version 1,” Linguistic Data Consortium, Philadelphia, 2006.
  25. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1993.
  26. L. E. Baum, T. Petrie, G. Soules, and N. Weiss, “A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains,” The Annals of Mathematical Statistics, vol. 41, no. 1, 1970, pp. 164–171.
  27. S. Katz, “Estimation of probabilities from sparse data for the language model component of a speech recognizer,” IEEE Trans. Acoustics, Speech and Signal Processing, vol. 35, no. 3, 1987, pp. 400–401.
  28. C.-M. Karat, C. Halverson, D. Horn, and J. Karat, “Patterns of entry and correction in large vocabulary continuous speech recognition systems,” in Proc. of the SIGCHI conference on Human Factors in Computing Systems, 1999, pp. 568–575.

International Journal of Scientific Research in Science, Engineering and Technology

Downloads

Published

2023-04-30

Issue

Volume 10, Issue 2, March-April-2023

Section

Research Articles

License

Copyright (c) IJSRSET

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Sushruthi Sheri, Palle Karthik Reddy, Mr. Narsappa "AIR-Writing Word Recognition" International Journal of Scientific Research in Science, Engineering and Technology (IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 10, Issue 2, pp.369-374, March-April-2023.