Personalized Drone Interaction : Adaptive Hand Gesture Control with Facial Authentication

Idris Seidu; Jafaar  Olasunkanmi Lawal

doi:10.32628/IJSRSET241146

Authors

Idris Seidu Department of Mechanical Engineering, Boston University, Boston, Massachusetts, U.S.A Author
Jafaar Olasunkanmi Lawal Department of Mathematics, Federal College of Education Technical Ekiadolor, Benin City, Edo State, Nigeria Author

DOI:

https://doi.org/10.32628/IJSRSET241146

Keywords:

Computer Vision, Adaptive Hand Gesture Control, DJI Tello drone, Biometric Authentication, Robotics

Abstract

This paper presents a novel system for personalized drone interaction, integrating adaptive hand gesture control with facial authentication. Utilizing the DJI Tello drone equipped with a 5 MP camera, the system employs advanced computer vision and machine learning techniques to ensure secure and intuitive control. Facial recognition using the Histogram of Oriented Gradients (HOG) method and FaceNet model verifies user identity, while MediaPipe and a custom convolutional neural network (CNN) facilitate accurate hand gesture recognition. The system’s real-time processing capabilities ensure seamless and responsive user interaction. Experimental results demonstrate the system’s robustness and accuracy in various scenarios, highlighting its potential for diverse applications such as security, entertainment, and personal assistance.

Downloads

Download data is not yet available.

References

A. Yemelyev, K. Moldamurat and R. B. Seksenbaeva, “Development and Implementation of Automated UAV Flight Algorithms for Inertial Navigation Systems,” in 2021 IEEE International Conference on Smart Information Systems and Technologies (SIST), 2021. DOI: https://doi.org/10.1109/SIST50301.2021.9465965

V. Kangunde, R. S. Jamisola and E. Theophilus, “A review on drones controlled in real-time,” International Journal of Dynamics and Control, vol. 9, pp. 1832-1846, 2021. DOI: https://doi.org/10.1007/s40435-020-00737-5

R. Gunturu, K. N. Durgaa, T. S. Harshaa and S. F. Ahamed, “Development of Drone Based Delivery System Using Pixhawk Flight Controller,” Transportation Modes eJournal, 2020. DOI: https://doi.org/10.2139/ssrn.3734801

O. Adeleke, O. Ojekanmi and I. Seidu, “Development and Performance Evaluation of a Quadcopter,” INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING AND MANAGEMENT, vol. 3, p. 1116, 2021.

K. M. Hasan and S. Ahsan, “Design and development of an aircraft type portable drone for surveillance and disaster management,” International Journal of Intelligent Unmanned Systems, 2018. DOI: https://doi.org/10.1108/IJIUS-02-2018-0004

K. Natarajan, T.-H. D. Nguyen and M. Mete, “Hand Gesture Controlled Drones: An Open Source Library,” 2018 1st International Conference on Data Intelligence and Security (ICDIS), pp. 168-175, 2018. DOI: https://doi.org/10.1109/ICDIS.2018.00035

Y. Ma, Y. Liu, R. Jin, X. Yuan, R. Sekha, S. Wilson and R. Vaidyanathan, “Hand gesture recognition with convolutional neural networks for the multimodal UAV control,” 2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 198-203, 2017. DOI: https://doi.org/10.1109/RED-UAS.2017.8101666

Z. Zhao, H. I. Luo, G.-H. Song, Z. Chen, Z.-m. Lu and X. Wu, “Web-based interactive drone control using hand gesture.,” The Review of scientific instruments, vol. 89 1, p. 014707, 2018. DOI: https://doi.org/10.1063/1.5004004

B. Latif, N. Buckley and E. L. Secco, “Hand Gesture and Human-Drone Interaction,” in Intelligent Systems with Applications, 2022. DOI: https://doi.org/10.1007/978-3-031-16075-2_20

Y. Yu, X. Wang, Z. Zhong and Y. Zhang, “ROS-based UAV control using hand gesture recognition,” 2017 29th Chinese Control And Decision Conference (CCDC)}, pp. 6795-6799, 2017. DOI: https://doi.org/10.1109/CCDC.2017.7978402

C.-C. Tsai, C.-C. Kuo and Y.-L. Chen, “3D Hand Gesture Recognition for Drone Control in Unity*},” 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)}, pp. 985-988, 2020. DOI: https://doi.org/10.1109/CASE48305.2020.9216807

A. Sarkar, K. A. Patel, R. K. G. Ram and G. K. Capoor, “Gesture control of drone using a motion controller,” 2016 International Conference on Industrial Informatics and Computer Systems (CIICS), pp. 1-5, 2016. DOI: https://doi.org/10.1109/ICCSII.2016.7462401

F. Naseer, G. Ullah, M. A. Siddiqui, M. J. Khan, K. S. Hong and N. Naseer, “Deep Learning-Based Unmanned Aerial Vehicle Control with Hand Gesture and Computer Vision,” 2022 13th Asian Control Conference (ASCC), pp. 1-6, 2022. DOI: https://doi.org/10.23919/ASCC56756.2022.9828347

M. K. Z. B. A. Mutalib, “Flying Drone Controller by Hand Gesture Using Leap Motion,” International Journal of Advanced Trends in Computer Science and Engineering, 2020.

S. Khaksar, L. Checker, B. Borazjani and I. Murray, “Design and Evaluation of an Alternative Control for a Quad-Rotor Drone Using Hand-Gesture Recognition,” Sensors (Basel, Switzerland), vol. 23, 2023. DOI: https://doi.org/10.3390/s23125462

V. K. Nguyen and R. Alba-Flores, “UAVs Control Using 3D Hand Keypoint Gestures,” SoutheastCon 2022, pp. 140-144, 2022. DOI: https://doi.org/10.1109/SoutheastCon48659.2022.9764030

K. Haratiannejadi, N. Elhami Fard and R. R. Selmic, “Smart Glove and Hand Gesture-based Control Interface For Multi-rotor Aerial Vehicles,” in 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019. DOI: https://doi.org/10.1109/SMC.2019.8914464

A. Budiyanto, M. I. Ramadhan, I. Burhanudin, H. H. Triharminto, B. Santoso and I. Indonesia, “Navigation control of Drone using Hand Gesture based on Complementary Filter Algorithm,” Journal of Physics: Conference Series, vol. 1912, 2021. DOI: https://doi.org/10.1088/1742-6596/1912/1/012034

T. Begum, I. T. Haque and V. Keselj, “Deep Learning Models for Gesture-controlled Drone Operation,” in 2020 16th International Conference on Network and Service Management (CNSM), 2020. DOI: https://doi.org/10.23919/CNSM50824.2020.9269056

A. A. Bandala, J. M. Z. Maningo, E. Sybingco, R. R. P. Vicerra, E. P. Dadios, J. D. D. Guillarte, J. O. P. Salting, M. J. A. S. Santos and B. A. E. Sarmiento, “Development of Leap Motion Capture Based - Hand Gesture Controlled Interactive Quadrotor Drone Game,” in 2019 7th International Conference on Robot Intelligence Technology and Applications (RiTA), 2019. DOI: https://doi.org/10.1109/RITAPP.2019.8932800

A. Menshchikov, D. Ermilov, I. Dranitsky, L. Kupchenko, M. Panov, M. V. Fedorov and A. Somov, “Data-Driven Body-Machine Interface for Drone Intuitive Control through Voice and Gestures,” in IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society, 2019. DOI: https://doi.org/10.1109/IECON.2019.8926635

F. Mahmud, M. T. Khatun, S. T. Zuhori, S. Afroge, M. Aktar and B. Pal, “Face recognition using Principle Component Analysis and Linear Discriminant Analysis,” in 2015 International Conference on Electrical Engineering and Information Communication Technology (ICEEICT), 2015. DOI: https://doi.org/10.1109/ICEEICT.2015.7307518

C. Mageshkumar, R. Thiyagarajan, S. Natarajan, S. Arulselvi and G. Sainarayanan, “Gabor features and LDA based face recognition with ANN classifier,” in 2011 International Conference on Emerging Trends in Electrical and Computer Technology, 2011. DOI: https://doi.org/10.1109/ICETECT.2011.5760234

L. H. Chan, S. H. S. Salleh and C.-M. Ting, “PCA, LDA and neural network for face identification,” in 2009 4th IEEE Conference on Industrial Electronics and Applications, 2009.

S. D. Sarkar and A. K. B. Shenoy, “Face Recognition using Artificial Neural Network and Feature Extraction,” in 2020 7th International Conference on Signal Processing and Integrated Networks (SPIN), 2020. DOI: https://doi.org/10.1109/SPIN48934.2020.9071378

P. Borisagar, S. Jani, Y. Agrawal and R. Parekh, “An Efficient and Compact Review of Face Recognition Techniques,” in 2020 IEEE International Students' Conference on Electrical,Electronics and Computer Science (SCEECS), 2020. DOI: https://doi.org/10.1109/SCEECS48394.2020.143

G. R. Ragul, C. Mageshkumar, R. Thiyagarajan and R. Mohan, “Comparative study of statistical models and classifiers in face recognition,” in 2013 International Conference on Information Communication and Embedded Systems (ICICES), 2013. DOI: https://doi.org/10.1109/ICICES.2013.6508221

“Ryze Tello Drone,” [Online]. Available: url{https://modelforce.eu/en/product/ryze-tello-drone/. [Accessed 8 July 2024].

G. Adam, “Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning,” 2016. [Online]. Available: url{https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78. [Accessed 8 July 2024].

“MediaPipe Hands: Real-Time Hand Tracking and Gesture Recognition,” [Online]. Available: url{https://mediapipe.readthedocs.io/en/latest/solutions/hands.html. [Accessed 8 July 2024].