TY - GEN
T1 - Skeleton-based hand gesture recognition by using multi-input fusion lightweight network
AU - Hu, Qihao
AU - Gao, Qing
AU - Gao, Hongwei
AU - Ju, Zhaojie
PY - 2022/8/4
Y1 - 2022/8/4
N2 - Skeleton-based hand gesture recognition has achieved great success in recent years. However, most of the existing methods cannot extract spatiotemporal features well due to the skeleton noise. In real applications, some large models also suffer from a huge number of parameters and low execution speed. This paper presents a lightweight skeleton-based hand gesture recognition network by using multi-input fusion to address those issues. We convey two joint-oriented features: Center Joint Distances (CJD) feature and Center Joint Angles (CJA) feature as the static branch. Besides, the motion branch consists of Global Linear Velocities (GLV) feature and Local Angular Velocities (LAV) feature. Fusing static and motion branches, a robust input can be generated and fed into a lightweight CNN-based network to recognize hand gestures. Our method achieves 95.8% and 92.5% hand gesture recognition accuracy with only 2.24M parameters on the 14 gestures and 28 gestures of the SHREC’17 dataset. Experimental results show that the proposed method outperforms state-of-the-art (SOAT) methods.
AB - Skeleton-based hand gesture recognition has achieved great success in recent years. However, most of the existing methods cannot extract spatiotemporal features well due to the skeleton noise. In real applications, some large models also suffer from a huge number of parameters and low execution speed. This paper presents a lightweight skeleton-based hand gesture recognition network by using multi-input fusion to address those issues. We convey two joint-oriented features: Center Joint Distances (CJD) feature and Center Joint Angles (CJA) feature as the static branch. Besides, the motion branch consists of Global Linear Velocities (GLV) feature and Local Angular Velocities (LAV) feature. Fusing static and motion branches, a robust input can be generated and fed into a lightweight CNN-based network to recognize hand gestures. Our method achieves 95.8% and 92.5% hand gesture recognition accuracy with only 2.24M parameters on the 14 gestures and 28 gestures of the SHREC’17 dataset. Experimental results show that the proposed method outperforms state-of-the-art (SOAT) methods.
KW - joint-oriented feature Second Keyword
KW - multi-input fusion
KW - skeleton-based hand gesture recognition
UR - http://www.scopus.com/inward/record.url?scp=85135839465&partnerID=8YFLogxK
UR - https://icira2022.org/
U2 - 10.1007/978-3-031-13844-7_3
DO - 10.1007/978-3-031-13844-7_3
M3 - Conference contribution
AN - SCOPUS:85135839465
SN - 9783031138430
T3 - Lecture Notes in Computer Science
SP - 24
EP - 34
BT - Intelligent Robotics and Applications. ICIRA 2022
A2 - Liu, Honghai
A2 - Ren, Weihong
A2 - Yin, Zhouping
A2 - Liu, Lianqing
A2 - Jiang, Li
A2 - Gu, Guoying
A2 - Wu, Xinyu
PB - Springer
T2 - 15th International Conference on Intelligent Robotics and Applications, ICIRA 2022
Y2 - 1 August 2022 through 3 August 2022
ER -