Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Real-time sign language recognition based on video stream

Real-time sign language recognition based on video stream In this paper, a real-time Chinese sign language recognition system is investigated. This system can recognise deaf-mute Chinese sign language, and output the recognition results in real time through text. A Chinese sign language dataset is firstly created with a normal RGB camera, and the entire dataset contains 500,000 video samples. In order to improve the recognition accuracy of the system for real-time applications, three-dimensional convolutional neural network (3D-CNN) is investigated, combined with optical flow processing base on total variation regularisation and L1-norm robust (TV-L1). A two-step down-frame processing is employed to extract the equal number of key frames from each video stream, and finally put into 3D-CNN to extract feature vectors. Comparative studies are conducted with that of the hidden Markov model (HMM) and recurrent neural network (RNN), with 92.6% recognition accuracy on a dataset containing 1,000 vocabularies. A complete real-time sign language recognition system is finally developed and reported, which is composed of a human interaction interface, a motion detection module, a hand and head detection module, and a video acquisition mechanism. Experimental results verify the generalisation performance of the system in real-time. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Systems, Control and Communications Inderscience Publishers

Real-time sign language recognition based on video stream

Download PDF
17 pages

Loading next page...
 
/lp/inderscience-publishers/real-time-sign-language-recognition-based-on-video-stream-G0nz5Uv9Li

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Inderscience Publishers
Copyright
Copyright © Inderscience Enterprises Ltd
ISSN
1755-9340
eISSN
1755-9359
DOI
10.1504/IJSCC.2021.114616
Publisher site
See Article on Publisher Site

Abstract

In this paper, a real-time Chinese sign language recognition system is investigated. This system can recognise deaf-mute Chinese sign language, and output the recognition results in real time through text. A Chinese sign language dataset is firstly created with a normal RGB camera, and the entire dataset contains 500,000 video samples. In order to improve the recognition accuracy of the system for real-time applications, three-dimensional convolutional neural network (3D-CNN) is investigated, combined with optical flow processing base on total variation regularisation and L1-norm robust (TV-L1). A two-step down-frame processing is employed to extract the equal number of key frames from each video stream, and finally put into 3D-CNN to extract feature vectors. Comparative studies are conducted with that of the hidden Markov model (HMM) and recurrent neural network (RNN), with 92.6% recognition accuracy on a dataset containing 1,000 vocabularies. A complete real-time sign language recognition system is finally developed and reported, which is composed of a human interaction interface, a motion detection module, a hand and head detection module, and a video acquisition mechanism. Experimental results verify the generalisation performance of the system in real-time.

Journal

International Journal of Systems, Control and Communications – Inderscience Publishers

Published: Jan 1, 2021

There are no references for this article.