IDR Logo

Please use this identifier to cite or link to this item:
Title: Fingertip detection and tracking for recognition of air-writing in videos
Authors: Mukherjee S.
Ahmed S.A.
Dogra D.P.
Kar S.
Roy P.P.
Keywords: Air-writing
Fingertip detection and tracking
Hand pose detection
Handwritten character recognition
Human-computer interaction (HCI)
Issue Date: 2019
Abstract: Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1% while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11% using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems. � 2019 Elsevier Ltd
Appears in Collections:Research Publications

Files in This Item:
There are no files associated with this item.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.