Techniques for online gesture recognition on workstations

摘要

We describe an automatic, unsupervised procedure for the recognition of a small set of gestures drawn with the aid of a mouse in the Windows environment of a modern computer workstation. The paper considers the application to various shorthand gestures which are commonly used in the design of computer networks. Furthermore, the method incorporates a facility for automatic addition of new gestures and the deletion of unnecessary gestures so that the approach may be easily adapted to other sets of gestures. The proposed procedure utilizes certain characteristic shape features extracted from each gesture together with appropriate decision rules. This is backed up by a menu-based correction/confirmation utility. The problem is of particular interest since the gestures are subject to considerable internal rescaling and changes in orientation because of both the individual's drawing style and the nature of the application. The characteristic features are primitives chosen by inspection of the geometrical properties of the expected shapes backed up with a video-recorded psychological study of drawing style. We present some experimental results for a specific set of ‘network design gestures’ which indicate the potential for accurate gesture recognition.