Exploration of directional-predictive sounds for nonvisual interaction with graphs

摘要

Sonification of stylus movements accompanied with kinesthetic feedback is one of possible techniques to develop cross-modal coordination in the absence of visual information. The investigated problems are the following: how to minimize the number of sounds while increasing the information they contain and how to choose a natural sonification grammar which would not require extra cognitive efforts. We demonstrate two case studies of employing directional-predictive sounds (DPS). Stylus movements were sonified through three sound signals taking into account the exploration behavior and the concept of the capture radius. The performance of eight subjects was evaluated in terms of the stylus deviation in relation to the points of the virtual graph, a length of the scanpaths, and the task completion time. When stylus movements were accompanied with the DPS signals within four capture radiuses, the deviation of the stylus from the graph inspected was always less than one capture radius. The scanpaths were 24–40% shorter in length and the task completion times decreased by 20–25%. We also demonstrate the game application which was designed to optimize an exploration behavior enhanced by the DPS. The results of the proposed sonification technique based on the model of the exploration behavior are discussed.