A third modality of natural language?

摘要

In the late Eighties the Natural Language Processing community began appreciating the role of multimodality in interactive systems. Intelligent multimodal systems are systems that integrate natural language (generally so far keyboard-based input, shortly also voice) with other media such as gestures in input or graphics in output. The perspective of what can be called visible interactive communication is discussed and considered as a possible new modality of natural language, after the spoken and the written ones. This should not be confused with the type of hypermedia that are now being developed. There, basically, the interface space is finite, even if one dimension may be added. Here the infinite creativity of human language is potentially preserved as the fundamental communication instrument.