Picture semantics for integrating text and diagram input

摘要

The saying ‘a picture is worth a thousand words’ exemplifies the great value of pictures in describing a scenario. Pictures convey spatial information in a compact form, allowing textual descriptions to concentrate on the non-spatial (henceforth, contextual) properties of objects. The difficult task in integrating text and diagrammatic input to a system is to establish coreference — matching object references in the text to objects in the diagram.