Script-based inference and memory retrieval in subsymbolic story processing

摘要

DISCERN is an integrated natural language processing system built entirely from distributed neural networks. It reads short narratives about stereotypical event sequences, stores them in episodic memory, generates fully expanded paraphrases of the narratives, and answers questions about them. Processing in DISCERN is based on hierarchically-organized backpropagation modules, communicating through a central lexicon of word representations. The lexicon is a double feature map system that transforms each orthographic word symbol into its semantic representation and vice versa. The episodic memory is a hierarchy of feature maps, where memories are stored “one-shot” at different locations. Several high-level phenomena emerge automatically from the special properties of distributed neural networks in this model. DISCERN learns to infer unmentioned events and unspecified role fillers, generates expectations and defaults, and exhibits plausible lexical access errors and memory interference behavior. Word semantics, memory organization, and appropriate script inferences are automatically extracted from examples. DISCERN shows that high-level natural language processing is feasible through integrated subsymbolic systems. Subsymbolic control of high-level behavior and representing and learning abstractions are the two main challenges in scaling up the approach to more open-ended tasks.