Theory formation by heuristic search: The nature of heuristics II: Background and examples

摘要

Machine learning can be categorized along many dimensions, an important one of which is ‘degree of human guidance or forethought’. This continuum stretches from rote learning, through carefully-guided concept-formation by observation, out toward independent theory formation. Six years ago, the am program was constructed as an experiment in this latter kind of learning by discovery. Its source of power was a large body of heuristics, rules which guided it toward fruitful topics of investigation, toward profitable experiments to perform, toward plausible hypotheses and definitions. Since that time, we have gained a deeper insight into the nature of heuristics and the nature of the process of forming and extending theories empirically. ‘The Nature of Heuristics I’ paper presented the theoretical basis for this work, with an emphasis on how heuristics relate to each other. This paper presents our accretion model of theory formation, and gives many examples of its use in producing new discoveries in various fields. These examples are drawn from runs of a program called eurisko, the successor to am, that embodies the accretion model and uses a corpus of heuristics to guide its behavior. Since our model demands the ability to discover new heuristics periodically, as well as new domain objects and operators, some of our examples illustrate that process as well. ‘The Nature of Heuristics III’ paper describes the architecture of the eurisko program, and conclusions we have made from its behavior.