End-user inputs and the performance of uncertainty representation schemes

摘要

Most of the approaches to uncertain reasoning developed for expert systems require user judgements about the degree to which evidence is present or absent. These user inputs are incorporated directly into the uncertainty calculations and influence the advice a completed system offers. Although all the other system parameters are set by the developer when the system is built and “tuned,” user inputs are likely to vary according to individual opinions. This empirical study examines the effects of user inputs on system accuracy. Subjects used one of two uncertain reasoning models to build and tune a system that captured their knowledge of a hypothetical, inherently uncertain domain. In the course of doing so, each subject also provided personal judgments of how evidence values for a uniform set of test cases mapped onto the uncertainty parameter values input by users. Our analysis examines the error introduced by using each subject's inputs for the test cases in conjunction with each of the other subject's systems. We also discuss the practical implications of our findings for system builders.