Multiple-level information source and entropy-reduced transformation models

摘要

An approach has been proposed to analyse systematically the changes in entropy which occur in the different stages of a pattern recognizer. It models the entire pattern recognition system as a multiple level information source (MLIS). For a typical recognition system there are four levels in this information source, i.e. IS1 to IS4, and they can be divided into two categories: entropy-reduced and entropy-increased. By examining the internal structures of a pattern recognition system, MLIS allows us to address all the different factors which increase the entropy at the different levels of the entire system. Such factors include (1) IS1: Intrinsic characteristics of the target pattern set, (2) IS2: Distortions due to the transducer and digitization, and (3) IS4: Information loss due to feature extraction process. A theoretical analysis of the entropy distribution in the MLIS indicates that in order to improve the performance of a pattern recognition system, the entropy of the MLIS must be reduced in all the different levels. Specifically, IS1, IS2 and IS4 must be converted. To perform this conversion, a theoretical framework called entropy-reduced transformation (ERT) has been developed. Two examples are presented to illustrate how the proposed MLIS solves a practical engineering problem of pattern recognition and how the ERT is applied to design a pattern recognizer.