Learning about the Parameter of the Bernoulli Model

摘要

We consider the problem of learning as much information as possible about the parameterθof the Bernoulli model {Pθ∣θ∈[0, 1]} from the statistical datax∈{0, 1}n,n⩾1 being the sample size. Explicating this problem in terms of the Kolmogorov complexity and Rissanen's minimum description length principle, we construct a computable point estimator which (a) extracts from x all information it contains aboutθ, and (b) discards all sample noise inx. Our result is closely connected with Rissanen's theorem about the optimality of his scheme of coding statistical data.