Moreover, if the components of z are uncorrelated and have equal variance such that C _ Z = \ sigma ^ 2 I, where I is an identity matrix, then W = ( A ^ TA ) ^ {-1 } A ^ T is identical to the ordinary least square estimate.
12.
He is known for Stein's paradox in decision theory, which shows that ordinary least squares estimates can be uniformly improved when many parameters are estimated; for Stein's lemma, giving a formula for the covariance of one random variable with the value of a function of another when the two random variables are National Academy of Sciences.
13.
Then in a supplement to his 1810 paper written after he had seen Gauss's work, he showed that the central limit theorem provided a Bayesian justification for least squares : if one were combining observations, each one of which was itself the mean of a large number of independent observations, then the least squares estimates would not only maximize the likelihood function, considered as a posterior distribution, but also minimize the expected posterior error, all this without any assumption as to the error distribution or a circular appeal to the principle of the arithmetic mean.
