Improved Ridge Estimation with Singular Value Correction Constraints

It is well known that ridge estimation is better than least squares estimation for ill posed problems, however the least squares estimation is unbiased compare to ridge estimation which is biased. Actually, ridge estimation reduces the variance by introducing bias so as to improve the MSE (mean square error). Therefore, ridge estimation is better than least squares estimation in term of MSE. Since the MSE is composed of variance and bias, the performance of the ridge estimation can be shown clearly through computing the variance and bias. Through analyzing the changes of variance and bias of ridge estimation, we know that ridge estimation correct the singular values of the ill posed matrix to reduce the variance and introduce the bias. When the reduced variance is much more than the introduced bias, the MSE can be reduced. However, ridge estimation correct all the singular values in the ill-conditioned matrix. Correcting the big singular values cannot reduce the variance of the estimation effectively but introduce much bias into the estimation. In view of this, improved ridge estimation is proposed in this paper to constrain the correction of the singular values. The new ridge estimation only correct the small singular values which are determined by comparing the variance reduction with bias introduction of singular value correction. Theoretical analysis clearly shows the feasibility of the improved ridge estimation. The experiment on the basis of the Fredholm integral equation of the first kind is carried out to demonstrate the effectiveness of the new ridge estimation. The results show that the improved ridge estimation performs much better than ridge estimation in stability and accuracy.