Kalman filter is a common algorithm for initial alignment of inertial navigation system. Its premise is to model the system state and get accurate statistical characteristics of system noise and observation noise. Under the condition of inaccurate model and observation noise, Kalman filter will tend to decline in accuracy or even result in divergence, thus will affect the alignment accuracy. In order to solve this problem, an improved fading Kalman filter is proposed in this paper. Multiple fading factors are introduced to adjust the prediction error covariance matrix, and a Chi-square test method is designed to check the filter state, based on the statistical characteristics of residuals. The introduction of the fading factor is more reasonable, and the algorithm's adaptivity is enhanced. The improved Kalman filter is applied to the initial alignment problem of inertial navigation system. Simulation and real data experiment results show that, the proposed algorithm can improve the accuracy and robustness of Kalman filter effectively.