Abstract: For three-dimensional coordinate transformation, it's impossible using the Gauss-Markov model to obtain optimal parameter estimation from the functional model with error in its coefficient matrix. On the other hand, errors-in-variables model has difficulty expressing the functional model, and partial errors-in-variables model is complex as well as too much parameters to be estimated for the quasi-observation method. Therefore, Gauss-Helmert model is employed for three-dimensional coor-dinate transformation. The target function of the proposed model is established based on Newton-Gauss iterative algorithm, and the estimated method and its derivation procedure also are presented in this paper. Beyond the above process, we proposed a new robust estimation method for the proposed model, which is based on the normalized residual error and takes the influence of gross error on both observation and structure spaces into consideration. Meanwhile, derivational process of statistical tests and iterative algorithm are presented. The simulation experiment results show that the proposed estimation method has the same accuracy as the traditional method, which has robust with angular dimension and other additional conditions, but less estimation parameters. In addition, the new robust estimation method has effective robustness when comparing with the other existing robust total least square methods for the coordinate transformation.