Abstract: In order to evaluate the quality of the distorted image，it is necessary to calculate the similarity degree between the distorted image and the original image. By integrating gradient magnitude and gradient phase of image with structural similarity(SSIM)，this paper proposed a new image quality assessment model—gradient similarity(GSIM)，and the image quality assessment algorithm based on this model.Compared with the SSIM model and the Uradient-based model，this new model not only includes luminance，contrast and structure of image，but more important lies in that it adds gradient phase information on the new model. The result of experiments，through evaluating 982 distoned images in the LIVE database and 924 remote sensing images compression，shows that this new model is superior to traditional models of MSE，PSNR，SSIM and the Uradient-based model.This new model，contrast with traditional model of SSIM，can find better solutions to the problem of objective assessment on seriously distorted images inconsistent with the subjective perception，and also the problem of the mixing evaluation effectiveness relatively worse to multiple types distorted images.Therefore，this new model can truly reflect the quality of the visual perception of the distorted image with higher assessment reliability.