Abstract: Damaged buildings detection from high-resolution remote sensing image helps to quick disaster losses evaluation after the earthquake. This paper presents a new method to detect damaged buildings using spectral gradient local spatial statistics, based on the analysis of gradient distribution characteristics of damaged buildings in the high-resolution remote sensing image. Firstly, spectral gradient image is obtained by Prewitt gradient operator. Secondly, local spatial statistics is used to evaluate the spectral gradient correlation within the roofs, and to generate the preliminary results. At last, the post processing steps, including minimal value analysis and shadow detection, are taken to optimize preliminary results and obtain the final results. The experiment results using a Quickbird image of Yushu earthquake and optical aerial image of Yingjiang earthquake demonstrate the effectiveness of the proposed method, which provides an overall accuracy of higher than 80%, are better than traditional methods.