Abstract: As is known to all,high resolution remote sensing imagery contains plenty of spectral and spatial information which represent objects' inner physical attributes and outer distribution and composite structure respectively.According to its characteristics mentioned above,we proposed an building extraction method based on spatial-spectral information on high resolution remote sensing imagery.On the basis of feature unit extraction through multi-scale segmentation and vectorization,we selected building samples automatically from feature units according to characteristics like shape and spectral information.Then,we preceded the building area recognition on the whole image using the template constructed by buildings' shape,and done edge detection and thinning inside the building regions to extract the outline of building.Last,we implemented the algorithm mentioned above,and proved its validity through experiments.