Abstract: Built-up areas, which refer to the areas covered by buildings, are important man-made geographical objects, especially in an urban environment. With the increasing availability of high-resolution satellite images, built-up area information can be obtained at a much finer scale. However, the increased spatial resolution makes the built-up areas spectrally more heterogeneous and structurally more complex, which poses a big challenge to the automatic detection of built-up areas. In this paper, a novel built-up area extraction method is proposed based on patch representation and merging algorithm for high-resolution satellite images. First, with the corner context constraints, the image is subdivided into small patches, which are regarded as the basic units of image processing. Afterward, the spatial variability of the image patch is modeled through spatial semivariogram, and texture and structural features are extracted by well-defined parameters to characterize the curve of semivariogram, and to achieve the integrated representation of multiple features for each image patch through a principle component analysis (PCA). Finally, the built-up patches are classified by the similarity of the spatial structural features and further merged into built-up areas. The experiments are conducted on image data from sensors of ZY-3 and QuickBird, and the results show that the proposed method can effectively extract built-up areas from high-resolution satellite images and show good adaptability as the image resolution changes. By using patch-based representation and merging, it can not only avoid the shortcomings of the traditional pixel-based methods and the image segmentation in the object-oriented method, but also can facilitate the modeling and description of the texture and structural features of built-up areas.