Abstract: Buildings detection is an important research in the area of computer vision and digital photogrammetry.A new approach of buildings detection based on deformable templates is presented in this paper.This method is based on the synergy of different information to address the problem of buildings detection from a complex aerial image.The problem of building localization and identification is treated as a process of matching a deformable template to the building boundary in the aerial image that inputs to the system.The features of the buildings such as image intensity,image gradient and geometric constraints etc.are described by energy terms.The weighted sum of each energy term is the total energy of the deformable template.Prior knowledge of a building shape is described by a parameter vector that consists of a set of parameters.The prior geometric information is obtained from seeds.The operator of the system specifies the seeds by activating the mouse.The image intensity information and image gradient information are obtained from the input aerial image by pre processing.Simulated annealing is used in the optimizing process.The optimizing parameter vector can give the geometric and semantic information of the target in image experimental results are presented to demonstrate the efficiency of this approach.