Abstract: This paper proposes an approach for automatic extracting shadows of building from remote sensing image on the basis of the method that gray statistics is done in one side of the structure lines to detect shadows.The approach,which combines constraints of gray,geometry and context,and others assistant information,realizes the shadow extraction of buildings.Finally the image of IKONOS and aerial photograph are experimented to extract shadows of building.Results of the experiment show that this approach is effective and robust.