Abstract: Pedestrian detection is one of the key technologies in the large video data to extract information, which is an important link in the process of large video data mining. This is a difficult problem because pedestrian can vary from place to place and time to time. The changes in illumination and viewpoint, variability in shape, non-rigid deformations all can cause variations. In order to achieve a fast and robust pedestrian detection, this paper proposes a pedestrian detection algorithm based on sparse multi-scale image segmentation and cascade deformable part model. Through the sparse multi-scale image segmentation algorithm based on texture, lots of background region is eliminated and the interesting area is extracted. In the segmented interesting area, a general method is used for building cascade classifiers from part-based deformable models such as pictorial structures. Pictorial structures describe objects by a collection of parts included in a deformable configuration. Each part stands for local appearance properties of a part of the body while the deformable configuration is presented by spring-like connections between parts. The model focuses primarily on the case of star-structured models and show how a simple algorithm based on partial hypothesis pruning can speed up object detection. A discriminative procedure called Latent SVM is used to train these models. Lots of experiments are conducted on public data sets TUD-Crossing and TUD-Pedestrian. Experimental results show that little detection accuracy is increased by our algorithm, and the detection speed is improved obviously.