Abstract: In the existing studies of video synopsis, the lack of trajectory geographic direction and partition expression forthe moving objects' trajectories is the key problem yet to be solved.A novel approach for surveillance video synopsis was proposed, in which trajectory geographic direction was considered.The homography model was used for mapping the trajectory from image space to geographic space, and the trajectory analyzation of geographic direction was carried out by clustering the Trajectories. On the basis of mapping and clustering, the video synopsis background was built by selecting the virtual scene viewpoint, and moving objects' display trajectory was determined by creating the fitting centerline, and the expressing order was determined by creating the expressing model of the cluster. Finally the video synopsis was completed. By conducting several experiments, the effectiveness and flexibility of the approach is validated.