Abstract: UAV platform instability causes large geometric deformation in UAV images and unsatisfactory matching accuracy. To address this problem, Point Cloud Optimization for UAV images based on a non-fixed initial patch algorithm is proposed in this paper. By using the difference to calculate approximate normal vector of local tangent plane instead of the differential, and by using this approximate normal vector to establish the initial patch. Two groups of images in campus of Northwestern University and Yangjiang Area were used to test this method. Experimental results show that the Point Cloud Optimization for UAV Image based on a non-fixed initial patch algorithm improved the Patch-based Least Squares Image Matching method, and was superior to the optimization method in PMVS. This method increased the efficiency and the accuracy of point cloud optimization.