Abstract: A method of balancing the optimize accuracy and efficiency of the mesh to achieve mesh adaptive fast optimization is proposed to solve the problem of low efficiency of mesh optimization in the existing 3D reconstruction based on images. After calculating the initial value of the vertex gradient along the triangular normal vector based on the image gray information, the mesh of each triangle is marked differently according to the different results:The active triangles and the lazy triangles, through the active in the mesh is refined to abandon the optimization of the lazy triangles in exchanging for improvement of efficiency. The optimization of the efficiency is fast improving while obtaining better mesh vertices. We choose classical images church and fountain data in the 3D reconstruction experiment of computer vision and the widely used of unmanned aerial vehicle (UAV) images in photogrammetry to reconstruct mesh to refine. The mesh model optimization time and precision comparison results show that this algorithm can greatly improve the mesh optimize efficiency to obtain the 3D model faster.