Abstract: Objectives: Pixel offset tracking (POT) for optical remote sensing imagery is a widely used approach to inverting co-seismic deformation fields and monitoring landslides. The traditional pixel offset tracking method estimates the displacement of the central pixel by searching for the matching window with the highest correlation, which is computationally inefficient and suffers from inaccurate deformation boundary extraction due to the decoherence effects in the region with dynamical deformation. This paper introduces the optical flow field model commonly used in computer vision to the pixel offset tracking problem to obtain accurate surface deformation efficiently. Methods: The optical flow field method applicable to optical remote sensing images and the improved inversion algorithm for the time series analysis are proposed to inverse the surface deformation. Experiments on the simulated co-seismic deformation fields in Tajikistan are detailed to assess the feasibility and the minimum detectable deformation of the optical flow field method. The advantages of the proposed method over computational cost and deformation boundary extraction accuracy are illustrated by the coseismic deformation field of the California earthquake and the displacement of the Baige landslide. Furthermore, the performance on estimating large gradient deformation and the robustness of the improved time series inversion algorithm are discussed by analyzing the timeseries deformation of the Baige landslide. Results: The results show that compared with the traditional window correlation matching method, the optical flow field method has an offset tracking accuracy of 0.032 pixels, which improves the computational efficiency by about 20 times, and the accuracy of the deformation zone is improved by 26%. The time-series weighted inversion algorithm reduces the uncertainties in the estimation of east-west and north-south displacements of optical remote sensing images by 16.2% and 12.4%, respectively. Conclusions: Our method alleviates the pixel offset tracking problem in the boundary region with large gradient deformation.