Abstract: This paper presents a novel approach to estimate the fundamental matrix. The proposed method takes the traditional RANSAC method as the basic framework and selects the optimum inlier set combined with the inliers distribution and the average distance of the points to the epipolar lines. Then the M-Estimators method is used to compute the fundamental matrix. Experiment on a large number of simulated and real image data show that our algorithm can achieve a more precise estimation of the fundamental matrix.