Abstract: To resolve the oblique image relative orientation problem, in this paper a novel method for relative orientation is proposed based on a fundamental matrix. Firstly, the fundamental matrix is constructed by direct linear transformation, then the elements of relative orientation are deduced with an optimal matrix. Accuracy factors are analyzed for this model, and finally, to get a more accurate solution deploying a rigorous model for relative orientation to the y-parallax, an iterative method is employed that takes elements derived from the essential matrix as the approximate values of the relative orientation elements. Experimental results show that our proposed method is feasible for both traditional aerial images and oblique images, and meets the requirements for stereo-model reconstruction in photogrammetric operations.