Abstract: The traditional approach for relative orientation is the least-square solution of coplanar equation with more than five correspondence rays when the approximations acquired by direct methods.We present a hybrid genetic algorithm which utilizes the global convergence of the traditional genetic algorithm and the local convergence of the gradient-based search algorithms.And unit quaternion is used to represent 3D rotation instead of traditional Euler angles to enlarge the convergent region.The algorithm can effectively eliminate the premature of the traditional genetic algorithm and rapidly converge to the optimal solutions without initial values.