Abstract: A method of video features correlation analysis is proposed,by using the latent self-correlations and cross-correlation which exist in the visual features and semantics,it can retain the core element of the semantic structure in video content,and eliminate the interference induced by the redundant relationships.It is proved in practice that proposed method make better outcome of semantic-based video retrieval,and to improve the computational efficiency by reducing the size of matrix.