Application of Intelligent Video Frame Interpolation for Luojia3-01 Satellite

Objectives: Optical video satellites have been developed in many countries over the past two decades. The dynamic and high spatial resolution data obtained by optical video satellites provide a new technical means for the dynamic observation of the earth. Led by Wuhan University, a new generation of intelligent mapping remote sensing scientific test satellite "Luojia3-01" was successfully launched into orbit in early 2023, achieving the acquisition of color high-definition satellite video through gaze mode. However, the original video frame rate is low, only 6 frame per-second (fps). To further improve the fluency and reduce the stuttering effect of visual perception, this paper carries out related research on intelligent video frame interpolation (VFI) of Luojia3-01 satellite video. Methods: First, a satellite video image stabilization method based on the perspective transformation model is proposed after analyzing the imaging errors during the gaze mode. Second, considering that there are great differences between daily video and satellite video, and there is no available satellite VFI dataset at present. This paper built a VFI dataset termed as Luojia3_VFISet based on the stabilized satellite videos, which covers different scenes. Finally, based on the FLAVR VFI network without optical flow module, the FLAVR_Plus VFI network is proposed to further improve the interpolation effect for satellite video by introducing the feature coding information of different scales from the encoder into the decoding process. Results: The experimental results show that interpolation of the FLAVR_Plus network raises the PSNR and SSIM to 35.5446 dB and 0.9179 respectively and enhances the quality of synthesized intermediate frames with hardly-observed artifacts under different scenarios. Compared with other methods, the proposed network can improve the PSNR by 0.5% to 7.2% and the SSIM by 0.5% to 8.7%. Conclusion: In the paper, the application of satellite VFI is studied by taking “Luojia3-01” satellite as an example. The proposed Luojia3_VFISet dataset is conducive to the development of related research. The proposed FLAVR_Plus VFI network can effectively improve the fluency of the Loujia3-01 satellite video by generating interframes without artifacts and provide more interframe information for subsequent applications.