王新晟, 王密, 潘俊, 肖晶, 李路, 张昊楠, 赵简平. 珞珈三号01星视频智能插帧应用研究[J]. 武汉大学学报 ( 信息科学版). DOI: 10.13203/j.whugis20230439
引用本文: 王新晟, 王密, 潘俊, 肖晶, 李路, 张昊楠, 赵简平. 珞珈三号01星视频智能插帧应用研究[J]. 武汉大学学报 ( 信息科学版). DOI: 10.13203/j.whugis20230439
WANG Xinsheng, WANG Mi, PAN Jun, XIAO Jing, LI Lu, ZHANG Haonan, ZHAO Jianping. Application of Intelligent Video Frame Interpolation for Luojia3-01 Satellite[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20230439
Citation: WANG Xinsheng, WANG Mi, PAN Jun, XIAO Jing, LI Lu, ZHANG Haonan, ZHAO Jianping. Application of Intelligent Video Frame Interpolation for Luojia3-01 Satellite[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20230439

珞珈三号01星视频智能插帧应用研究

Application of Intelligent Video Frame Interpolation for Luojia3-01 Satellite

  • 摘要: 近20年来世界各国普遍开展了光学视频卫星的研制工作,光学视频卫星获取的高动态、高空间分辨率的数据为对地动态观测提供了新的技术手段。由武汉大学牵头研制的新一代智能测绘遥感科学试验卫星“珞珈三号01星”于2023年年初成功发射入轨,该星通过凝视成像模式,可获取对地彩色高清视频,但原始视频帧率较低,仅为6帧/s。为进一步提升珞珈三号01星视频的流畅度,降低视觉观感的卡顿效果,开展了面向珞珈三号01星视频插帧的相关研究。首先,针对卫星凝视成像过程中的误差进行了分析,提出一种基于帧间透视变换模型的视频稳像方法,实现了原始视频数据的预处理;其次,考虑到日常可见光视频与卫星视频之间存在较大差异,且目前暂无可用的卫星视频插帧数据集。基于预处理后的稳像视频,构建了一个涵盖不同场景的卫星插帧数据集,命名为Luojia3_VFISet。最后,基于无需光流模块参与的FLAVR视频插帧网络,本文通过将不同尺度的特征编码信息引入解码过程,提出了FLAVR_Plus视频插帧网络,进一步提升了卫星视频的插帧效果。实验结果表明,FLAVR_Plus网络插帧结果的PSNR指标达到35.5446 dB,精度提升约0.5%~7.2%,SSIM指标可达0.9179,同比提升约0.5%~8.7%。本文研究以珞珈三号01星为例,针对卫星视频插帧应用展开了研究,构建的卫星视频插帧数据集有助于相关研究工作的开展,提出的插帧网络针对不同场景均能生成质量良好,无明显拖影的中间帧,可有效地提升珞珈三号01星视频的流畅度,可为后续的卫星视频相关应用提供更多的帧间信息。

     

    Abstract: Objectives: Optical video satellites have been developed in many countries over the past two decades. The dynamic and high spatial resolution data obtained by optical video satellites provide a new technical means for the dynamic observation of the earth. Led by Wuhan University, a new generation of intelligent mapping remote sensing scientific test satellite "Luojia3-01" was successfully launched into orbit in early 2023, achieving the acquisition of color high-definition satellite video through gaze mode. However, the original video frame rate is low, only 6 frame per-second (fps). To further improve the fluency and reduce the stuttering effect of visual perception, this paper carries out related research on intelligent video frame interpolation (VFI) of Luojia3-01 satellite video. Methods: First, a satellite video image stabilization method based on the perspective transformation model is proposed after analyzing the imaging errors during the gaze mode. Second, considering that there are great differences between daily video and satellite video, and there is no available satellite VFI dataset at present. This paper built a VFI dataset termed as Luojia3_VFISet based on the stabilized satellite videos, which covers different scenes. Finally, based on the FLAVR VFI network without optical flow module, the FLAVR_Plus VFI network is proposed to further improve the interpolation effect for satellite video by introducing the feature coding information of different scales from the encoder into the decoding process. Results: The experimental results show that interpolation of the FLAVR_Plus network raises the PSNR and SSIM to 35.5446 dB and 0.9179 respectively and enhances the quality of synthesized intermediate frames with hardly-observed artifacts under different scenarios. Compared with other methods, the proposed network can improve the PSNR by 0.5% to 7.2% and the SSIM by 0.5% to 8.7%. Conclusion: In the paper, the application of satellite VFI is studied by taking “Luojia3-01” satellite as an example. The proposed Luojia3_VFISet dataset is conducive to the development of related research. The proposed FLAVR_Plus VFI network can effectively improve the fluency of the Loujia3-01 satellite video by generating interframes without artifacts and provide more interframe information for subsequent applications.

     

/

返回文章
返回