| Citation: |
ZHANG Zhiqi, WANG Mi, CAO Jinshan, LIU Chuang, LIAO Dunbo. Object-Space-Consistency-Based Real-Time Stabilization Approach for Luojia3-01 Video Data[J]. Geomatics and Information Science of Wuhan University, 2024, 49(6): 899-910. DOI: 10.13203/j.whugis20230329
|
High-resolution agile optical satellites can continuously observe specific regions or targets, thereby realizing applications such as regional monitoring and target tracking. In order to obtain stable and high-quality video data, continuous observation data need to be geographically corrected and stabilized. This process requires a large amount of calculation, but the on-board computing and storage device cannot meet the demand. Therefore, this process can only be completed afterwards by on-ground system, resulting in a significant delay in information acquisition.
This paper proposes an object-space-consistency-based real-time stabilization approach for Luojia3-01 satellite video data, which is based on data characteristics and computing capability of embedded on-board hardware. The proposed approach employs real-time geometric positioning to extract and correct the region of interest (ROI) in real-time during satellite imaging. Additionally, it employs the consistency of geographical information in real-time align adjacent image frames to ensure the relative accuracy between sequence frames. On the basis, it generates a sequence of video frames of ROI with geographical encoding in real-time through the construction of a parallel pipeline of algorithms.
The experiments are conducted by the actual data of Luojia3-01 satellite with the frame rate of 6 frame per second, and the proposed approach can realize the faster processing based on the embedded on-board processing hardware. Furthermore, the obtained video frame sequence with geographic encoding and the inter-frame stabilized accuracy is better than 0.328 pixels.
The proposed approach can meet the requirements of on-board real-time service.
| [1] |
Dial G, Bowen H, Gerlach F, et al. IKONOS Satellite, Imagery, and Products[J]. Remote Sensing of Environment, 2003, 88（1/2）： 23-36.
|
| [2] |
李德仁. 我国第一颗民用三线阵立体测图卫星： 资源三号测绘卫星[J]. 测绘学报, 2012, 41（3）： 317-322.
Li Deren. China’s First Civilian Three-Line-Array Stereo Mapping Satellite： ZY-3[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41（3）： 317-322.
|
| [3] |
史伟国, 周立民, 靳颖. 全球高分辨率商业遥感卫星的现状与发展[J]. 卫星应用, 2012（3）： 43-50.
Shi Weiguo, Zhou Limin, Jin Ying. Present Situation and Development of Global High-Resolution Commercial Remote Sensing Satellites[J]. Satellite Application, 2012（3）： 43-50.
|
| [4] |
王景泉. 商业遥感卫星市场的现状与发展趋势[J]. 卫星应用, 2012（1）： 49-53.
Wang Jingquan. Present Situation and Development Trend of Commercial Remote Sensing Satellite Market[J]. Satellite Application, 2012（1）： 49-53.
|
| [5] |
杨靖宇. 摄影测量数据GPU并行处理若干关键技术研究[D]. 郑州： 信息工程大学, 2011.
Yang Jingyu. Study on Parallel Processing Technologies of Photogrammetry Data Based on GPU[D].Zhengzhou： Information Engineering University, 2011.
|
| [6] |
李德仁, 沈欣, 龚健雅, 等. 论我国空间信息网络的构建[J]. 武汉大学学报（信息科学版）, 2015, 40（6）： 711-715.
Li Deren, Shen Xin, Gong Jianya, et al. On Construction of China’s Space Information Network[J]. Geomatics and Information Science of Wuhan University, 2015, 40（6）： 711-715.
|
| [7] |
李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45（4）： 379-384.
Li Deren. Towards Geo-spatial Information Science in Big Data Era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45（4）： 379-384.
|
| [8] |
李德仁. 从珞珈系列卫星到东方慧眼星座[J]. 武汉大学学报（信息科学版）, 2023,48（10）： 1557-1565.
Li Deren. From the Luojia Series Satellites to the Oriental Smart Eye Constellation[J]. Geomatics and Information Science of Wuhan University,2023,48（10）： 1557-1565.
|
| [9] |
张兵. 智能遥感卫星系统[J]. 遥感学报, 2011, 15（3）： 415-431.
Zhang Bing. Intelligent Remote Sensing Satellite System[J]. National Remote Sensing Bulletin, 2011, 15（3）： 415-431.
|
| [10] |
张博. 基于多DSP的高可靠并行星载计算机系统关键技术研究[D]. 上海： 上海交通大学, 2014.
Zhang Bo. A Research on Key Technologies for the High Reliability On-board Parallel System Based on Multiple DSPs[D].Shanghai： Shanghai Jiao Tong University, 2014.
|
| [11] |
Plante J, Shaw H. Evaluation of 3D Plus Packaging Test Structures for NASA Goddard Space Flight Center[C]//European Space Components Conference, Toulouse, France, 2022.
|
| [12] |
卫宁, 郭红霞, 于伦正,等. 二次包封CMOS器件电子辐照实验研究[J]. 空间科学学报, 2003, 23（5）： 396-400.
Wei Ning, Guo Hongxia, Yu Lunzheng, et al. Experimental Study of CMOS Device of Secondary Package with Electron Irradiation [J]. Chinese Journal of Space Science, 2003, 23（5）： 396-400.
|
| [13] |
王一奇, 赵发展, 刘梦新, 等. 基于RHBD技术的深亚微米抗辐射SRAM电路的研究[J]. 半导体技术, 2012, 37（1）： 18-23.
Wang Yiqi, Zhao Fazhan, Liu Mengxin, et al. Study on the Deep Submicron Radiation Hardened SRAM Circuit for RHBD Technologies[J]. Semiconductor Technology, 2012, 37（1）： 18-23.
|
| [14] |
Franklin A D, Koswatta S O, Farmer D, et al. Scalable and Fully Self-Aligned N-type Carbon Nanotube Transistors with Gate-all-Around[C]//International Electron Devices Meeting, San Francisco, USA, 2012.
|
| [15] |
Nan H Q, Wang W, Choi K. Circuit Design for Carbon Nanotube Field Effect Transistors[C]//International SoC Design Conference, Jeju, South Korea, 2012.
|
| [16] |
Zhang Z Q, Qu Z, Liu S Y, et al. Expandable On-board Real-Time Edge Computing Architecture for Luojia3 Intelligent Remote Sensing Satellite[J]. Remote Sensing, 2022, 14（15）： 3596.
|
| [17] |
Zhang Z Q, Wei L, Xiang S, et al. Task-Driven Onboard Real-Time Panchromatic Multispectral Fusion Processing Approach for High-Resolution Optical Remote Sensing Satellite[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16： 7636-7661.
|
| [18] |
满益云, 李海超. 低轨视频卫星成像特性分析[J]. 航天器工程, 2015, 24（5）： 52-57.
Man Yiyun, Li Haichao. Imaging Characteristics Analysis for LEO Video Satellite[J]. Spacecraft Engineering, 2015, 24（5）： 52-57.
|
| [19] |
吉淑娇, 朱明, 胡汉平. 基于特征点匹配的电子稳像技术[J]. 中国光学, 2013, 6（6）： 841-849.
Ji Shujiao, Zhu Ming, Hu Hanping. Aero-Borne Electronic Image Stabilization Based on Feature Point Matching[J]. Chinese Optics, 2013, 6（6）： 841-849.
|
| [20] |
刘祥磊, 童小华, 马静. 视频测量影像序列椭圆形人工目标点快速识别和跟踪方法[J]. 测绘学报, 2015, 44（6）： 663-669.
Liu Xianglei, Tong Xiaohua, Ma Jing. A Systemic Algorithm of Elliptical Artificial Targets Identification and Tracking for Image Sequences from VideoGrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44（6）： 663-669.
|
| [21] |
Kim H S, Lee J H, Kim C K, et al. Zoom Motion Estimation Using Block-Based Fast Local Area Scaling[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22（9）： 1280-1291.
|
| [22] |
Kim T, Lee S, Paik J. Combined Shape and Feature-Based Video Analysis and Its Application to Non-rigid Object Tracking[J]. IET Image Processing, 2011, 5（1）： 87.
|
| [23] |
易盟. 基于特征点的图像配准及其在稳像中的应用[D]. 西安： 西安电子科技大学, 2013.
Yi Meng. Image Registration Based on Feature Point and Its Application to Electronic Image Stabilization[D].Xi’an： Xidian University, 2013.
|
| [24] |
Lim A, Ramesh B, Yang Y, et al. Real-Time Optical Flow-Based Video Stabilization for Unmanned Aerial Vehicles[J]. Journal of Real⁃Time Image Processing, 2019, 16（6）： 1975-1985.
|
| [25] |
Erturk S. Digital Image Stabilization with Sub-image Phase Correlation Based Global Motion Estimation[J]. IEEE Transactions on Consumer Electronics, 2003, 49（4）： 1320-1325.
|
| [26] |
Walha A, Wali A, Alimi A M. Video Stabilization with Moving Object Detecting and Tracking for Aerial Video Surveillance[J]. Multimedia Tools and Applications, 2015, 74（17）： 6745-6767.
|
| [27] |
Kumar S, Azartash H, Biswas M, et al. Real-Time Affine Global Motion Estimation Using Phase Correlation and Its Application for Digital Image Stabilization[J]. IEEE Transactions on Image Processing： A Publication of the IEEE Signal Processing Society, 2011, 20（12）： 3406-3418.
|
| [28] |
张毛磊, 陈建国, 袁宏永, 等. 六旋翼飞行平台的视频稳像技术[J]. 清华大学学报（自然科学版）, 2014, 54（11）： 1412-1416.
Zhang Maolei, Chen Jianguo, Yuan Hongyong, et al. Video Stabilization on a Six-Rotor Aircraft Platform[J]. Journal of Tsinghua University （Science and Technology）, 2014, 54（11）： 1412-1416.
|
| [29] |
Matsushita Y, Ofek E, Ge W N, et al. Full-Frame Video Stabilization with Motion Inpainting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28（7）： 1150-1163.
|
| [30] |
Aguilar W G, Angulo C. Real-Time Model-Based Video Stabilization for Microaerial Vehicles[J]. Neural Processing Letters, 2016, 43（2）： 459-477.
|
| [31] |
Zhang G F, Hua W, Qin X Y, et al. Video Stabilization Based on a 3D Perspective Camera Model[J]. The Visual Computer, 2009, 25（11）： 997-1008.
|
| [32] |
Murthy K, Shearn M, Smiley B D, et al. SkySat-1： Very High-Resolution Imagery from a Small Satellite[C]// Sensors, Systems, and Next-Generation Satellites Conference, Amsterdam, Netherlands,2014.
|
| [33] |
李锋. 空间红外对地观测视频相机电子稳像与目标跟踪算法研究[D]. 上海： 中国科学院上海技术物理研究所, 2018.
Li Feng. Research on the Electronic Image Stabilization and Target Tracking Algorithm for Space Infrared Earth Observation Video Camera[D].Shanghai： Shanghai Institute of Technical Physics, Chinese Academy of Sciences, 2018.
|
| [34] |
Wang M, Yang B, Hu F, et al. On-orbit Geometric Calibration Model and Its Applications for High-Resolution Optical Satellite Imagery[J]. Remote Sensing, 2014, 6（5）： 4391-4408.
|
| [35] |
Xu W, Gong J Y, Wang M. Development, Application, and Prospects for Chinese Land Observation Satellites[J]. Geo⁃spatial Information Science, 2014, 17（2）： 102-109.
|
| [36] |
王兵学, 雍杨, 唐黎, 等. 基于相关系数拟合的亚像素定位方法[J]. 红外技术, 2010, 32（11）： 629-631.
Wang Bingxue, Yong Yang, Tang Li, et al. Sub-pixel Localization Method Based on Relative Coefficient Fitting[J]. Infrared Technology, 2010, 32（11）： 629-631.
|
| [37] |
李德仁, 王密, 杨芳. 新一代智能测绘遥感科学试验卫星珞珈三号01星[J]. 测绘学报, 2022, 51（6）： 789-796.
Li Deren, Wang Mi, Yang Fang. A New Generation of Intelligent Mapping and Remote Sensing Scientific Test Satellite Luojia-3 01[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51（6）： 789-796.
|
| [38] |
王密, 杨芳. 智能遥感卫星与遥感影像实时服务[J]. 测绘学报, 2019, 48（12）： 1586-1594.
Wang Mi, Yang Fang. Intelligent Remote Sensing Satellite and Remote Sensing Image Real-Time Service[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48（12）： 1586-1594.
|
| [39] |
周楠, 曹金山, 肖蕾, 等. 带有地理编码的光学视频卫星物方稳像方法[J]. 武汉大学学报（信息科学版）, 2023, 48（2）： 308-315.
Zhou Nan, Cao Jinshan, Xiao Lei, et al. A Geo-coded Stabilization Approach for Optical Video Satellites in Object Space[J]. Geomatics and Information Science of Wuhan University, 2023, 48（2）： 308-315.
|
| [1] | GONG Xuewen, WANG Fuhong. Impact of Multipath Error and Noise of Space-Borne GPS Code Measurements on Real-Time Onboard Orbit Determination[J]. Geomatics and Information Science of Wuhan University, 2018, 43(7): 1048-1055. DOI: 10.13203/j.whugis20160223 |
| [2] | GONG Xuewen, WANG Fuhong. Autonomous Orbit Determination of HY2A and ZY3 Missions Using Space-borne GPS Measurements[J]. Geomatics and Information Science of Wuhan University, 2017, 42(3): 309-313. DOI: 10.13203/j.whugis20140892 |
| [3] | ZHOU Xuhua, WANG Xiaohui, ZHAO Gang, PENG Hailong, WU Bin. The Precise Orbit Determination for HY2A Satellite Using GPS,DORIS and SLR Data[J]. Geomatics and Information Science of Wuhan University, 2015, 40(8): 1000-1005. DOI: 10.13203/j.whugis20130730 |
| [4] | MA Yang, OU Jikun, YUAN Yunbin, HUO Xingliang, DING Wenwu. Estimation of GPS Antenna Phase Center Variation and Its Effect on Precise Orbit Determination of LEOs[J]. Geomatics and Information Science of Wuhan University, 2015, 40(7): 894-900. DOI: 10.13203/j.whugis20130626 |
| [5] | QIN Jian, GUO Jinyun, KONG Qiaoli, LI Guowei. Precise Orbit Determination of Jason-2with Precision of CentimetersBased on Satellite-borne GPS Technique[J]. Geomatics and Information Science of Wuhan University, 2014, 39(2): 137-141. DOI: 10.13203/j.whugis20120686 |
| [6] | WANG Fuhong, XU Qichao, GONG Xuewen, ZHANG Wei. Application of a Gravity Acceleration Approximation Function in the PreciseReal-Time Orbit Determination Using Space-borne GPS Measurements[J]. Geomatics and Information Science of Wuhan University, 2014, 39(1): 47-51. |
| [7] | LI Wenwen, LI Min, SHI Chuang, ZHAO Qile. Jason-2 Precise Orbit Determination Using DORIS RINEX Phase Data[J]. Geomatics and Information Science of Wuhan University, 2013, 38(10): 1207-1211. |
| [8] | GUO Jing, ZHAO Qile, LI Min, HU Zhigang. Centimeter Level Orbit Determination for HY2A Using GPS Data[J]. Geomatics and Information Science of Wuhan University, 2013, 38(1): 52-55. |
| [9] | WANG Fuhong. A Kalman Filtering Algorithm for Precision Real-Time Orbit Determination with Space-Borne GPS Measurements[J]. Geomatics and Information Science of Wuhan University, 2010, 35(6): 653-656. |
| [10] | GENG Jianghui, SHI Chuang, ZHAO Qile, LIU Jingnan. GPS Precision Orbit Determination from Combined Ground and Space-borne Data[J]. Geomatics and Information Science of Wuhan University, 2007, 32(10): 906-909. |
| 1. |
高贤君，冉树浩，张广斌，杨元维. 基于多特征融合与对象边界联合约束网络的建筑物提取. 武汉大学学报(信息科学版). 2024(03): 355-365 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: