Objectives Emergency remote sensing mapping can provide support for decision-making in disaster assessment or disaster relief, and therefore plays an important role in disaster response.Traditional emergency remote sensing mapping methods use the decryption algorithms based on manual retrieval and image editing tools when processing sensitive targets. Although the traditional methods can achieve target recognition, they are inefficient and cannot meet the immediate requirements of disaster relief, which are unable to be released or applied in time. The main objective is to propose a method for auto-detecting and hiding of sensitive targets in emergency remote sensing mapping to accelerate the rapid production and release emergency remote sensing products.
Methods Because of the huge size of remote sensing images, it is not realistic to directly hide sensitive targets. We propose a two-stage processing method automatic target detection and hiding of sensitive targets method, which consists of two neural network models: target detection model and generative adversarial network model. Firstly, Mask R-CNN, a well-known and effective target detection method, was used to detect sensitive targets from massive remote sensing data and generate target coordinates and Masks. Then, Deepfill model, one of GAN(generative adversarial networks) models, can ignore other normal areas and directly hide sensitive objects in the local area based on the coordinates and Masks information. The aircraft objects in the remote sensing image was used as an application example to verify the feasibility of our method, furthermore, we added the reconstruction of loss function, candidate frame optimization of region recommendation network, Mask optimization algorithm, and attention mechanism reconstruction based on the characteristics of the aircraft objects. Mask R-CNN model and Deepfill model have different training principles, so we trained and tuned them separately, and finally combine the trained models. We randomly extracted images from RSOD(remote sensing object detection) and DOTA(a large-scale dataset for object detection in aerial images) to form a new dataset. A total of 1 607 images were obtained for Mask R-CNN model training, and 9 502 images were used for Deepfill model training. All these samples are divided into training set and verification set according to the ratio of 0.8: 0.2. The performance of the Mask R-CNN model was evaluated by precision, recall rate, missing detect rate and F1-score; the performance evaluation indicators of the Deepfill model are PSNR(peak signal to noise ratio) and SSIM(structural similarity).
Results 46 images were extracted separately from the original dataset to test the performance of the trained models. In the target detect stage, the accuracy of the benchmark model was 98.13%, the recall rate was 44.21%, the missed detection rate was 55.79%, and the F1-score was 60.96%. Many targets were not detected. For comparison, the accuracy of our method reaches 94.65%, the recall rate reaches 81.89%, which is 85.23% higher than the benchmark model; the missed detection rate reaches 18.11%, which is 67.54% lower than the benchmark model; the F1-score reaches 87.81%, which is 44.05% higher than the benchmark model. In the inpainting stage, the average PSNR in this method reaches 32.26, and the average SSIM value is 0.98.
Conclusions In the proposed method, the recall rate and F1-score of aircraft targets in remote sensing images have been significantly improved, the inpainting processing effect is reasonable and natural, and the overall time of the emergency remote sensing mapping process is saved, indicating that the two-stage model works well. In the future, it can further expand the detection and processing of other sensitive targets, accelerate the production and release efficiency of disaster emergency response map products, and thus improve the ability of disaster prevention and relief.
DownLoad: