Text Modality Assisted Guided Mural Inpainting Algorithm

Objectives: In order to solve the problems of existing deep learning algorithms in the inpainting of murals, which only consider the prior information of mural images to guide the inpainting of damaged areas, and lack of text information to guide the inpainting of murals, resulting in semantic inconsistency and lack of details in the inpainting results, a text modality-assisted guided mural inpainting algorithm is proposed. Methods: First, a text modality-assisted guided mural inpainting network is proposed, which uses the text information as the control guidance for mural inpainting, and provides context-guided repair information for mural images. Secondly, a text filtering module is constructed to obtain the text features of the damaged area by filtering the mask image and the complementary image, and a cross-modal semantic enhancement module is designed to enhance the filtered text features to improve the consistency between text semantics and image semantics. Then, the upsampled texture detail inpainting network is designed to achieve bidirectional fusion of shallow and deep features to obtain mural images with fine-grained features. Finally, the normalized discriminator is used to match the recovered murals to the game, and the recovered murals are obtained. Results: The experimental results of real mural restoration show that this method can effectively repair damaged murals, and is superior to comparative algorithms in subjective and objective evaluation. Conclusions: The proposed method can effectively repair damaged murals, achieving better visual perception and coordination.