Message Board

Respected readers, authors and reviewers, you can add comments to this page on any questions about the contribution, review,        editing and publication of this journal. We will give you an answer as soon as possible. Thank you for your support!

Name
E-mail
Phone
Title
Content
Verification Code
Turn off MathJax
Article Contents

WEI Haodong, YI Yaohua, YU Changhui, LIN Liyu. Text Super-resolution Method with Attentional Mechanism and Sequential Units[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220158
Citation: WEI Haodong, YI Yaohua, YU Changhui, LIN Liyu. Text Super-resolution Method with Attentional Mechanism and Sequential Units[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220158

Text Super-resolution Method with Attentional Mechanism and Sequential Units

doi: 10.13203/j.whugis20220158
  • Received Date: 2022-07-14
    Available Online: 2022-08-19
  • Objectives: The text in street view images is the clue to perceive and understand scene information. Low-resolution street view images lack details in the text region, leading to poor recognition accuracy. Super-resolution can be introduced as pre-processing to reconstruct edge and texture details of the text region. To improve text recognition accuracy, we propose a text super-resolution network combining attentional mechanism and sequential units. Methods: A hybrid residual attention structure is proposed to extract spatial information and channel information of the image text region, learning multi-level feature representation. A sequential unit is proposed to extract sequential prior information between texts in the image through bidirectional gated recurrent units. Using gradient prior knowledge as the constraint, a gradient prior loss is designed to sharpen character boundaries. Results and Conclusions: In order to verify the effectiveness of the proposed method, we use real scene text images in TextZoom and synthetic text images to carry out comparative analysis experiments. Experimental results show that the proposed method can reconstruct clear text edges and rich text texture details, and improve text recognition accuracy of street view images.
  • [1] Wang W J, Xie E Z, Sun P Z, et al. TextSR:Content-Aware Text Super-Resolution Guided by Recognition, 2019[OL]. https://arxiv.org/pdf/1909.07113.pdf, 2022
    [2] Wang W J, Xie E Z, Liu X B, et al. Scene Text Image Super-Resolution in the Wild[M]//Computer Vision-ECCV 2020. Cham:Springer International Publishing, 2020:650-666
    [3] Dong C, Zhu X M, Deng Y B, et al. Boosting Optical Character Recognition:A Super-Resolution Approach, 2015[OL]. https://arxiv.org/pdf/1506.02211.pdf, 2022
    [4] Peyrard C, Baccouche M, Mamalet F, et al. ICDAR2015 competition on text image super-resolution[C]//201513th International Conference on Document Analysis and Recognition (ICDAR). Tunis, Tunisia.:1201-1205
    [5] Dong C, Loy C C, He K M, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307
    [6] Pandey R K, Vignesh K, Ramakrishnan A G, et al. Binary Document Image Super Resolution for Improved Readability and OCR Performance, 2018[OL]. https://arxiv.org/pdf/1812.02475.pdf, 2022
    [7] Nakao R, Iwana B K, Uchida S. Selective super-resolution for scene text images[C]//2019 International Conference on Document Analysis and Recognition (ICDAR). Sydney, NSW, Australia.:401-406
    [8] Lin K, Liu Y B, Li T H, et al. Text image super-resolution by image matting and text label supervision[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Long Beach, CA, USA.:1722-1727
    [9] Wang Z H, Chen J, Hoi S C H. Deep Learning for Image Super-Resolution:A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10):3365-3387
    [10] Liao Haibin, Chen Youbin, Chen Qinghu. Non-Local Similarity Dictionary Learning Based Super-Resolution for Improved Face Recognition[J]. Geomatics and Information Science of Wuhan University, 2016, 41(10):1414-1420
    [11] Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, HI, USA.:1132-1140
    [12] Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need, 2017[OL]. https://arxiv.org/pdf/1706.03762.pdf, 2022
    [13] Fu J, Liu J, Tian H J, et al. Dual attention network for scene segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA.:3141-3149
    [14] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image Is Worth 16x16 Words:Transformers for Image Recognition at Scale, 2021[OL]. https://arxiv.org/pdf/2010.11929.pdf, 2022
    [15] Zhao H S, Jia J Y, Koltun V. Exploring self-attention for image recognition[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA.:10073-10082
    [16] Zhang Y L, Li K P, Li K, et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks[C]//Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany, 2018
    [17] Muqeet A, Iqbal M T B, Bae S H. HRAN:Hybrid Residual Attention Network for Single Image Super-Resolution[J]. IEEE Access, 7:137020-137029
    [18] Wang Y Y, Su F, Qian Y. Text-attentional conditional generative adversarial network for super-resolution of text images[C]//2019 IEEE International Conference on Multimedia and Expo. Shanghai, China.:1024-1029
    [19] Anwar S, Barnes N. Densely Residual Laplacian Super-Resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3):1192-1204
    [20] Shi W Z, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA.:1874-1883
    [21] He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition, 2015[OL]. https://arxiv.org/pdf/1512.03385.pdf, 2022
    [22] Li J C, Xie E Z, Fang F M. Multi-Scale Residual Network for Image Super-Resolution[C]//Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany, 2018
    [23] Shi B G, Yang M K, Wang X G, et al. ASTER:An Attentional Scene Text Recognizer with Flexible Rectification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(9):2035-2048
    [24] Sun J, Sun J, Xu Z B, et al. Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement[J]. IEEE Transactions on Image Processing, 2011, 20(6):1529-1542
    [25] Tran H T M, Phuoc T H. Deep Laplacian Pyramid Network for Text Images Super-Resolution[J]//Proceedings of the 2019 IEEE-RIVF International Conference on Computing and Communication Technologies, Danang, Vietnam, 2019
    [26] Jaderberg M, Simonyan K, Zisserman A, et al. Spatial Transformer Networks[C]//Proceedings of the 29th Annual Conference on Neural Information Processing Systems, Montreal, Canada, 2015
    [27] Lai W S, Huang J B, Ahuja N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017
    [28] Luo C J, Jin L W, Sun Z H. MORAN:A Multi-Object Rectified Attention Network for Scene Text Recognition[J]. Pattern Recognition, 2019, 90:109-118
    [29] Shi B G, Bai X, Yao C. An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(11):2298-2304
    [30] Geng C, Chen L, Zhang X, et al. Adversarial Text Image Super-Resolution using Sinkhorn Distance[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
    [31] Xue M, Huang Z, Liu R, et al. A Novel Attention Enhanced Residual-In-Residual Dense Network for Text Image Super-Resolution[C]//2021 IEEE International Conference on Multimedia and Expo (ICME), 2021.
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Article Metrics

Article views(140) PDF downloads(20) Cited by()

Related
Proportional views

Text Super-resolution Method with Attentional Mechanism and Sequential Units

doi: 10.13203/j.whugis20220158

Abstract: Objectives: The text in street view images is the clue to perceive and understand scene information. Low-resolution street view images lack details in the text region, leading to poor recognition accuracy. Super-resolution can be introduced as pre-processing to reconstruct edge and texture details of the text region. To improve text recognition accuracy, we propose a text super-resolution network combining attentional mechanism and sequential units. Methods: A hybrid residual attention structure is proposed to extract spatial information and channel information of the image text region, learning multi-level feature representation. A sequential unit is proposed to extract sequential prior information between texts in the image through bidirectional gated recurrent units. Using gradient prior knowledge as the constraint, a gradient prior loss is designed to sharpen character boundaries. Results and Conclusions: In order to verify the effectiveness of the proposed method, we use real scene text images in TextZoom and synthetic text images to carry out comparative analysis experiments. Experimental results show that the proposed method can reconstruct clear text edges and rich text texture details, and improve text recognition accuracy of street view images.

WEI Haodong, YI Yaohua, YU Changhui, LIN Liyu. Text Super-resolution Method with Attentional Mechanism and Sequential Units[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220158
Citation: WEI Haodong, YI Yaohua, YU Changhui, LIN Liyu. Text Super-resolution Method with Attentional Mechanism and Sequential Units[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220158
Reference (31)

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return