logo

SCIENTIA SINICA Informationis, Volume 50 , Issue 2 : 275-288(2020) https://doi.org/10.1360/N112018-00192

The algorithm of image super-resolution reconstruction via separable dictionaries

More info
  • ReceivedSep 14, 2018
  • AcceptedJan 17, 2019
  • PublishedFeb 12, 2020

Abstract

Traditional sparse representation-based super-resolution algorithms need to divide images into patches and then stack them into columns. This operation ignores the intrinsic 2D structure and spatial correlation inherent in patches. In order to fully exploit 2D spatial correlation in image patches, we combine the sparse representation ability of the separable dictionary in both the horizontal and vertical directions, and propose an algorithm for image super-resolution based on a separable dictionary. The experimental results show that our proposed algorithm not only improves the efficiency of image super-resolution, but also improves the PSNR and SSIM (i.e., about 0.2-dB PSRN better than traditional methods, and 0.01 SSIM better than existing methods).


Funded by

国家自然科学基金(61872034,61572067,61572063,61572461,11790305)

贵州省自然科学基金([2019]1064)

中央高校基本科研业务费(2017JBZ108)


References

[1] Tian Y, Yap K H. Joint Image Registration and Super-Resolution From Low-Resolution Images With Zooming Motion. IEEE Trans Circuits Syst Video Technol, 2013, 23: 1224-1234 CrossRef Google Scholar

[2] Yang M C, Wang Y C F. A Self-Learning Approach to Single Image Super-Resolution. IEEE Trans Multimedia, 2013, 15: 498-508 CrossRef Google Scholar

[3] Yang S F, Zhao R Z. Image super-resolution reconstruction based on low-rank matrix and dictionary learning. J Comput Res Dev, 2016, 53: 884--891. Google Scholar

[4] Liu C X, Zhang F L, Li M. Image super-resolution based on extreme learning machine. J Hebei Univ Technol, 2016, 46: 11--16. Google Scholar

[5] Jiang J, Tang J M, Pan Z Y. Super-resolution reconstruction based on structure tensor's eigenvalue and classification dictionary. In: Proceedings of IEEE International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, 2016. 452--456. Google Scholar

[6] Yang J C, Wright J, Huang T S. Image Super-Resolution Via Sparse Representation. IEEE Trans Image Process, 2010, 19: 2861-2873 CrossRef PubMed ADS Google Scholar

[7] Yang J C, Wang Z W, Lin Z. Coupled Dictionary Training for Image Super-Resolution. IEEE Trans Image Process, 2012, 21: 3467-3478 CrossRef ADS Google Scholar

[8] Zeyde R, Elad M, Protter M. On single image scale-up using sparse representation. In: Proceeding of the 7th International Conference on Curves and Surfaces, Avignon, France, 2010. 711--730. Google Scholar

[9] Xie H Y, Xie Z G, Hu W, et al. Optimization of dictionary atoms in image sparse representations and its application. J Southeast Univ (Natl Sci Edit), 2014, 44: 116--122. Google Scholar

[10] Engan K, Aase S O, Hus?y J H. Multi-frame compression: theory and design. Signal Processing, 2000, 80: 2121-2140 CrossRef Google Scholar

[11] Engan K, Aase S O, et al. Method of optimal directions for frame design. In: Proceedings of IEEE International Conference on Acoutics, Speech, and Signal Processing, Phoenix, 1999. Google Scholar

[12] Aharon M, Elad M, Bruckstein A. rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Trans Signal Process, 2006, 54: 4311-4322 CrossRef ADS Google Scholar

[13] Hawe S, Kleinsteuber M, Diepold K. Analysis Operator Learning and its Application to Image Reconstruction. IEEE Trans Image Process, 2013, 22: 2138-2150 CrossRef PubMed ADS arXiv Google Scholar

[14] Hawe S, Seibert M, Kleinsteuber M. Separable dictionary learning. In: Proceedings of Conference on Computer Vision and Pattern Recognition, Porland, 2013. 438--445. Google Scholar

[15] Zhang F Z, Cen Y G, Zhao R Z, et al. Analytic separable dictionary learning based on oblique manifold. Neurocomputing, 2016, 236: 32--38. Google Scholar

[16] Zhang F Z, Cen Y G, Zhao R Z, et al. Improved separable dictionary learnig. In: Proceedings of IEEE International Conference on Signal Processing, Chengdu, 2016. 884--889. Google Scholar

[17] Fang Y, Wu J J, Huang B M. 2D sparse signal recovery via 2D orthogonal matching pursuit. Sci China Inf Sci, 2012, 55: 889-897 CrossRef Google Scholar

[18] Rubinstein R, Bruckstein A M, Elad M. Dictionaries for Sparse Representation Modeling. Proc IEEE, 2010, 98: 1045-1057 CrossRef Google Scholar

[19] Lee H, Battle A, Raina R, et al. Efficient sparse coding algorithms. In: Proceedings of Internation Conference on Neural Information Processing Systems, Vancouver, 2006. 801--808. Google Scholar

[20] Timofte R, De V, Gool L V. Anchored neighborhood regression for fast example-based super-resolution. In: Proceeding of IEEE Conference on Computer Vision, Washington, 2013. 1920--1927. Google Scholar

[21] Dong C, Loy C C, He K. Image Super-Resolution Using Deep Convolutional Networks.. IEEE Trans Pattern Anal Mach Intell, 2016, 38: 295-307 CrossRef PubMed Google Scholar

  • Figure 1

    The flaw chart of K-SVD algorithm

  • Figure 2

    (Color online) The image super-resolution reconstruction based on sparse representation

  • Figure 3

    The flow chart of separable dictionary learning based on oblique manifold [14]

  • Figure 4

    (Color online) The graphics of (a) dictionary and (b) sparse represent coefficients

  • Figure 5

    The flow chart of separable dictionary learning based on 2D sparse coding and oblique manifold[15,16]

  • Figure 6

    (Color online) The super-resolution results of flower with magnification 4. (a) The original image; (b) Bicubic; (c) Zedye [8]; (d) ANR [20]; (e) SRCNN [21]; (f) SeDiLSR (ours); (g) ISeDiLSR (ours); (h) ASeDiLSR (ours)

  • Figure 7

    The super-resolution results of facial images with magnification 3

  • Table 1   Average PSNR, SSIM and time (s) on Set5, Set14 and B100
    Set5 Set14 B100
    2 3 4 2 3 4 2 3 4
    Bicubic
    PSNR 33.66 30.39 28.42 30.23 27.54 26 29.32 27.15 25.92
    SSIM 0.93 0.87 0.81 0.87 0.77 0.69 0.832 0.733 0.663
    Yang et al.
    PSNR 31.41 28.31 27.72
    SSIM 0.88 0.79 0.759
    Time 42.14 96.82 30.04
    Zedye et al.
    PSNR 35.78 31.9 29.69 31.81 28.66 26.89 30.39 27.87 26.51
    SSIM 0.95 0.89 0.85 0.898 0.81 0.73 0.866 0.767 0.693
    Time 4.63 2.08 1.25 3.92 4.14 1.22 2.49 1.25 0.79
    ANR
    PSNR 35.79 31.9 26.69 31.76 28.64 26.86 30.42 27.88 26.51
    SSIM 0.949 0.898 0.844 0.899 0.808 0.734 0.869 0.77 0.696
    Time 0.78 0.45 0.34 0.97 0.9 0.46 0.59 0.4 0.31
    SRCNN
    PSNR 36.66 32.389 30.48 32.454 29 27.5 31.36 28.21 26.9
    SSIM 0.954 0.905 0.865 0.906 0.813 0.749 0.887 0.778 0.7
    Time 3.854 3.546 3.863 8.345 7.81 8.396 5.766 5.51 5.804
    SeDiLSR (ours)
    PSNR 35.52 31.945 29.66 32.028 28.93 27.05 31.32 28.46 26.95
    SSIM 0.95 0.9 0.85 0.917 0.831 0.76 0.9 0.8 0.73
    Time 3.135 0.8016 0.38 5.65 0.84 0.6 3.69 0.595 0.44
    ASeDiLSR (ours)
    PSNR 35.34 32.0 29.7 32.059 28.97 27.09 31.11 25.5 26.8
    SSIM 0.953 0.902 0.85 0.918 0.833 0.76 0.904 0.806 0.733
    Time 2.879 0.437 0.34 5.6 0.816 0.57 3.7 0.55 0.36
    ISeDiLSR (ours)
    PSNR 35.56 32.05 29.72 32.062 29.01 27.1 31.35 28.53 26.99
    SSIM 0.954 0.903 0.849 0.918 0.83 0.76 0.9 0.81 0.734
    Time 2.829 0.444 0.315 5.596 0.82 0.576 3.69 0.619 0.39
  • Table 2   The super-resolution results of the datasets of LFW, ORL and Yale
    Bicubic SeDiLSR (ours) ASeDiLSR (ours) ISeDiLSR (ours)
    PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
    2
    LFW 35.122 0.9616 35.5396 0.968 36.3721 0.9698 35.9968 0.9661
    ORL 30.3001 0.9123 30.5224 0.9195 30.7368 0.9209 30.7202 0.9201
    Yale 26.2624 0.8817 26.1103 0.8817 26.5393 0.8854 26.8538 0.8873
    3
    LFW 29.9856 0.8849 30.2636 0.8961 30.6429 0.8984 30.7381 0.8986
    ORL 27.6624 0.8345 27.8313 0.8433 27.9484 0.8444 27.9499 0.8435
    Yale 23.0754 0.7635 23.0021 0.7624 23.502 0.7735 23.8688 0.7892
    4
    LFW 27.0311 0.7886 27.2905 0.8043 27.4714 0.806 27.6124 0.8102
    ORL 26.0258 0.7593 26.2167 0.7691 26.2728 0.7698 26.3004 0.7704
    Yale 20.9202 0.4181 20.8746 0.4188 21.3545 0.5681 21.7321 0.5265
  • Table 3   The time consumption for the facial images in LFW, ORL and Yale (s)
    Algorithm 2 3 4
    LFW ORL Yale LFW ORL Yale LFW ORL Yale
    SeDiLSR (ours) 0.095 0.0841 0.138 0.0984 0.0703 0.07 0.0949 0.091 0.0581
    ASeDiLSR (ours) 0.0522 0.0827 0.0168 0.028 0.0848 0.052 0.0287 0.1292 0.027
    ISeDiLSR (ours) 0.0906 0.0826 0.1826 0.0964 0.0843 0.0796 0.0844 0.1268 0.1805

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备17057255号       京公网安备11010102003388号