logo

Journals Books
Advanced Search
Sign in Register

中文

SCIENTIA SINICA Informationis
Download PDF
SCIENTIA SINICA Informationis, Volume 50 , Issue 12 : 1850(2020) https://doi.org/10.1360/SSI-2019-0167

Biomimetic adaptive memristive cellular neural network for image enhancement

Yawen ZHENG 1, Xiaofang HU 2,*, Yue ZHOU 3, Li LUO 3, Shukai DUAN 2
More info
  • ReceivedAug 8, 2019
  • AcceptedNov 22, 2019
  • PublishedOct 21, 2020
Previous Table of Contents Next
Share
Rating

Abstract

Cellular neural networks (CNN) have a simple local interconnect structure and high-speed parallel processing capability. As the basic model for a constructing artificial retina, CNNs can be applied to image enhancement in the machine vision field. However, existing image enhancement methods based on CNNs face several challenges. For example, when using common fixed templates, it is difficult to obtain ideal results when handling complex images in real-world applications. In addition, the lack of bionic considerations can cause failures when simulating the powerful global and local adaptive adjustment characteristics of human vision. Therefore, this paper proposes a biomimetic adaptive memristive CNN (BAM-CNN) that combines CNNs, human visual adaptive tri-Gaussian theory and memristor, and nano information devices. The proposed CNN can be used for image enhancement. Specifically, the memristive CNN is constructed based on emerging memristors that are programmable, non-volatile, and synapse-plastic. Merged with the tri-Gaussian model for the receptive field of neurons, an adaptive CNN template design algorithm for biomimetic image enhancement is proposed using the image processing features of the Gaussian kernel function and CNNs. In this paper, gray-scale and color images are taken as target examples in image enhancement experiments. The experimental results demonstrate that the proposed BAM-CNN significantly improves the global brightness, local contrast, and sharpness of the image. This paper provides a novel design and implementation scheme for adaptive templates of CNNs, which can improve CNNs' biomimetic characteristics and hardware implementation feasibility. The proposed BAM-CNN can be used to develop innovative techniques for intelligent image processing besides image enhancement.


Funded by

国家自然科学基金(61976246,61601376,61672436)

国家重点研发计划(2018YFB1306600,2018YFB1306604)

中国博士后科学基金特别资助(2018T110937)

重庆市留学人员创业创新支持计划(cx2019126)


References

[1] 王 , 周 , 吴 . 基于HVS 与PCNN 的彩色图像增强. Sci Sin-Inf, 2010, 40: 909-924 CrossRef Google Scholar

[2] Tang Q L, Sang N, Liu H H, et al. Detecting natural image contours by combining visual perception and machinelearning. SciSin Inform, 2013, 43: 1124--1135[唐奇伶, 桑农, 刘海华, 等. 视觉感知结合学习的自然图像轮廓检测. SciSin Inform, 2013, 43: 1124--1135]. Google Scholar

[3] Shen H, Li X, Zhang L. Compressed Sensing-Based Inpainting of Aqua Moderate Resolution Imaging Spectroradiometer Band 6 Using Adaptive Spectrum-Weighted Sparse Bayesian Dictionary Learning. IEEE Trans Geosci Remote Sens, 2014, 52: 894-906 CrossRef ADS Google Scholar

[4] Xinghua Li , Huanfeng Shen , Liangpei Zhang . Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning. IEEE Trans Geosci Remote Sens, 2014, 52: 7086-7098 CrossRef ADS Google Scholar

[5] Muhammad F R, Asmala A, Othman M, et al. Quantifying haze from satellite using haze optimized transformation (HOT). Appl Math Sci, 2015, 9, 1407-1416. Google Scholar

[6] Ding C, Dong L L, Xu W H. Image details enhancement by gradient field bi-interval equalization. Acta Electron Sin, 2017, 45: 1165-1174. Google Scholar

[7] Kolb H. How the retina works. Am Sci, 2003, 91: 29-35. Google Scholar

[8] Tao L, Asari V K. A robust image enhancement technique for improving image visual quality in shadowed scenes. In: Image and Video Retrieval. Berlin: Springer, 2005. 395--404. Google Scholar

[9] Land E H. The retina theory of color vision. Sci Am, 1997, 237: 108-129. Google Scholar

[10] Jobson D J, Rahman Z, Woodell G A. Properties and performance of a center/surround retinex. IEEE Trans Image Process, 1997, 6: 451-462 CrossRef PubMed ADS Google Scholar

[11] Wang S J, Ding X H, Liao Y H, et a1. A novel bio-inspired algorithm for color image enhancement. Acta Electron Sin, 2008, 36: 1970-1973. Google Scholar

[12] Webster M. Human colour perception and its adaptation．Network-Comp Neural, 1996, 17: 587-601. Google Scholar

[13] Chao-Yi L, Xing P, Yi-Xiong Z. Role of the extensive area outside the x-cell receptive field in brightness information transmission. Vision Res, 1991, 31: 1529-1540 CrossRef Google Scholar

[14] Zhao G Z, Duan H B. Progresses in biological eagle-eye vision technology. SST, 2017, 47: 514-523 CrossRef Google Scholar

[15] Solomon S G, White A J R, Martin P R. Extraclassical Receptive Field Properties of Parvocellular, Magnocellular, and Koniocellular Cells in the Primate Lateral Geniculate Nucleus. J Neurosci, 2002, 22: 338-349 CrossRef Google Scholar

[16] Nolt M, Kumbhani R, Palmer L. Contrast-dependent spatial summation in the lateral geniculate nucleus and retina of the cat. J Neurosci, 2004, 92: 1708-1717. Google Scholar

[17] Ringach D L, Shapley R M, Hawken M J. Orientation Selectivity in Macaque V1: Diversity and Laminar Dependence. J Neurosci, 2002, 22: 5639-5651 CrossRef Google Scholar

[18] Chua L, Yang L. Cellular neural networks: theory. IEEE Trans Circ Syst, 1998, 35: 1257-1272. Google Scholar

[19] Zhang H, Wang Z. Global asymptotic stability of delayed cellular neural networks.. IEEE Trans Neural Netw, 2007, 18: 947-950 CrossRef PubMed Google Scholar

[20] Wang L, Yang T, Hu X. Color image edge extraction using memristor-based CNN. Sci Sin-Inf, 2017, 47: 863-877 CrossRef Google Scholar

[21] Roska T, Boros T, Radványi A. Detecting moving and standing objects using cellular neural networks. Int J Circ Theor Appl, 1992, 20: 613-628 CrossRef Google Scholar

[22] Suzuki H, Matsumoto T, Chua L O. A CNN handwritten character recognizer. Int J Circ Theor Appl, 1992, 20: 601-612 CrossRef Google Scholar

[23] Bouzerdoum A, Pinter R B. Shunting inhibitory cellular neural networks: derivation and stability analysis. IEEE Trans Circ Syst I: Fundamental Theory Appl, 1993, 40: 215--221. Google Scholar

[24] Kondo K, Lguch M, Lshigaki H, et al. Design of complex-valued CNN filters for medical image enhancement. In: Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference, 2001. 1642--1646. Google Scholar

[25] Kim H, Sah M, Yang C, et al. Neural synaptic weighting with a pulse-based memristor circuit. IEEE Trans Circ Syst, 2012, 59: 148-158. Google Scholar

[26] Hyuncheol C, Ram B, Sedong P, et al. Linear programming of voltage-controlled memristors with an anti-serial memristor circuit. In: Proceedings of IEEE International Symposium on Circuits and Systems, 2015. 1142--1145. Google Scholar

[27] Hu X F, Feng G, Duan S K, et al. A memristive multilayer cellular neural network with applications to image processing. IEEE Trans Neural Netw Learn Syst, 2017, 28: 1889-1901. Google Scholar

[28] Kvatinsky S, Fruedman E, Kolodny A, et al. TEAM: ThrEshold adaptive memristor model. IEEE Trans Circ Syst I: Regular Papers, 2012, 60: 211-221. Google Scholar

[29] Wang L, Li H, Duan S. Pavlov associative memory in a memristive neural network and its circuit implementation. Neurocomputing, 2016, 171: 23-29 CrossRef Google Scholar

[30] Yu Y H, Chang C C. A new edge detection approach based on image context analysis. Image Vision Computing, 2006, 24: 1090-1102 CrossRef Google Scholar

[31] Wang L D, Duan M T, Duan S K, et al. Neural networks based on STDP rules and memristor bridge synapses withapplications in image processing. Sci Sin Inform, 2014, 44: 920--930 [王丽丹, 段美涛, 段书凯, 等. 基于STDP 规则和忆阻桥突触的神经网络及图像处理. 中国科学: 信息科学, 2014, 44: 920--930]. Google Scholar

[32] Zheng M W ,Li L X,Peng H P, et al. Fixed-time synchronization of memristive fuzzy BAM cellular neural networks with time-varying delays based on feedback controllers. IEEE Access, 2018, 6, 12085-12102. Google Scholar

[33] Kvatinsky S, Ramadan M, Fruedman E, et al. VTEAM: A general model for voltage-controlled memristors. IEEE Trans Circ Syst II: Express Briefs, 2015, 62: 786-790. Google Scholar

[34] Duan S, Hu X, Dong Z. Memristor-based cellular nonlinear/neural network: design, analysis, and applications.. IEEE Trans Neural Netw Learning Syst, 2015, 26: 1202-1213 CrossRef PubMed Google Scholar

[35] Kim H, Sah M, Yang C, et al. Memristor bridge synapses. P IEEE, 2011, 100: 2061-2070. Google Scholar

[36] Agaian S S, Silver B, Panetta K A. Transform Coefficient Histogram-Based Image Enhancement Algorithms Using Contrast Entropy. IEEE Trans Image Process, 2007, 16: 741-758 CrossRef PubMed ADS Google Scholar

  • Click to view Download high-quality image

    Figure 1

    (Color online) CNN topology model

  • Click to view Download high-quality image

    Figure 2

    (Color online) Memristive neuron circuit

  • Click to view Download high-quality image

    Figure 3

    (Color online) Memristive bridge synapse circuit

  • Click to view Download high-quality image

    Figure 4

    (Color online) Template of CNN with $3\times~3$ neighborhood

  • Click to view Download high-quality image

    Figure 5

    (Color online) Spatial structure of tri-Gaussian function model. (a) Intermediate exciting module; (b) surrounding inhibition module; (c) edge exciting module; (d) tri-Gaussian model

  • Click to view Download high-quality image

    Figure 6

    (a) The curve of the sensitivity $A_1$ of receptive field sensitivity zone to Con; (b) The curve of the radius $\sigma_2$ of receptive field suppression zone to Lum

  • Click to view Download high-quality image

    Figure 7

    Color image enhancement process

  • Click to view Download high-quality image

    Figure 8

    (Color online) Enhanced effect comparison. (a) Room; (b) Tower

  • Click to view Download high-quality image

    Figure 9

    (Color online) House image bionic enhancement. (a) Original image; (b) tri-Gaussian model; (c) adaptive tri-Gaussian model; (d) linearized adaptive tri-Gaussian model; (e) CNN fused with adaptive tri-Gaussian model; (f) BAM-CNN

  • Click to view Download high-quality image

    Figure 10

    (Color online) Couple image bionic enhancement. (a) Original image; (b) tri-Gaussian model; (c) adaptive tri-Gaussian model; (d) linearized adaptive tri-Gaussian model; (e) CNN fused with adaptive tri-Gaussian model; (f) BAM-CNN

  • Table 1   Comparison of the effects (EME) of the six enhancement methods
    No Origin HE Retinex HF WTGSL BAM-CNN
    1652.4262 793.3994 772.3151 793.2867 775.8123 765.9624 796.9342
    2 568.8201 698.4078 661.9563 686.7693 559.4216 720.0508 818.2679
    3 428.2354 634.4120 611.5280 624.0064 431.6045 695.7376 676.8831
    4 214.3378 260.6652 190.0746 255.1763 201.0840 351.3541 388.8343
    5 668.4836 755.3688 397.2203 761.9537 725.5078 751.9514 907.3139
    6 235.5166 379.8441 242.6934 458.4887 235.3616 443.3711 459.3054
    7 392.0305 639.9298 579.2962 656.0020 427.1555 624.6440 600.0791
    8 536.4732 673.0242 494.2527 747.4195 554.4671 653.2434 794.5018
    9 366.8843 438.9627 382.8986 466.4004 391.8464 441.3711 542.5105
    10 553.1127 707.4049 573.0594 728.4262 600.4264 686.2060 760.8238
    11 496.2885 464.8486 493.9914 455.6647 492.6860 552.0750 535.5896
    12 204.1674 276.9846 215.0037 214.5829 166.0431 204.1674 406.9582
    13 181.6859 242.8295 202.4264 290.8129 234.6048 235.5157 346.4498
    14 571.5830 860.7786 571.9902 436.9282 560.3997 866.4457 798.4308
    15 240.2980 215.3778 240.9412 226.5606 240.5550 240.8652 337.4020
  • Table 2   Comparison of the effects of the six enhancement models
    No. Evaluation index Origin TGM ATGM LATGM CNN-TGM CNN-ATGM BAM-CNN
    1 EME 516.7271 574.7419 575.0046 575.0046 676.8831 676.8831 676.8831
    SMD2 4050246 7195030 7282118 7270258 30977436 31472958 31520264
    2 EME 243.7382 275.5480 275.5480 275.5480 459.1279 459.3054 459.3054
    SMD2 1594525 3167996 3179766 3178265 5450193 5604472 5615845
    3 EME 424.0944 643.9899 643.9899 643.9899 644.3513 600.0791 600.0791
    SMD2 98474377 166954625 167164355 167153301 186762626 185927054 185878806
    4EME 375.9151 428.4620 428.4620 428.4620 535.5896 535.5896 535.5896
    SMD2 3786709 7293439 7337408 7331335 19553768 19613745 30031026
    5 EME 818.3787 313.2185 313.2185 313.2185 313.1769 313.1769 313.1769
    SMD2 14373051 17552085 17587905 17577407 31787078 31811886 31801532
    6EME 638.3819 449.2624 449.2653 449.2624 789.0090 789.0090 796.9342
    SMD2 41016820 50170828 50303836 50288306 81008360 80827900 239619331
    7EME 545.3247 691.5614 691.5614 691.5614 760.7921 760.8160 760.8238
    SMD2 37916146 76333187 76946567 76829900 294066564 293730676 293743101
    8 EME 409.6000 321.9660 321.9660 321.9660 330.6879 378.2778 378.2835
    SMD2 1578587 3015839 3045405 3042080 10409059 10914315 10925921
    9EME 587.5047 702.2330 702.2330 702.2330 714.8868 818.2679 818.2679
    SMD2 53099370 43866606 43869362 44086474 121734667 147002589 147304736
    10 EME 469.7769 615.2829 615.2829 615.2829 741.8416 741.9090 741.9090
    SMD2 15012384 55617848 56008629 55950050 233029446 232946196 232927961
    11 EME 687.2862 687.2862 687.2862 687.2862 687.2862 687.2862 687.2862
    SMD2 3603688 5887550 5914441 5911444 17806158 15963913 16147183
    12 EME 240.2980 326.2426 326.6014 326.6014 337.4020 337.4020 337.4020
    SMD2 3012108 13620247 13682373 13674132 23959523 24344829 24304184
    13 EME 728.6271 728.3553 728.3553 728.3553 728.4551 728.3553 728.3553
    SMD2 255930555 53888055 54143765 54098895 154063947 133577888 135459162
    14 EME 562.2581 685.2473 685.2473 685.2473 688.5457 688.5457 688.5457
    SMD2 2491177 3833026 3890344 3884939 17480651 17498549 17490717
    15 EME 605.3837 519.2382 519.2382 519.2382 572.9556 573.0366 573.0366
    SMD2 67393400 65403398 65802209 65740477 171877694 171273138 171206923
    16 EME 260.7194 288.1046 288.2781 288.2781 383.6287 383.6287 383.6287
    SMD2 661453 4483283 4529402 4523482 14942187 13842137 13762008
    17 EME 303.6955 239.0931 239.0918 239.0931 410.2250 410.2250 410.2250
    SMD2 1093784 3612458 3645373 3642404 11263263 10117695 10056009
    18 EME 263.7576 211.8428 211.9743 211.8428 380.7958 380.7958 380.7958
    SMD2 710809 7606842 9192230 7655291 31747832 29260373 29079092
    19 EME 303.2979 209.0489 209.0489 209.0489 409.5968 409.5968 409.5968
    SMD2 676089 4377827 4418391 4399980 14625390 12534276 12534276
    20 EME 200.8583 342.2648 342.2648 342.2648 342.3068 342.3068 342.3068
    SMD2 34374241 50726776 50936495 50862594 93001493 93160126 93095410