logo

SCIENCE CHINA Information Sciences, Volume 64 , Issue 2 : 122403(2021) https://doi.org/10.1007/s11432-020-3040-1

A modified supervised learning rule for training a photonic spiking neural network to recognize digital patterns

More info
  • ReceivedApr 15, 2020
  • AcceptedAug 7, 2020
  • PublishedJan 20, 2021

Abstract


Acknowledgment

This work was supported in part by National Key Research and Development Program of China (Grant No. 2018YFB2200500), National Natural Science Foundation of China (Grant Nos. 61974177, 61674119), China Scholarship Council, Postdoctoral Science Foundation in Shaanxi Province of China, in part by Fundamental Research Funds for the Central Universities, and the Innovation Fund of Xidian University (Grant No. 5001-20109195456).


References

[1] Giard M H, Peronnet F. Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study. J Cognitive Neuroscience, 1999, 11: 473-490 CrossRef Google Scholar

[2] Alibart F, Zamanidoost E, Strukov D B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat Commun, 2013, 4: 2072 CrossRef ADS Google Scholar

[3] Diehl P U, Cook M. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci, 2015, 9 CrossRef Google Scholar

[4] Park S, Chu M, Kim J. Electronic system with memristive synapses for pattern recognition. Sci Rep, 2015, 5: 10123 CrossRef ADS Google Scholar

[5] Prezioso M, Merrikh-Bayat F, Hoskins B D. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 2015, 521: 61-64 CrossRef ADS arXiv Google Scholar

[6] Zhang Y, Li Y, Wang X. Synaptic Characteristics of Ag/AgInSbTe/Ta-Based Memristor for Pattern Recognition Applications. IEEE Trans Electron Devices, 2017, 64: 1806-1811 CrossRef ADS Google Scholar

[7] Coomans W, Gelens L, Beri S. Solitary and coupled semiconductor ring lasers as optical spiking neurons. Phys Rev E, 2011, 84: 036209 CrossRef ADS arXiv Google Scholar

[8] Ren Q, Zhang Y, Wang R. Optical spike-timing-dependent plasticity with weight-dependent learning window and reward modulation. Opt Express, 2015, 23: 25247-25258 CrossRef ADS Google Scholar

[9] Shen Y, Harris N C, Skirlo S. Deep learning with coherent nanophotonic circuits. Nat Photon, 2017, 11: 441-446 CrossRef ADS arXiv Google Scholar

[10] Xiang S Y, Zhang H, Guo X X. Cascadable Neuron-Like Spiking Dynamics in Coupled VCSELs Subject to Orthogonally Polarized Optical Pulse Injection. IEEE J Sel Top Quantum Electron, 2017, 23: 1-7 CrossRef ADS Google Scholar

[11] Deng T, Robertson J, Hurtado A. Controlled Propagation of Spiking Dynamics in Vertical-Cavity Surface-Emitting Lasers: Towards Neuromorphic Photonic Networks. IEEE J Sel Top Quantum Electron, 2017, 23: 1-8 CrossRef ADS Google Scholar

[12] Zhang Y, Xiang S, Gong J. Spike encoding and storage properties in mutually coupled vertical-cavity surface-emitting lasers subject to optical pulse injection. Appl Opt, 2018, 57: 1731-1737 CrossRef ADS Google Scholar

[13] Xiang S, Zhang Y, Gong J. STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs. IEEE J Sel Top Quantum Electron, 2019, 25: 1-9 CrossRef ADS Google Scholar

[14] Feldmann J, Youngblood N, Wright C D. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 2019, 569: 208-214 CrossRef ADS Google Scholar

[15] Robertson J, Wade E, Kopp Y. Toward Neuromorphic Photonic Networks of Ultrafast Spiking Laser Neurons. IEEE J Sel Top Quantum Electron, 2020, 26: 1-15 CrossRef ADS Google Scholar

[16] Xu S, Wang J, Wang R. High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays: erratum. Opt Express, 2020, 28: 21854 CrossRef ADS Google Scholar

[17] Xiang S, Ren Z, Zhang Y. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on a VCSEL-SA. Opt Lett, 2020, 45: 1104-1107 CrossRef ADS Google Scholar

[18] Widrow B, Hoff M E. Adaptive Switching Circuits. IRE WESCON Convention Record, 1960. Google Scholar

[19] Barlow H B. Unsupervised Learning. Neural Computation, 1989, 1: 295-311 CrossRef Google Scholar

[20] Ponulak F, Kasi??ski A. Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting. Neural Computation, 2010, 22: 467-510 CrossRef Google Scholar

[21] Bi G, Poo M. Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type. J Neurosci, 1998, 18: 10464-10472 CrossRef Google Scholar

[22] Fok M P, Tian Y, Rosenbluth D. Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. Opt Lett, 2013, 38: 419-421 CrossRef ADS Google Scholar

[23] Toole R, Fok M P. Photonic implementation of a neuronal algorithm applicable towards angle of arrival detection and localization. Opt Express, 2015, 23: 16133-16141 CrossRef ADS Google Scholar

[24] Xiang S, Gong J, Zhang Y. Numerical Implementation of Wavelength-Dependent Photonic Spike Timing Dependent Plasticity Based on VCSOA. IEEE J Quantum Electron, 2018, 54: 1-7 CrossRef ADS Google Scholar

[25] Toole R, Tait A N, Ferreira de Lima T. Photonic Implementation of Spike-Timing-Dependent Plasticity and Learning Algorithms of Biological Neural Systems. J Lightwave Technol, 2016, 34: 470-476 CrossRef ADS Google Scholar

[26] Song Z, Xiang S, Ren Z. Spike Sequence Learning in a Photonic Spiking Neural Network Consisting of VCSELs-SA With Supervised Training. IEEE J Sel Top Quantum Electron, 2020, 26: 1-9 CrossRef ADS Google Scholar

[27] Xiang S, Ren Z, Song Z. Computing Primitive of Fully VCSEL-Based All-Optical Spiking Neural Network for Supervised Learning and Pattern Classification. IEEE Trans Neural Netw Learning Syst, 2020, : 1-12 CrossRef Google Scholar

[28] Nahmias M A, Shastri B J, Tait A N. IEEE J Sel Top Quantum Electron, 2013, 19: 1-12 CrossRef ADS Google Scholar

[29] Li Q, Wang Z, Cui C. Simulating the spiking response of VCSEL-based optical spiking neuron. Optics Commun, 2018, 407: 327-332 CrossRef ADS Google Scholar

  • Figure 1

    (Color online) Schematic diagram of photonic SNN. (a) The photonic SNN consists of 30 presynaptic neurons and 10 postsynaptic neurons with all-to-all connection. $N_1$–$N_{30}$: photonic presynaptic neurons; $N_{31}$–$N_{40}$: photonic postsynaptic neuron; $W_{1,31}$: the weight device between $N_1$ and $N_{31}$; $W_{2,31}$–$W_{30,40}$: the weight devices are similar to $W_{1,31}$; WCU can control all weights in the red dashed box. (b) Thirty presynaptic neurons are putted as a rectangle. (c) An example of the connection between $N_1$ and $N_{31}$. The red signals “$s$", “+", and “$-$" donate the learning rule simply. (d) Flowchart of learning process.

  • Figure 2

    Ten digital images of size $5\times6$ pixels.

  • Figure 3

    (Color online) The result of recognition after training. The insets are responses of 10 postsynaptic neurons for digital “5" image. (a)–(j) The detail of postsynaptic neurons outputs for digital “5" image.

  • Figure 4

    (Color online) The weights in the learning process. (a) Representative weights in learning process; (b) all weights of synapses connected with $N_{35}$ in the learning process.

  • Figure 5

    (Color online) The error in the learning process. (a)–(j) The error in learning process when test images are digital “1"–“9", “0" images, respectively; (k) the error in learning process when test digital images is injected in turn.

  • Figure 6

    (Color online) $E_{\rm~e}$ in learning process with different learning rates $\Delta\omega$ (a1)–(a3) and different jitters of $\Delta\omega$ (b1)–(b3).

  • Figure 7

    (Color online) $E_{\rm~e}$ in learning process with different $\omega_0$. (a1), (a2) Different distribution of $\omega_0$, $\omega_0~\sim~U[0,~0.5]$; protectłinebreak (b1) $\omega_0~\sim~U[0,~0.25]$; (b2) $\omega_0~\sim~U[0,~1]$; (c1) $\omega_0~\sim~U[0.25,~0.75]$; (c2) $\omega_0~\sim~U[0.5,~1]$.

  • Figure 8

    (Color online) $E_{\rm~e}$ in learning process with different $I$. (a) $I=1.9$ mA, $\triangle~\omega~=~0.5$; (b) $I~=~2.1$ mA, $\triangle~\omega~=~0.0025$; protectłinebreak (c) $I~=~2.2$ mA, $\triangle~\omega~=~0.0025$.

  • Table 1  

    Table 1The adjustment of weight based on the proposed modified supervised learning rule in one iteration

    Presynaptic output Actual output Desired output $\Delta\omega_{i,j}$
    Case 1 $\surd$ $\surd$ $\surd$ 0
    Case 2 $\surd$ $\surd$ $\times$ $-\xi$
    Case 3 $\surd$ $\times$ $\surd$ $+\xi$
    Case 4 $\surd$ $\times$ $\times$ 0
    Case 5 $\times$ 0
  • Table 2  

    Table 2VCSEL-SA parameters [12,28,29]

    Parameter Gain region Absorber region
    Cavity volume $V_{\rm~a,s}$ $2.4\times10^{-18}$ m$^3$ $2.4\times10^{-18}$ m$^3$
    Confinement factor $\Gamma_{\rm~a,s}$ $0.06$ $0.05$
    Carrier lifetime $\tau_{\rm~a,s}$ 1 ns 100 ps
    Differential gain/loss $g_{\rm~a,s}$ $2.9\times10^{-12}$ m$^3$s$^{-1}$ $1.45\times10^{-12}$ m$^3$s$^{-1}$
    Transparency carrier density $n_{\rm~0a,s}$ $1.1\times10^{24}$ m$^3$ $0.89\times10^{24}$ m$^3$
    Bias current $I_{\rm~a,s}$ 0 mA 2 mA/2.15 mA
    Speed of light c $3\times10^{8}$ m/s
    Spontaneous emission coupling factor $\beta$ $10^{-4}$
    Bimolecular recombination term $B_r$ $10\times10^{-16}$ m$^3$s$^{-1}$
    Output power coupling coefficient $\eta_c$ $0.4$
    Photon lifetime $\tau_{\rm~ph}$ $4.8\times10^{-12}$ s