logo

SCIENCE CHINA Information Sciences, Volume 61, Issue 6: 060423(2018) https://doi.org/10.1007/s11432-018-9425-1

Artificial neural networks based on memristive devices

More info
  • ReceivedFeb 22, 2018
  • AcceptedMar 27, 2018
  • PublishedMay 15, 2018

Abstract

The advent of memristive devices and the continuing researchand development in the field of neuromorphic computing show great potentialas an alternative to traditional von Neumann computing and bring us evercloser to realizing a true “thinking machine”. Novel neural networkarchitectures and algorithms inspired by the brain are becoming more andmore attractive to keep up with computing needs, relying on intrinsicparallelism and reduced power consumption to outperform more conventionalcomputing methods. This article provides an overview of various neuralnetworks with an emphasis on networks based on memristive emerging devices,with the advantages of memristor neural networks compared with purecomplementary metal oxide semiconductor (CMOS) implementations. A generaldescription of neural networks is presented, followed by a survey ofprominent CMOS networks, and finally networks implemented using emergingmemristive devices are discussed, along with the motivation for developingmemristor based networks and the computational potential these networkspossess.


References

[1] Mack C A. Fifty years of Moore's law. IEEE Trans Semicond Manufact, 2011, 24: 202-207 CrossRef Google Scholar

[2] Schulz M. The end of the road for silicon? Nature, 1999, 399: 729--730. Google Scholar

[3] von Neumann J. First draft of a report on the EDVAC. IEEE Ann HistComput, 1993, 15: 11--21. Google Scholar

[4] Anderson H C. Neural network machines. IEEE Potentials, 1989, 8: 13--16. Google Scholar

[5] Squire L R, Berg D, Bloom F, et al. Fundamental neuroscience. Curr Opin Neurobiol, 2008, 10: 649--654. Google Scholar

[6] Pedretti G, Milo V, Ambrogio S. Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity. Sci Rep, 2017, 7: 5288 CrossRef PubMed ADS Google Scholar

[7] Zamarre no-Ramos C, Camu nas-Mesa L A, Pérez-Carrasco J A, et al. On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front Neurosci-Switz, 2011, 5: 1--22. Google Scholar

[8] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436-444 CrossRef PubMed ADS Google Scholar

[9] Li C, Hu M, Li Y. Analogue signal and image processing with large memristor crossbars. Nat Electron, 2018, 1: 52-59 CrossRef Google Scholar

[10] Hebb D O. The first stage of perception: growth of the assembly BT — the organization of behavior. Organ Behav, 1949, 4: 60--78. Google Scholar

[11] Wang Z, Joshi S, Savel'ev S E. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat Mater, 2017, 16: 101-108 CrossRef PubMed ADS Google Scholar

[12] Hu M, Strachan J P, Li Z, et al. Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. In: Proceedings of the 53rd ACM/EDAC/IEEE Design Automation Conference (DAC), Austin, 2016. 1--6. Google Scholar

[13] Shafiee A, Nag A, Muralimanohar N, et al. ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 14--26. Google Scholar

[14] Chi P, Li S, Xu C, et al. PRIME: a novel processing-in-memory architecture for neural network computation in reRAM-based main memory. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 27--39. Google Scholar

[15] Tang S, Yin S, Zheng S, et al. AEPE: an area and power efficient RRAM crossbar-based accelerator for deep CNNs. In: Proceedings of IEEE 6th Non-Volatile Memory Systems and Applications Symposium (NVMSA), Hsinchu, 2017. 1--6. Google Scholar

[16] Yao P, Wu H, Gao B. Face classification using electronic synapses. Nat Commun, 2017, 8: 15199 CrossRef PubMed ADS Google Scholar

[17] Wang Z, Joshi S, Savel'ev S. Fully memristive neural networks for pattern classification with unsupervised learning. Nat Electron, 2018, 1: 137-145 CrossRef Google Scholar

[18] Sheridan P M, Cai F, Du C. Sparse coding with memristor networks. Nat Nanotech, 2017, 12: 784-789 CrossRef PubMed Google Scholar

[19] Hu M, Graves C E, Li C. Memristor-based analog computation and neural network classification with a dot product engine. Adv Mater, 2018, 30: 1705914 CrossRef PubMed Google Scholar

[20] Li C, Belkin D, Li Y, et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat Commun, in press, 2018. Google Scholar

[21] Zarudnyi K, Mehonic A, Montesi L. Spike-timing dependent plasticity in unipolar silicon oxide RRAM devices. Front Neurosci, 2018, 12: 57 CrossRef Google Scholar

[22] Yu S M, Li Z W, Chen P-Y, et al. Binary neural network with 16 Mb RRAM macro chip for classification and online training. In: Proceedings of IEEE International Electron Devices Meeting (IEDM), San Francisco, 2016. 1--4. Google Scholar

[23] Prezioso M, Merrikh-Bayat F, Hoskins B D. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 2015, 521: 61-64 CrossRef PubMed ADS arXiv Google Scholar

[24] Yu S. Neuro-inspired computing with emerging nonvolatile memorys. Proc IEEE, 2018, 106: 260-285 CrossRef Google Scholar

[25] Alibart F, Zamanidoost E, Strukov D B, et al. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat Commun, 2013, 2013: 2072. Google Scholar

[26] Merrikh B F, Prezioso M, Chakrabarti B, et al. Advancing memristive analog neuromorphic networks: increasing complexity, and coping with imperfect hardware components,. arXiv Google Scholar

[27] Du C, Cai F, Zidan M A. Reservoir computing using dynamic memristors for temporal information processing. Nat Commun, 2017, 8: 2204 CrossRef PubMed ADS Google Scholar

[28] Mehonic A, Kenyon A J. Emulating the electrical activity of the neuron using a silicon oxide RRAM cell. Front Neurosci-Switz, 2016, 10: 57. Google Scholar

[29] Chen Y J, Luo T, Liu S L, et al. DaDianNao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture, Cambridge, 2014. 609--622. Google Scholar

[30] Merolla P A, Arthur J V, Alvarez-Icaza R. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 2014, 345: 668-673 CrossRef PubMed ADS Google Scholar

[31] Jouppi N P, Young C, Patil N, et al. In-Datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, Toronto, 2017. 1--12. Google Scholar

[32] Gawande N A, Landwehr J B, Daily J A, et al. Scaling deep learning workloads: NVIDIA DGX-1/Pascal and intel knights landing. In: Proceedings of 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Lake Buena Vista, 2017. 399--408. Google Scholar

[33] Chua L O. Memristor — the missing circuit element. IEEE Trans Circuits Syst, 1971, 18: 507--519. Google Scholar

[34] Strukov D B, Snider G S, Stewart D R. The missing memristor found. Nature, 2008, 453: 80-83 CrossRef PubMed ADS Google Scholar

[35] Yang J J, Pickett M D, Li X. Memristive switching mechanism for metal/oxide/metal nanodevices. Nat Nanotech, 2008, 3: 429-433 CrossRef PubMed Google Scholar

[36] Joshua Yang J, Miao F, Pickett M D. The mechanism of electroforming of metal oxide memristive switches. Nanotechnology, 2009, 20: 215201 CrossRef PubMed ADS Google Scholar

[37] Chua L. Resistance switching memories are memristors. Appl Phys A, 2011, 102: 765-783 CrossRef ADS Google Scholar

[38] Yang J J, Strukov D B, Stewart D R. Memristive devices for computing. Nat Nanotech, 2013, 8: 13-24 CrossRef PubMed ADS Google Scholar

[39] Yang J J, Williams R S. Memristive devices in computing system: promises and challenges. ACM J Emerg Tech Com, 2013, 9: 1--20. Google Scholar

[40] Pickett M D, Strukov D B, Borghetti J L. Switching dynamics in titanium dioxide memristive devices. J Appl Phys, 2009, 106: 074508 CrossRef ADS Google Scholar

[41] Alibart F, Gao L, Hoskins B D. High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology, 2012, 23: 075201 CrossRef PubMed ADS arXiv Google Scholar

[42] Choi S, Sheridan P, Lu W D. Data clustering using memristor networks. Sci Rep, 2015, 5: 10492 CrossRef PubMed ADS Google Scholar

[43] Yang J J, Zhang M X, Strachan J P. High switching endurance in TaO$_{x}$ memristive devices. Appl Phys Lett, 2010, 97: 232102 CrossRef ADS Google Scholar

[44] Choi B J, Torrezan A C, Strachan J P. High-speed and low-energy nitride memristors. Adv Funct Mater, 2016, 26: 5290-5296 CrossRef Google Scholar

[45] Yoon J H, Zhang J, Ren X. Truly Electroforming-Free and Low-Energy Memristors with Preconditioned Conductive Tunneling Paths. Adv Funct Mater, 2017, 27: 1702010 CrossRef Google Scholar

[46] Li C, Han L, Jiang H. Three-dimensional crossbar arrays of self-rectifying Si/SiO$_{2}$/Si memristors. Nat Commun, 2017, 8: 15666 CrossRef PubMed ADS Google Scholar

[47] Shulaker M M, Hills G, Park R S. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature, 2017, 547: 74-78 CrossRef PubMed ADS Google Scholar

[48] Yu S, Wu Y, Jeyasingh R. An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation. IEEE Trans Electron Devices, 2011, 58: 2729-2737 CrossRef ADS Google Scholar

[49] Chang T, Yang Y, Lu W. Building Neuromorphic Circuits with Memristive Devices. IEEE Circuits Syst Mag, 2013, 13: 56-73 CrossRef Google Scholar

[50] Chang T, Jo S H, Kim K H. Synaptic behaviors and modeling of a metal oxide memristive device. Appl Phys A, 2011, 102: 857-863 CrossRef ADS Google Scholar

[51] Gaba S, Sheridan P, Zhou J. Stochastic memristive devices for computing and neuromorphic applications. Nanoscale, 2013, 5: 5872-5878 CrossRef PubMed ADS arXiv Google Scholar

  • Figure 1

    (Color online) Neuron and synapse structure. (a) Simplified biological structure of pre-synaptic and post-synaptic neurons showing the axon, synapses, and dendrites [6]@Copyright 2017 Scientific Reports; (b) biological synaptic structure and function [7]@Copyright 2011 Frontiers in Neuroscience.

  • Figure 2

    (Color online) (a) Memristor crossbar for vector matrix multiplication (VMM); (b) current (mA) output was experimentally measured for a 6464 crossbar for VMM and the resulting current from column 1 and column 64 were compared with the expected values using software VMM. Non-idealities in the devices and circuitry result in a drastic difference between the ideal output and the actual output [9]@Copyright 2018 Nature Electronics.

  • Figure 3

    (a) ISAAC architecture using tiles [13]. (b) Simplified design of PRIME [14]. External I/O data is fed into RRAM crossbar based banks. Memory subarrays only store data. Full function subarrays can store data or perform computation, using a CMOS controller to enable configuration. The buffer subarrays serve as data buffers for the full function subarrays. (c) AEPE Implementation with efficient tile architecture [15]. The register buffer feeds into M rows of crossbar based PE arrays through an assigned DAC for each row.

  • Figure 4

    (Color online) (a) A single-layer is mapped onto the one transistor one memristor (1T1R) array; (b) micrograph of the 1024 cell 1T1R array fabricated using CMOS compatible processes [16]@Copyright 2017 Nature Communications.

  • Figure 5

    (Color online) (a) Photograph of fabricated 1T1R crossbars. Two dies are shown, each one containing various array sizes from 4$\times$4 to 128$\times$64 cells. (b) Micrograph of four cells in a 1T1R array (scale bar, 10 mm). (c) Relatively linear I-V curves for all the devices over the chosen conductance range. (d) Device state retention and read disturbance (1000 cycles of 0.2 V read pulses) at room temperature show no discernible drift [9]@Copyright 2017 Nature Electronics.

  • Figure 6

    (Color online) (a) Synapse array with peripheral circuitry; (b) a pre-synaptic device controls the FET gate of the 1T1R structure, while the post-synaptic device receives the input current and controls the synapse top electrode to induce synaptic current and stimulate synaptic potentiation or depression during the fire [6]@Copyright 2017 Scientific Reports.

  • Figure 7

    (Color online) Diffusive memristor neuron schematic. The diffusive memristor behaves in a fundamentally similar manner as a biological ion channel on the soma membrane of a neuron [17]@Copyright 2018 Nature Electronics.

  • Figure 8

    (Color online) (a) All memristor network fabricated using (b) and (c) drift memristors for synapses and (d) and (e) diffusive memristors for neurons [17]@Copyright 2018 Nature Electronics.

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1