logo

SCIENCE CHINA Information Sciences, Volume 63 , Issue 6 : 160401(2020) https://doi.org/10.1007/s11432-020-2863-y

Towards an intelligent photonic system

More info
  • ReceivedJan 21, 2020
  • AcceptedApr 1, 2020
  • PublishedMay 9, 2020

Abstract

The emerging intelligence technologies represented by deep learning have broadened their applications to various fields. Beyond the conventional electronics-based processing systems, the convergence of photonics and artificial intelligence (AI) technology enhances the performance and learning ability of AI. In this review, we propose the concept of an intelligent photonic system (IPS), illustrating it as a developing architecture with three different versions. For each version of IPS, we review several representative studies. Moreover we discuss the challenges towards an IPS and provide some prospects for the future development.


Acknowledgment

This work was supported by National Key RD Program of China (Grant No. 2019YFB2203700) and National Natural Science Foundation of China (Grant No. 61822508).


References

[1] Kikuchi K. Fundamentals of Coherent Optical Fiber Communications. J Lightw Technol, 2016, 34: 157-179 CrossRef ADS Google Scholar

[2] Yao J. Microwave Photonics. J Lightw Technol, 2009, 27: 314-335 CrossRef ADS Google Scholar

[3] Liang J, Wang L V. Single-shot ultrafast optical imaging. Optica, 2018, 5: 1113-1127 CrossRef ADS Google Scholar

[4] Chen J H, Li D R, Xu F. Optical Microfiber Sensors: Sensing Mechanisms, and Recent Advances. J Lightwave Technol, 2019, 37: 2577-2589 CrossRef ADS Google Scholar

[5] Capmany J, Novak D. Microwave photonics combines two worlds. Nat Photon, 2007, 1: 319-330 CrossRef ADS Google Scholar

[6] Sun C, Wade M T, Lee Y. Single-chip microprocessor that communicates directly using light. Nature, 2015, 528: 534-538 CrossRef PubMed ADS Google Scholar

[7] Khan M H, Shen H, Xuan Y. Ultrabroad-bandwidth arbitrary radiofrequency waveform generation with a silicon photonic chip-based spectral shaper. Nat Photon, 2010, 4: 117-122 CrossRef ADS Google Scholar

[8] Zhuang L, Roeloffzen C G H, Hoekman M. Programmable photonic signal processor chip for radiofrequency applications. Optica, 2015, 2: 854-859 CrossRef ADS arXiv Google Scholar

[9] Miller D A B. Self-configuring universal linear optical component [Invited]. Photon Res, 2013, 1: 1 CrossRef Google Scholar

[10] Pérez D, Gasulla I, Crudgington L. Multipurpose silicon photonics signal processor core. Nat Commun, 2017, 8: 636 CrossRef PubMed ADS Google Scholar

[11] Perez D, Gasulla I, Capmany J. Toward Programmable Microwave Photonics Processors. J Lightwave Technol, 2018, 36: 519-532 CrossRef ADS Google Scholar

[12] Zhang J, Yao J. A Microwave Photonic Signal Processor for Arbitrary Microwave Waveform Generation and Pulse Compression. J Lightwave Technol, 2016, 34: 5610-5615 CrossRef ADS Google Scholar

[13] García-Meca C, Lechago S, Brimont A. On-chip wireless silicon photonics: from reconfigurable interconnects to lab-on-chip devices. Light Sci Appl, 2017, 6: e17053-e17053 CrossRef PubMed ADS Google Scholar

[14] Silver D, Huang A, Maddison C J. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529: 484-489 CrossRef PubMed ADS Google Scholar

[15] Silver D, Schrittwieser J, Simonyan K. Mastering the game of Go without human knowledge. Nature, 2017, 550: 354-359 CrossRef PubMed ADS Google Scholar

[16] Topol E J. High-performance medicine: the convergence of human and artificial intelligence.. Nat Med, 2019, 25: 44-56 CrossRef PubMed Google Scholar

[17] Abdel-Hamid O, Mohamed A, Jiang H. Convolutional Neural Networks for Speech Recognition. IEEE/ACM Trans Audio Speech Lang Process, 2014, 22: 1533-1545 CrossRef Google Scholar

[18] Brown N, Sandholm T. Superhuman AI for multiplayer poker. Science, 2019, 365: 885-890 CrossRef PubMed ADS Google Scholar

[19] Winfield A. Ethical standards in robotics and AI. Nat Electron, 2019, 2: 46-48 CrossRef Google Scholar

[20] Wang J G, Zhou L B. Traffic Light Recognition With High Dynamic Range Imaging and Deep Learning. IEEE Trans Intell Transp Syst, 2019, 20: 1341-1352 CrossRef Google Scholar

[21] Minasian R A. Photonic signal processing of microwave signals. IEEE Trans Microwave Theor Techn, 2006, 54: 832-846 CrossRef ADS Google Scholar

[22] Miller D A B. Perfect Optics With Imperfect Components. Optica, 2015, 2: 747-750 CrossRef ADS Google Scholar

[23] Yang G, Zou W, Yu L. Compensation of multi-channel mismatches in high-speed high-resolution photonic analog-to-digital converter. Opt Express, 2016, 24: 24061-24074 CrossRef PubMed ADS Google Scholar

[24] Minzioni P, Alberti F, Schiffini A. Techniques for Nonlinearity Cancellation Into Embedded Links by Optical Phase Conjugation. J Lightwave Technol, 2005, 23: 2364-2370 CrossRef ADS Google Scholar

[25] Park S W, Park J Y, Bong K, et al. An energy-efficient and scalable deep learning/inference processor with tetra-parallel MIMD architecture for big data applications. IEEE Trans Biomed Circ Syst, 2015, 9: 838--848. Google Scholar

[26] Waldrop M M. The chips are down for Moore's law.. Nature, 2016, 530: 144-147 CrossRef PubMed ADS Google Scholar

[27] Tait A N, Ferreira de Lima T, Nahmias M A. Silicon Photonic Modulator Neuron. Phys Rev Appl, 2019, 11: 064043 CrossRef ADS arXiv Google Scholar

[28] Denève S, Alemi A, Bourdoukan R. The Brain as an Efficient and Robust Adaptive Learner.. Neuron, 2017, 94: 969-977 CrossRef PubMed Google Scholar

[29] Roy K, Jaiswal A, Panda P. Towards spike-based machine intelligence with neuromorphic computing. Nature, 2019, 575: 607-617 CrossRef PubMed ADS Google Scholar

[30] Maass W, Natschl?ger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations.. Neural Computation, 2002, 14: 2531-2560 CrossRef PubMed Google Scholar

[31] Ma W, Zidan M A, Lu W D. Neuromorphic computing with memristive devices. Sci China Inf Sci, 2018, 61: 060422. Google Scholar

[32] Wu N J. Neuromorphic vision chips. Sci China Inf Sci, 2018, 61: 060421. Google Scholar

[33] Yan B N, Chen Y R, Li H. Challenges of memristor based neuromorphic computing system. Sci China Inf Sci, 2018, 61: 060425. Google Scholar

[34] Cully A, Clune J, Tarapore D. Robots that can adapt like animals. Nature, 2015, 521: 503-507 CrossRef PubMed ADS arXiv Google Scholar

[35] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica, 2019, 6: 921-943 CrossRef ADS Google Scholar

[36] Rivenson Y, G?r?cs Z, Günaydin H. Deep learning microscopy. Optica, 2017, 4: 1437-1443 CrossRef ADS arXiv Google Scholar

[37] Sinha A, Lee J, Li S. Lensless computational imaging through deep learning. Optica, 2017, 4: 1117-1125 CrossRef ADS arXiv Google Scholar

[38] Wu Y, Rivenson Y, Zhang Y. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica, 2018, 5: 704-710 CrossRef ADS arXiv Google Scholar

[39] Zhang X, Chen Y, Ning K. Deep learning optical-sectioning method. Opt Express, 2018, 26: 30762-30772 CrossRef PubMed ADS Google Scholar

[40] Manifold B, Thomas E, Francis A T. Denoising of stimulated Raman scattering microscopy images via deep learning.. Biomed Opt Express, 2019, 10: 3860-3874 CrossRef PubMed Google Scholar

[41] Esman D J, Ataie V, Kuo B P P. Comb-Assisted Cyclostationary Analysis of Wideband RF Signals. J Lightwave Technol, 2017, 35: 3705-3712 CrossRef ADS Google Scholar

[42] Ma M, Adams R, Chen L R. Integrated Photonic Chip Enabled Simultaneous Multichannel Wideband Radio Frequency Spectrum Analyzer. J Lightwave Technol, 2017, 35: 2622-2628 CrossRef ADS Google Scholar

[43] Fortier T, Baumann E. 20 years of developments in optical frequency comb technology and applications. Commun Phys, 2019, 2: 153 CrossRef ADS arXiv Google Scholar

[44] Hammond A M, Camacho R M. Designing integrated photonic devices using artificial neural networks. Opt Express, 2019, 27: 29620-29638 CrossRef PubMed ADS arXiv Google Scholar

[45] Malkiel I, Mrejen M, Nagler A. Plasmonic nanostructure design and characterization via Deep Learning. Light Sci Appl, 2018, 7: 60 CrossRef PubMed ADS Google Scholar

[46] Laporte F, Dambre J, Bienstman P. Highly parallel simulation and optimization of photonic circuits in time and frequency domain based on the deep-learning framework PyTorch. Sci Rep, 2019, 9: 5918 CrossRef PubMed ADS Google Scholar

[47] Zahavy T, Dikopoltsev A, Moss D. Deep learning reconstruction of ultrashort pulses. Optica, 2018, 5: 666-673 CrossRef ADS arXiv Google Scholar

[48] Xu S, Zou X, Ma B. Deep-learning-powered photonic analog-to-digital conversion. Light Sci Appl, 2019, 8: 66 CrossRef PubMed ADS Google Scholar

[49] Zou X, Xu S, Li S. Optimization of the Brillouin instantaneous frequency measurement using convolutional neural networks. Opt Lett, 2019, 44: 5723-5726 CrossRef PubMed ADS Google Scholar

[50] Shen Y, Harris N C, Skirlo S. Deep learning with coherent nanophotonic circuits. Nat Photon, 2017, 11: 441-446 CrossRef ADS arXiv Google Scholar

[51] Lin X, Rivenson Y, Yardimci N T. All-optical machine learning using diffractive deep neural networks. Science, 2018, 361: 1004-1008 CrossRef PubMed ADS arXiv Google Scholar

[52] Hamerly R, Bernstein L, Sludds A. Large-Scale Optical Neural Networks Based on Photoelectric Multiplication. Phys Rev X, 2019, 9: 021032 CrossRef ADS arXiv Google Scholar

[53] Bangari V, Marquez B A, Miller H. Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs). IEEE J Sel Top Quantum Electron, 2020, 26: 1-13 CrossRef ADS arXiv Google Scholar

[54] Williamson I A D, Hughes T W, Minkov M. Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks. IEEE J Sel Top Quantum Electron, 2020, 26: 1-12 CrossRef ADS arXiv Google Scholar

[55] George J K, Mehrabian A, Amin R. Neuromorphic photonics with electro-absorption modulators. Opt Express, 2019, 27: 5181-5191 CrossRef PubMed ADS arXiv Google Scholar

[56] Zuo Y, Li B, Zhao Y. All-optical neural network with nonlinear activation functions. Optica, 2019, 6: 1132-1137 CrossRef ADS arXiv Google Scholar

[57] Mourgias-Alexandris G, Tsakyridis A, Passalis N. An all-optical neuron with sigmoid activation function. Opt Express, 2019, 27: 9620-9630 CrossRef PubMed ADS Google Scholar

[58] Miscuglio M, Mehrabian A, Hu Z. All-optical nonlinear activation function for photonic neural networks [Invited]. Opt Mater Express, 2018, 8: 3851-3863 CrossRef ADS arXiv Google Scholar

[59] Hughes T W, Minkov M, Shi Y. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica, 2018, 5: 864-871 CrossRef ADS arXiv Google Scholar

[60] Xu S, Wang J, Wang R. High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays. Opt Express, 2019, 27: 19778 CrossRef PubMed ADS Google Scholar

[61] Xu S F, Wang J, Zou W W. High-energy-efficiency integrated photonic convolutional neural networks,. arXiv Google Scholar

[62] Prucnal P R, Shastri B J. Neuromorphic Photonics. Boca Raton: CRC Press, 2017. Google Scholar

[63] Nahmias M A, Shastri B J, Tait A N. A Leaky Integrate-and-Fire Laser Neuron for Ultrafast Cognitive Computing. IEEE J Sel Top Quantum Electron, 2013, 19: 1-12 CrossRef ADS Google Scholar

[64] Robertson J, Wade E, Kopp Y. Toward Neuromorphic Photonic Networks of Ultrafast Spiking Laser Neurons. IEEE J Sel Top Quantum Electron, 2020, 26: 1-15 CrossRef ADS Google Scholar

[65] Xiang S Y, Zhang H, Guo X X. Cascadable Neuron-Like Spiking Dynamics in Coupled VCSELs Subject to Orthogonally Polarized Optical Pulse Injection. IEEE J Sel Top Quantum Electron, 2017, 23: 1-7 CrossRef ADS Google Scholar

[66] Prucnal P R, Shastri B J, Ferreira de Lima T. Recent progress in semiconductor excitable lasers for photonic spike processing. Adv Opt Photon, 2016, 8: 228-299 CrossRef ADS Google Scholar

[67] Chakraborty I, Saha G, Roy K. Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials. Phys Rev Appl, 2019, 11: 014063 CrossRef ADS Google Scholar

[68] Xiang S, Ren Z, Zhang Y. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on a VCSEL-SA.. Opt Lett, 2020, 45: 1104-1107 CrossRef PubMed Google Scholar

[69] Cheng Z, Ríos C, Pernice W H P. On-chip photonic synapse. Sci Adv, 2017, 3: e1700160 CrossRef PubMed ADS Google Scholar

[70] Tait A N, Nahmias M A, Shastri B J. Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing. J Lightwave Technol, 2014, 32: 4029-4041 CrossRef ADS Google Scholar

[71] Feldmann J, Youngblood N, Wright C D. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 2019, 569: 208-214 CrossRef PubMed ADS Google Scholar

[72] Xiang S, Zhang Y, Gong J. STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs. IEEE J Sel Top Quantum Electron, 2019, 25: 1-9 CrossRef ADS Google Scholar

[73] Ren Q, Zhang Y, Wang R. Optical spike-timing-dependent plasticity with weight-dependent learning window and reward modulation. Opt Express, 2015, 23: 25247-25258 CrossRef PubMed ADS Google Scholar

[74] Toole R, Tait A N, Ferreira de Lima T. Photonic Implementation of Spike-Timing-Dependent Plasticity and Learning Algorithms of Biological Neural Systems. J Lightwave Technol, 2016, 34: 470-476 CrossRef ADS Google Scholar

[75] Fok M P, Tian Y, Rosenbluth D. Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. Opt Lett, 2013, 38: 419-421 CrossRef PubMed ADS Google Scholar

[76] Ma B W, Chen J P, Zou W W. A DFB-LD-based photonic neuromorphic network for spatiotemporal pattern recognition. In: Proceedings of Optical Fiber Communication Conference, San Diego, 2020. M2K.2. Google Scholar

[77] Smit M, Leijtens X. Integration of passive and active components in InP-Based PICs In: Proceedings of Advances in Optical Sciences Congress, Honolulu, 2009. ITuB2. Google Scholar

[78] van Emmerik C I, Dijkstra M, de Goede M. Single-layer active-passive Al2O3 photonic integration platform. Opt Mater Express, 2018, 8: 3049-3054 CrossRef ADS Google Scholar

[79] de Valicourt G, Chang C M, Eggleston M S. Photonic Integrated Circuit Based on Hybrid III-V/Silicon Integration. J Lightwave Technol, 2018, 36: 265-273 CrossRef ADS Google Scholar

[80] Yoo S J B, Guan B, Scott R P. Heterogeneous 2D/3D Photonic Integrated Microsystems. Microsyst Nanoeng, 2016, 2: 16030 CrossRef PubMed ADS Google Scholar

[81] Hill M, Smit M, Crombez P, et al. Digital vs. Analog photonic integration In: Proceedings of Integrated Photonics and Nanophotonics Research and Applications, Boston, 2008. IWC1. Google Scholar

[82] Atabaki A H, Moazeni S, Pavanello F. Integrating photonics with silicon nanoelectronics for the next generation of systems on a chip. Nature, 2018, 556: 349-354 CrossRef PubMed ADS Google Scholar

[83] Sengupta K, Nagatsuma T, Mittleman D M. Terahertz integrated electronic and hybrid electronic-photonic systems. Nat Electron, 2018, 1: 622-635 CrossRef Google Scholar

  • Figure 1

    (Color online) The hierarchy of the IPS concept. DEC: digital electronic circuit.

  • Figure 2

    (Color online) The architecture of an AI-powered IPS.

  • Figure 3

    (Color online) (a) The training process of the DNN for microscopic imaging; (b) after training, the output of the DNN shows improved performance [35]@Copyright 2017 The Optical Society; (c) schematic of the deep learning microscopy.

  • Figure 4

    (Color online) Schematic of the DL-PADC architecture [44]@Copyright 2019 Springer Nature.

  • Figure 5

    (Color online) Optimized results of different signal formats using CNN-based method in BIFM, including linear frequency modulation (LFM) (up-chirp) (a), LFM (down-chirp) (b), nonlinear frequency modulation (NLFM) (c), binary a frequency-shift keying (BFSK) (d), and Costas frequency modulation (e) [45]@Copyright 2019 The Optical Society.

  • Figure 6

    (Color online) The architecture of an IPS with OANN-facilitated AI.

  • Figure 7

    (Color online) (a) Operation process of a two-layer OANN. (b) Feedback loop introduced in the experiment. protectłinebreak (c) The architecture of MZI-based OANN, which is tunable by the accompanied phase shifters as shown in (d) [50]@Copyright 2017 Springer Nature.

  • Figure 8

    (Color online) (a) Schematic of the diffraction-based OANN with multiple diffractive layers. The OANN implemented in experiments as a classifier (b) and an imager (c) [51]@Copyright 2018 The AAAS.

  • Figure 9

    (Color online) (a) Multi-layer schematic of the neural network; (b) the architecture of a single-layer OANN with coherent detection [52]@Copyright 2019 American Physical Society.

  • Figure 10

    (Color online) (a) A convolution using DEAP; (b) two convolutional units to perform a convolution [53]@Copyright 2020 IEEE.

  • Figure 11

    (Color online) (a) The optical convolution unit architecture; (b) the transmission rate versus the modulation voltage of the used modulators; (c) an illustration of the serialization method [60]@Copyright 2019 The Optical Society.

  • Figure 12

    (Color online) (a) Schematic of the CNN implementation with optical delay lines and the WDM; (b) detailed structure of the vector-matrix multiplication core [61].

  • Figure 13

    (Color online) The architecture of an IPS with OBNN-facilitated AI.

  • Figure 14

    (Color online) Simulation results of the laser with an SA featuring excitability. An optical pulse is released when the input perturbations accumulate to a threshold [63]@Copyright 2013 IEEE.

  • Figure 15

    (Color online) (a) The schematic of the photonic neuron with PCM (top) and the TE mode distribution (bottom); (b) the photonic synapse resembles the function of the biology synapse; (c) experimental setup of STDP realization by the photonic synapse [69]@Copyright 2017 The AAAS.

  • Figure 16

    (Color online) Broadcast-and-weight scheme, including a laser array, the WDM, spectral filters, and the waveguide loop [70]@Copyright 2014 IEEE.

  • Figure 17

    (Color online) (a) and (b) Schematic of the PCM-based spiking neural network; (c) the diagram of an integrated optical neuron; (d) optical micrograph of three optical neurons with four input ports respectively [71]@Copyright 2019 Springer Nature.

  • Figure 18

    (Color online) The experimental results of the sound azimuth measurement with DFB-LDs. The outputs when the delay is 3 $\mu$s (a) and 4 $\mu$s (b). The 2nd peak difference dependent on the delay (c) and the sound azimuth (d).

  • Figure 19

    (Color online) The DFB-LD-based spatiotemporal pattern recognition network with STDP learning module [72]@Copyright 2020 The Authors.

  • Table 1  

    Table 1Progress in large-scale hybrid integration

    Hybrid typeReferenceHighlights
    Active/Passive[77]Passive building blocks in the generic integration technology
    [78]First one-layer active-passive Al$_2$O$_{3}$ photonic integration
    Heterogenous[79]Systematical reviews on III-V/Silicon integration
    [80]Heterogeneous 2D/3D photonic integration
    Digital/Analog[81]Discussion on digital photonic integrated circuits
    [6]70 million transistors and 850 photonic components on a chip
    Photonic/Electronic[82]A way to integrate photonics with state-of-the-art nanoelectronics
    [83]Terahertz integrated hybrid electronic–photonic systems

Copyright 2020  CHINA SCIENCE PUBLISHING & MEDIA LTD.  中国科技出版传媒股份有限公司  版权所有

京ICP备14028887号-23       京公网安备11010102003388号