logo

SCIENTIA SINICA Informationis, Volume 49, Issue 12: 1626-1639(2019) https://doi.org/10.1360/SSI-2019-0093

Progress of deep learning-based target recognition in radar images

Zongxu PAN1,2,3,*, Quanzhi AN1,2,3, Bingchen ZHANG1,2,3
More info
  • ReceivedMay 8, 2019
  • AcceptedJun 12, 2019
  • PublishedDec 10, 2019

Abstract

Radar detection is an effective earth observation means, and target recognition in radar images is its important research direction. Deep learning has been successfully applied to many fields but training deep neural networks requires a mass of data. The lack of samples has become the major factor that impedes the application of deep learning approaches to target recognition in radar images. This paper reviews the research progress of deep learning based target recognition in radar images, with representative methods being combed and summarized. First, data augmentation and neural network models designed for the task of radar image target recognition are introduced. The paper then presents in detail the target recognition methods based on transfer learning, metric learning, and semi-supervised learning in radar images with few samples, which are proposed by our research group. Finally, existing problems and future development trends are discussed.


Funded by

国家自然科学基金(61701478)


References

[1] Dudgeon D E, Lacoss R T. An overview of automatic target recognition. Lincoln Lab J, 1993, 6: 3--10. Google Scholar

[2] Pan Z, Liu L, Qiu X. Fast Vessel Detection in Gaofen-3 SAR Images with Ultrafine Strip-Map Mode.. Sensors, 2017, 17: 1578 CrossRef PubMed Google Scholar

[3] An Q, Pan Z, You H. Ship Detection in Gaofen-3 SAR Images Based on Sea Clutter Distribution Analysis and Deep Convolutional Neural Network.. Sensors, 2018, 18: 334 CrossRef PubMed Google Scholar

[4] Kang M, Ji K F, Leng X G. Contextual region-based convolutional neural network with multilayer fusion for SAR ship detection. Remote Sens, 2017, 9: 860 CrossRef ADS Google Scholar

[5] Liu L, Chen G W, Pan Z X, et al. Inshore ship detection in SAR images based on deep neural networks. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2018. 25--28. Google Scholar

[6] An Q Z, Pan Z X, Liu L, et al. DRBox-v2: an improved detector with rotatable boxes for target detection in SAR images. IEEE Trans Geosci Remote Sens, 2019, 57: 8333--8349. Google Scholar

[7] Kuang G Y, Ji K F, Su Y, et al. A survey of researches on SAR ATR. J Image Graph, 2003, 8: 1115--1120. Google Scholar

[8] Hu F M, Zhang P, Yang R L, et al. SAR target recognition based on Gabor filter and sub-block statistical feature. In: Proceedings of IET International Radar Conference, 2009. Google Scholar

[9] Papson S, Narayanan R M. Classification via the Shadow Region in SAR Imagery. IEEE Trans Aerosp Electron Syst, 2012, 48: 969-980 CrossRef ADS Google Scholar

[10] Zhang H, Tian X, Wang C. Merchant Vessel Classification Based on Scattering Component Analysis for COSMO-SkyMed SAR Images. IEEE Geosci Remote Sens Lett, 2013, 10: 1275-1279 CrossRef ADS Google Scholar

[11] Chen J H, Zhang B, Wang C. Backscattering Feature Analysis and Recognition of Civilian Aircraft in TerraSAR-X Images. IEEE Geosci Remote Sens Lett, 2015, 12: 796-800 CrossRef ADS Google Scholar

[12] Pan Z, Qiu X, Huang Z. Airplane Recognition in TerraSAR-X Images via Scatter Cluster Extraction and Reweighted Sparse Representation. IEEE Geosci Remote Sens Lett, 2017, 14: 112-116 CrossRef ADS Google Scholar

[13] Zhao Q, Principe J C. Support vector machines for SAR automatic target recognition. IEEE Trans Aerosp Electron Syst, 2001, 37: 643-654 CrossRef ADS Google Scholar

[14] Margarit G, Tabasco A. Ship Classification in Single-Pol SAR Images Based on Fuzzy Logic. IEEE Trans Geosci Remote Sens, 2011, 49: 3129-3138 CrossRef ADS Google Scholar

[15] Srinivas U, Monga V, Raj R G. SAR Automatic Target Recognition Using Discriminative Graphical Models. IEEE Trans Aerosp Electron Syst, 2014, 50: 591-606 CrossRef ADS Google Scholar

[16] Cui Z Y, Cao Z J, Yang J Y, et al. SAR target recognition using nonnegative matrix factorization with L$_{1/2}~$ constraint. In: Proceedings of IEEE Radar Conference, 2014. 382--386. Google Scholar

[17] Dong G, Kuang G, Wang N. SAR Target Recognition via Joint Sparse Representation of Monogenic Signal. IEEE J Sel Top Appl Earth Observations Remote Sens, 2015, 8: 3316-3328 CrossRef ADS Google Scholar

[18] Pan Z X, Liu L, Lei B. Identity regularized sparse representation for automatic target recognition in SAR images. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2018. 37--40. Google Scholar

[19] Huang Y, Peia J, Yanga J. Neighborhood Geometric Center Scaling Embedding for SAR ATR. IEEE Trans Aerosp Electron Syst, 2014, 50: 180-192 CrossRef ADS Google Scholar

[20] Liu X, Huang Y L, Pei J F. Sample discriminant analysis for SAR ATR. IEEE Geosci Remote Sens Lett, 2014, 11: 2120-2124 CrossRef ADS Google Scholar

[21] Dong G G, Kuang G Y. Target Recognition in SAR Images via Classification on Riemannian Manifolds. IEEE Geosci Remote Sens Lett, 2015, 12: 199-203 CrossRef ADS Google Scholar

[22] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012. 1097--1105. Google Scholar

[23] Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. Google Scholar

[24] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 770--778. Google Scholar

[25] Xu F, Wang H P, Jin Y Q. Deep learning as applied in SAR target recognition and terrain classification. Journal of Radars, 2017, 6: 136--148. Google Scholar

[26] Keydel E R, Lee S W, Moore J T. MSTAR extended operating conditions: a tutorial. In: Proceedings of SPIE 2757, Algorithms for Synthetic Aperture Radar Imagery III, 1996. 228--242. Google Scholar

[27] Huang L, Liu B, Li B. OpenSARShip: A Dataset Dedicated to Sentinel-1 Ship Interpretation. IEEE J Sel Top Appl Earth Observations Remote Sens, 2018, 11: 195-208 CrossRef ADS Google Scholar

[28] Ding J, Chen B, Liu H. Convolutional Neural Network With Data Augmentation for SAR Target Recognition. IEEE Geosci Remote Sens Lett, 2016, : 1-5 CrossRef Google Scholar

[29] Wagner S A. SAR ATR by a combination of convolutional neural network and support vector machines. IEEE Trans Aerosp Electron Syst, 2016, 52: 2861-2872 CrossRef ADS Google Scholar

[30] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014. 2672--2680. Google Scholar

[31] Guo J, Lei B, Ding C. Synthetic Aperture Radar Image Synthesis by Using Generative Adversarial Nets. IEEE Geosci Remote Sens Lett, 2017, 14: 1111-1115 CrossRef ADS Google Scholar

[32] Bao X J, Pan Z X, Liu L, et al. SAR image simulation by generative adversarial networks. In: Proceedings of IGARSS, 2019. 9995--9998. Google Scholar

[33] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. 2015,. arXiv Google Scholar

[34] Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. 2017,. arXiv Google Scholar

[35] Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of Wasserstein GANs. 2017,. arXiv Google Scholar

[36] Cui Z, Zhang M, Cao Z. Image Data Augmentation for SAR Sensor via Generative Adversarial Nets. IEEE Access, 2019, 7: 42255-42268 CrossRef Google Scholar

[37] Auer S J. 3D synthetic aperture radar simulation for interpreting complex urban reflection scenarios. Dissertation for Ph.D. Degree. Munich: Technical University of Munich, 2011. Google Scholar

[38] Niu S R, Qiu X L, Peng L X, et al. Parameter prediction method of SAR target simulation based on convolutional neural networks. In: Proceedings of the 12th European Conference on Synthetic Aperture Radar, 2018. 106--110. Google Scholar

[39] Liu L, Pan Z X, Qiu X L, et al. SAR target classification with CycleGAN transferred simulated samples. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2018. 4411--4414. Google Scholar

[40] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), 2017. 2242--2251. Google Scholar

[41] Sun Z J, Xue L, Xu Y M. Recognition of SAR target based on multilayer auto-encoder and SNN. Int J Innov Comput Inf Control, 2013, 9: 4331--4341. Google Scholar

[42] Li X, Li C S, Wang P B, et al. SAR ATR based on dividing CNN into CAE and SNN. In: Proceedings of the 5th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR), 2015. 676--679. Google Scholar

[43] Geng J, Fan J, Wang H. High-Resolution SAR Image Classification via Deep Convolutional Autoencoders. IEEE Geosci Remote Sens Lett, 2015, 12: 2351-2355 CrossRef ADS Google Scholar

[44] Bentes C, Velotto D, Lehner S. Target classification in oceanographic SAR images with deep neural networks: architecture and initial results. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2015. 3703--3706. Google Scholar

[45] Chen S, Wang H, Xu F. Target Classification Using the Deep Convolutional Networks for SAR Images. IEEE Trans Geosci Remote Sens, 2016, 54: 4806-4817 CrossRef ADS Google Scholar

[46] Lin Z, Ji K, Kang M. Deep Convolutional Highway Unit Network for SAR Target Classification With Limited Labeled Training Data. IEEE Geosci Remote Sens Lett, 2017, 14: 1091-1095 CrossRef ADS Google Scholar

[47] Shang R, Wang J, Jiao L. SAR Targets Classification Based on Deep Memory Convolution Neural Networks and Transfer Parameters. IEEE J Sel Top Appl Earth Observations Remote Sens, 2018, 11: 2834-2846 CrossRef ADS Google Scholar

[48] Song Q, Xu F. Zero-Shot Learning of SAR Target Feature Space With Deep Generative Neural Networks. IEEE Geosci Remote Sens Lett, 2017, 14: 2245-2249 CrossRef ADS Google Scholar

[49] Xu B, Chen B, Liu H W, et al. Attention-based recurrent neural network model for radar high-resolution range profile target recognition. J Electron Inform Technol, 2016, 38: 2988--2995. Google Scholar

[50] Pan M, Jiang J, Kong Q. Radar HRRP Target Recognition Based on t-SNE Segmentation and Discriminant Deep Belief Network. IEEE Geosci Remote Sens Lett, 2017, 14: 1609-1613 CrossRef ADS Google Scholar

[51] Pei J, Huang Y, Sun Z. Multiview Synthetic Aperture Radar Automatic Target Recognition Optimization: Modeling and Implementation. IEEE Trans Geosci Remote Sens, 2018, 56: 6425-6439 CrossRef ADS Google Scholar

[52] He Z, Xiao H, Tian Z. Multi-View Tensor Sparse Representation Model for SAR Target Recognition. IEEE Access, 2019, 7: 48256-48265 CrossRef Google Scholar

[53] Zhang F, Hu C, Yin Q. Multi-Aspect-Aware Bidirectional LSTM Networks for Synthetic Aperture Radar Target Recognition. IEEE Access, 2017, 5: 26880-26891 CrossRef Google Scholar

[54] Zhou Y, Wang H, Xu F. Polarimetric SAR Image Classification Using Deep Convolutional Neural Networks. IEEE Geosci Remote Sens Lett, 2016, 13: 1935-1939 CrossRef ADS Google Scholar

[55] Zhang Z, Wang H, Xu F. Complex-Valued Convolutional Neural Network and Its Application in Polarimetric SAR Image Classification. IEEE Trans Geosci Remote Sens, 2017, 55: 7177-7188 CrossRef ADS Google Scholar

[56] Shang R, Wang G, A. Okoth M. Complex-Valued Convolutional Autoencoder and Spatial Pixel-Squares Refinement for Polarimetric SAR Image Classification. Remote Sens, 2019, 11: 522 CrossRef ADS Google Scholar

[57] Chen H, Zhang F, Tang B. Slim and Efficient Neural Network Design for Resource-Constrained SAR Target Recognition. Remote Sens, 2018, 10: 1618 CrossRef ADS Google Scholar

[58] Shao J, Qu C, Li J. A Lightweight Convolutional Neural Network Based on Visual Attention for SAR Image Target Classification.. Sensors, 2018, 18: 3039 CrossRef PubMed Google Scholar

[59] Min R, Lan H, Cao Z. A Gradually Distilled CNN for SAR Target Recognition. IEEE Access, 2019, 7: 42190-42200 CrossRef Google Scholar

[60] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. 2015,. arXiv Google Scholar

[61] Kang C Y, He C. SAR image classification based on the multi-layer network and transfer learning of mid-level representations. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016. 1146--1149. Google Scholar

[62] Marmanis D, Yao W, Adam F, et al. Artificial generation of big data for improving image classification: a generative adversarial network approach on SAR data. 2017,. arXiv Google Scholar

[63] Huang Z, Pan Z, Lei B. Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data. Remote Sens, 2017, 9: 907 CrossRef ADS Google Scholar

[64] Malmgren-Hansen D, Kusk A, Dall J. Improving SAR Automatic Target Recognition Models With Transfer Learning From Simulated Data. IEEE Geosci Remote Sens Lett, 2017, 14: 1484-1488 CrossRef ADS Google Scholar

[65] Huang Z L, Pan Z X, Lei B. What, where and how to transfer in SAR target recognition based on deep CNNs. IEEE Trans Geosci Remote Sens, 2019. doi: 10.1109/TGRS.2019.2947634. Google Scholar

[66] Borgwardt K M, Gretton A, Rasch M J. Integrating structured biological data by Kernel Maximum Mean Discrepancy.. Bioinformatics, 2006, 22: e49-e57 CrossRef PubMed Google Scholar

[67] Pan Z X, Bao X J, Wang B W, et al. Siamese network based metric learning for SAR target classification. In: Proceedings of IGARSS, 2019. 1342--1345. Google Scholar

[68] Bucher M, Herbin S, Jurie F. Hard negative mining for metric learning based zero-shot classification. In: Proceedings of European Conference on Computer Vision, 2016. 524--531. Google Scholar

[69] Chen G W, Liu L, Hu W L, et al. Semi-supervised object detection in remote sensing images using generative adversarial networks. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2018. 2503--2506. Google Scholar

[70] Salimans T, Goodfellow I, Zaremba W, et al. Improved techniques for training GANs. In: Proceedings of NIPS, 2016. 2234--2242. Google Scholar

[71] Zhou D Y, Bousquet O, Lal T N, et al. Learning with local and global consistency. In: Proceedings of NIPS, 2004. 284--291. Google Scholar

[72] Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules. In: Proceedings of NIPS, 2017. Google Scholar

[73] Thys S, Van Ranst W, Goedemé T. Fooling automated surveillance cameras: adversarial patches to attack person detection. 2019,. arXiv Google Scholar

  • Figure 1

    (Color online) Target recognition model architecture based on semi-supervised generative adversarial networks

  • Figure 2

    Samples generated from the generator. Generated samples after (a) 0 iterations, (b) 10000 iterations, andprotect łinebreak (c) 20000 iterations

  • Table 1   Comparison of recognition rates (%) of various methods
    SOC-10 SOC-20 SOC-30 EOC
    SGraph 56.45 69.86 72.49 62.90
    CNN 70.76 81.36 87.34 70.89
    SSGANs 85.65 89.86 93.07 75.50

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1