logo

SCIENTIA SINICA Informationis, Volume 49, Issue 12: 1606-1625(2019) https://doi.org/10.1360/SSI-2019-0100

A 3D building reconstruction method for SAR images based on deep neural network

Jiankun CHEN1,2,3,5, Lingxiao PENG1,2,4,5, Xiaolan QIU1,2,4,5,*, Chibiao DING1,4,5, Yirong WU5
More info
  • ReceivedMay 14, 2019
  • AcceptedAug 13, 2019
  • PublishedDec 16, 2019

Abstract

Owing to its momentous applications in urban planning, disaster monitoring, smart cities and other fields, 3D building reconstruction is an important topic in many research areas such as computer vision, photogrammetry, and remote sensing. The particularity and complexity of the microwave scattering mechanism bring great challenges to the 3D building reconstruction of SAR images, and the applicability and automation of existing methods need to be improved. This study constructs the overall framework of building detection in SAR images and 3D reconstruction based on deep learning and radar imaging mechanism. It puts forward a method of using a coupled equivalent complex valued convolutional neural network for building facade detection in SAR images, a method for RaySAR-based modeling simulation and point cloud generation for 3D model training, and a method for a 3D generation network for 3D building reconstruction from SAR images. Experiments using TerraSAR-X and GF-3 high resolution SAR images are carried out, producing good 3D reconstruction results. The proposed method is a useful attempt for 3D target reconstruction of SAR images and represents a new technical approach.


Funded by

国家自然科学基金(61725105)


References

[1] Gamba P, Houshmand B, Saccani M. Detection and extraction of buildings from interferometric SAR data. IEEE Trans Geosci Remote Sens, 2000, 38: 611-617 CrossRef ADS Google Scholar

[2] Bolter R. Reconstruction of man-made objects from high resolution SAR images. In: Proceedings of 2000 IEEE Aerospace Conference, 2000. 287--292. Google Scholar

[3] Zhao L J, Zhou X G, Kuang G Y. Building detection from urban SAR image using building characteristics and contextual information. EURASIP J Adv Signal Process, 2013, 2013(1): 56 CrossRef ADS Google Scholar

[4] Fu X Y, You H J, Fu K. 3-D reconstruction of building from single high-resolution SAR image. J Infrared Millim W, 2012, 31: 569--576 [傅兴玉, 尤红建, 付琨. 单幅高分辨率SAR图像建筑三维模型重构. 红外与毫米波学报, 2012, 31: 569--576. Google Scholar

[5] Xu F, Jin Y Q. Automatic Reconstruction of Building Objects From Multiaspect Meter-Resolution SAR Images. IEEE Trans Geosci Remote Sens, 2007, 45: 2336-2353 CrossRef ADS Google Scholar

[6] Pontes J K, Kong C, Sridharan S, et al. Image2mesh: a learning framework for single image 3D reconstruction. In: Proceedings of Asian Conference on Computer Vision. Berlin: Springer, 2018. 365--381. Google Scholar

[7] Kurenkov A, Ji J, Garg A, et al. Deformnet: Free-form deformation network for 3D shape reconstruction from a single image. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2018. 858--866. Google Scholar

[8] Kniaz V V, Remondino F, Knyaz V A. Generative Adversarial Networks for Single Photo 3d Reconstruction. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2019, XLII-2/W9: 403-408 CrossRef ADS Google Scholar

[9] Mandikal P, Murthy N, Agarwal M, et al. 3DLMNet: latent embedding matching for accurate and diverse 3D point cloud reconstruction from a single image. 2018,. arXiv Google Scholar

[10] Fan H Q, Su H, Guibas L J. A point set generation network for 3d object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 605--613. Google Scholar

[11] LeCun Y, Boser B, Denker J S. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1989, 1: 541-551 CrossRef Google Scholar

[12] Lecun Y, Bottou L, Bengio Y. Gradient-based learning applied to document recognition. Proc IEEE, 1998, 86: 2278-2324 CrossRef Google Scholar

[13] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems. 2012. 1097--1105. Google Scholar

[14] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2014. 818--833. Google Scholar

[15] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014,. arXiv Google Scholar

[16] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 770--778. Google Scholar

[17] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431--3440. Google Scholar

[18] Duan Y P, Tao X M, Han C Y, et al. Multi-scale convolutional neural network for SAR image semantic segmentation. In: Proceedings of IEEE Global Communications Conference, 2018. 1--6. Google Scholar

[19] Xu F, Wang H P, Jin Y Q. Deep learning as applied in SAR target recognition and terrain classification. J Radars, 2017, 6(2): 136-148 DOI: 10.12000/JR16130. Google Scholar

[20] Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. 2015,. arXiv Google Scholar

[21] Niu S R, Qiu X L, Peng L X, et al. Parameter prediction method of SARtarget simulation based on convolutional neural networks. In: Proceedings of European Conference on Synthetic Aperture Radar, 2018. 1--5. Google Scholar

[22] Xiang Y, Mottaghi R, Savarese S. Beyond PASCAL: a benchmark for 3D object detection in the wild. In: Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE, 2014. 75--82. Google Scholar

[23] Chang A X, Funkhouser T, Guibas L, et al. Shapenet: an information-rich 3D model repository. 2015,. arXiv Google Scholar

[24] Wu N Z, Song S, Khosla A, et al. 3D ShapeNets: a deep representation for volumetric shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 1912--1920. Google Scholar

[25] Lim J J, Pirsiavash H, Torralba A. Parsing Ikea objects: fine pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, 2013. Google Scholar

[26] Choy C B, Xu D, Gwak J Y, et al. 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016. 628--644. Google Scholar

[27] Wang N Y, Zhang Y D, Li Z W, et al. Pixel2Mesh: generating 3D mesh models from single RGB images. In: Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018. 55--71. Google Scholar

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1