logo

SCIENCE CHINA Information Sciences, Volume 59, Issue 9: 092103(2016) https://doi.org/10.1007/s11432-015-0957-4

Robust dense reconstruction by range merging based on confidence estimation

More info
  • ReceivedJan 18, 2016
  • AcceptedMar 7, 2016
  • PublishedAug 18, 2016

Abstract

Although the stereo matching problem has been extensively studied during the past decades, automatically computing a dense 3D reconstruction from several multiple views is still a difficult task owing to the problems of textureless regions, outliers, detail loss, and various other factors. In this paper, these difficult problems are handled effectively by a robust model that outputs an accurate and dense reconstruction as the final result from an input of multiple images captured by a normal camera. First, the positions of the camera and sparse 3D points are estimated by a structure-from-motion algorithm and we compute the range map with a confidence estimation for each image in our approach. Then all the range maps are integrated into a fine point cloud data set. In the final step we use a Poisson reconstruction algorithm to finish the reconstruction. The major contributions of the work lie in the following points: effective range-computation and confidence-estimation methods are proposed to handle the problems of textureless regions, outliers and detail loss. Then, the range maps are merged into the point cloud data in terms of a confidence-estimation. Finally, Poisson reconstruction algorithm completes the dense mesh. In addition, texture mapping is also implemented as a post-processing work for obtaining good visual effects. Experimental results are presented to demonstrate the effectiveness of the proposed approach.


Funded by

National Natural Science Foundation of China(61272326)

Startup Foundation for Introducing Talent of NUIST(2243141601013)


Acknowledgment

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant No. 61272326) and Startup Foundation for Introducing Talent of NUIST (Grant No. 2243141601013).


References

[1] Chen J W, Bautembach D, Izadi S. Scalable real-time volumetric surface reconstruction. ACM Trans Graphic, 2013, 32: 1-16 Google Scholar

[2] Chen Y D, Hao C Y, Wu W, et al. Recursive video segmentation (in Chinese). Sci Sin Inform, 2014, 44: 1361-1369 Google Scholar

[3] Hao C Y, Chen Y D, Wu W, et al. Image completion with perspective constraint based on a single image. Sci China Inf Sci, 2015, 58: 092109-1369 Google Scholar

[4] Collins R T. A space-sweep approach to true multi-image matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 1996. 358--363. Google Scholar

[5] Seitz S M, Dyer C R. Photorealistic scene reconstruction by voxel coloring. Int J Comput Vision, 1999, 35: 151-173 CrossRef Google Scholar

[6] Newcombe R A, Davison A J. Live dense reconstruction with a single moving camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 2010. 1498--1505. Google Scholar

[7] Chen Y D, Hao C Y, Wu W, et al. Live accurate and dense reconstruction from a handheld camera. Comput Animat Virtual Worlds, 2013, 24: 387-397 CrossRef Google Scholar

[8] Furukawa Y, Ponce J. Accurate, dense, and robust multiview stereopsis. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 1362-1376 CrossRef Google Scholar

[9] Furukawa Y, Curless B, Seitz S M, et al. Towards internet-scale multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 2010. 1434--1441. Google Scholar

[10] Wang Y, Ji X Y, Dai Q H. Key technologies of light field capture for 3d reconstruction in microscopic scene. Sci China Inf Sci, 2010, 53: 1917-1930 CrossRef Google Scholar

[11] Agarwal S, Furukawa Y, Snavely N, et al. Building rome in a day. Commun ACM, 2011, 54: 105-112 CrossRef Google Scholar

[12] Sun J, Li Y, Kang S B, et al. Symmetric stereo matching for occlusion handling. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, 2005. 399--406. Google Scholar

[13] Strecha C, Fransens R, van Gool L. Combined depth and outlier estimation in multi-view stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, 2006. 2394--2401. Google Scholar

[14] Ryan K, Camillo J T. Optical flow with geometric occlusion estimation and fusion of multiple frames. In: Proceedings of the 10th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, Hongkong, 2015. 364--377. Google Scholar

[15] Zhang G F, Jia J Y, Hua W, et al. Robust bilayer segmentation and motion depth estimation with a handheld camera. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 603-617 CrossRef Google Scholar

[16] Felzenszwalb P F, Huttenlocher D P. Efficient belief propagation for early vision. Int J Comput Vision, 2006, 70: 41-54 CrossRef Google Scholar

[17] Woodford O, Torr P, Reid I, et al. Global stereo reconstruction under second-order smoothness priors. IEEE Trans Pattern Anal Mach Intell, 2009, 31: 2115-2128 CrossRef Google Scholar

[18] Fuhrmann S, Goesele M. Fusion of depth maps with multiple scales. In: Proceedings of the SIGGRAPH Asia Conference. New York: ACM, 2011. 1--8. Google Scholar

[19] Zach C, Pock T, Bischof H. A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the IEEE International Conference on Computer Vision, Rio de Janeiro, 2007. 1--8. Google Scholar

[20] Stühmer J, Gumhold S, Cremers D. Real-time dense geometry from a handheld camera. In: Proceedings of the Conference on Pattern Recognition, Lecture Notes in Computer Science, Berlin, 2010. 11--20. Google Scholar

[21] Graber G, Pock T, Bischof H. Online 3d reconstruction using convex optimization. In: Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, 2011. 708--711. Google Scholar

[22] Bischof H. Dense reconstruction on-the-fly. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, 2012. 1450--1457. Google Scholar

[23] Shan Q, Curless B, Furukawa Y, et al. Occluding contours for multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, 2014. 4002--4009. Google Scholar

[24] Snavely N, Seitz S M, Szeliski R. Photo tourism: exploring photo collections in 3D. In: Proceedings of the ACM SIGGRAPH, New York, 2006. 835--846. Google Scholar

[25] Wu C C, Agarwal S, Curless B, et al. Multicore bundle adjustment. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Providence, 2011. 3057--3064. Google Scholar

[26] Dong Z L, Zhang G F, Jia J Y, et al. Efficient keyframe-based real-time camera tracking. Comput Vision Image Und, 2014, 118: 97-110 CrossRef Google Scholar

[27] Kazhdan M, Hoppe H. Screened Poisson surface reconstruction. ACM Trans Graphic, 2013, 32: 1-13 Google Scholar

[28] Kang S B, Szeliski R. Extracting view-dependent depth maps from a collection of images. Int J Comput Vision, 2004, 58: 139-163 CrossRef Google Scholar

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1