logo

SCIENTIA SINICA Informationis, Volume 51 , Issue 2 : 231(2021) https://doi.org/10.1360/SSI-2020-0334

A Web3D cloud rendering system for dynamic real-time lighting and shadow based on device power

More info
  • ReceivedOct 26, 2020
  • AcceptedDec 3, 2020
  • PublishedJan 12, 2021

Abstract


Funded by

国家自然科学基金面上项目(6207071897)

国家自然科学区域联合基金重点项目(U19A2063)


Supplement

Appendix

云渲染系统测试的设备环境与三维场景


References

[1] Zhao Q. Ten scientific and technical problems in virtual reality. Sci Sin-Inf, 2017, 47: 800-803 CrossRef Google Scholar

[2] Liu X, Xie N, Tang K. Lightweighting for Web3D visualization of large-scale BIM scenes in real-time. Graphical Model, 2016, 88: 40-56 CrossRef Google Scholar

[3] Chittaro L, Ranon R. Web3D technologies in learning, education and training. Comput Education, 2007, 49: 1-2 CrossRef Google Scholar

[4] Nah F F H. A study on tolerable waiting time: how long are Web users willing to wait?. Behaviour Inf Tech, 2004, 23: 153-163 CrossRef Google Scholar

[5] Shi S, Nahrstedt K, Campbell R. A real-time remote rendering system for interactive mobile graphics. ACM Trans Multimedia Comput Commun Appl, 2012, 8: 1-20 CrossRef Google Scholar

[6] Zhu M, Mondet S, Morin G, et al. Towards peer-assisted rendering in networked virtual environments. In: Proceedings of the 19th ACM International Conference on Multimedia, 2011. 183--192. Google Scholar

[7] Claypool M, Claypool K. Latency and player actions in online games. Commun ACM, 2006, 49: 40-45 CrossRef Google Scholar

[8] Rusinkiwicz S, Levoy M. QSplat: a multiresolution point rendering system for large meshes. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, 2000. 343--352. Google Scholar

[9] Sander P V,Snyder J, Gortler S J, et al. Texture mapping progressive meshes. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 2001. 409--416. Google Scholar

[10] Chen Z, Bodenheimer B, Barnes J F. Extending progressive meshes for use over unreliable networks. In: Proceedings of International Conference on Multimedia and Expo, 2003, 3: 253--256. Google Scholar

[11] Falby J S, Zyda M J, Pratt D R. NPSNET: Hierarchical data structures for real-time three-dimensional visual simulation. Comput Graphics, 1993, 17: 65-69 CrossRef Google Scholar

[12] Wei W, Jia J Y. An incremental SMLAOI algorithm for progressive downloading large scale Web3D scenes. In: Proceedings of Web3D'2009. pp.55--60. Google Scholar

[13] Dcoret X, Durand F, Sillion F X, et al. Billboard clouds for extreme model simplification. In: Proceedings of the 23th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2003. 689--696. Google Scholar

[14] Shade J, Gortler S, He L-W, et al. Layered depth images. In: Proceedings of SIGGRAPH 98 Conference Proceeding,1998. 231--242. Google Scholar

[15] Chang C F, Bishop G, Lastra A. LDI tree: a hierarchical representation for image-based rendering. In: Proceedings of SIGGRAPH 99, 1999. 291--298. Google Scholar

[16] Shi S, Kamali M, Nahrestedt K, et al. A high quality low-delay remote rendering system for 3D video. In: Proceedings of the International Conference on Multimedia. New York: ACM, 2010. 601--610. Google Scholar

[17] Crassin C, Luebke D, Mara M, et al. CloudLight: a system for amortizing indirect lighting in real-time rendering. J Comput Graph Tech, 2015, 4: 1--27. Google Scholar

[18] Cho K W, Seo J W, Kang J, et al. WingCache: application-aware caching for hybrid render farm. Int J Software Engineering and Its Appl, 2015, 9: 31--36 DOI: 10.14257/ijseia.2015.9.4.04. Google Scholar

[19] Zhao Z, Hwang K, Villeta J. Game cloud design with virtualized CPU/GPU servers and initial performance results. In: Proceedings of the 3rd Workshop on Scientific Cloud Computing. New York: ACM, 2012. 23--30. Google Scholar

[20] Shea R, Jiangchuan Liu R, Ngai E C H. Cloud gaming: architecture and performance. IEEE Network, 2013, 27: 16-21 CrossRef Google Scholar

[21] Huang C Y, Chen K T, Chen D Y. GamingAnywhere. ACM Trans Multimedia Comput Commun Appl, 2014, 10: 1-25 CrossRef Google Scholar

[22] Voglreiter P, Dokter M, Neff T. Shading atlasstreaming. In: Proceedings of SIGGRAPH Asia 2018 Technical Papers, 2018. 37: 199. Google Scholar

[23] Liu C, Ooi W T, Jia J. Cloud Baking. ACM Trans Multimedia Comput Commun Appl, 2018, 14: 1-20 CrossRef Google Scholar

[24] Bugeja K, Debattista K, Spina S. An asynchronousmethod for cloud-based rendering. Visual Comput,Springer Berlin Heidelberg (S0178-2789), 2018: 1-14. Google Scholar

[25] Wen L X, Jia J Y, Liang S. LPM: lightweight progressive meshes towards smooth transmis-sion of Web3D media over internet. In: Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. New York: ACM, 2014. 95--103. Google Scholar

[26] Wen L X, Xie N, Jia J Y. Client-driven strategy of large-scale scene streaming. In: Proceedings of the International Conference on Multimedia Modeling. Springer, 2016. 93--103. Google Scholar

[27] McMillan L Jr. An image-based approach to three-dimensional computer graphics. Dissertation for Ph.D. Degree. Chapel Hill: University of North Carolina at Chapel Hill, 1997. Google Scholar

[28] Mori Y, Fukushima N, Yendo T. View generation with 3D warping using depth information for FTV. Signal Processing-Image Communication, 2009, 24: 65-72 CrossRef Google Scholar

[29] Smolic A, Muller K, Dix K, et al. Intermediate viewinterpolation based onmultiviewvideo plus depth for advanced 3Dvideo systems. In: Prodeedings of the 15th IEEE International Conference on Image Processing, 2008. 2448--2451. Google Scholar

[30] Dachsbacher C, Stamminger M. Reflective shadow maps. In: Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games. New York: ACM, 2005. 203--231. Google Scholar

[31] Kaplanyan A, Dachsbacher C. Cascaded light propagation volumes for real-time indirect illumi-nation. In: Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. New York: ACM, 2010. 99--107. Google Scholar

[32] Crassin C, Neyret F, Sainz M. Interactive Indirect Illumination Using Voxel Cone Tracing. Comput Graphics Forum, 2011, 30: 1921-1930 CrossRef Google Scholar

  • Figure 1

    (Color online) OpenGL, OpenGL ES and WebGL versions and their relationship

  • Figure 2

    (Color online) Interactive latency of remote rendering

  • Figure 3

    (Color online) Device performance-oriented collaborative cloud rendering system

  • Figure 4

    (Color online) Overview of lighting and shadow rendering calculation in cloud rendering system

  • Figure 5

    (Color online) Rendering scheduling strategy for low-power hardware devices

  • Figure 6

    (Color online) Rendering scheduling strategy for high-power hardware devices

  • Figure 7

    (Color online) Performance oflighting and shadow rendering algorithm in cloud rendering system

  • Figure 8

    (Color online) Cloud-assisted rendering based on device power

  • Figure 9

    (Color online) Final scene rendering performance

  • Table 1   Test environment of cloud rendering system
    Hardware and Web front-end Web front-end Cloud back-end
    operating system with low-power with high-power
    CPU Intel i3-6100 3.7G Hz Intel i7-4720 2.6 GHz*2 2*Intel Xeon E5 2640 V4 2.4 GHz*10
    GPU Inter HD Graphics 520 Nvidia Geforce GTX860 Nvidia Quadro M6000
    Memory 4 G 8 G 128 G
    Operating system Windows 10.0 Windows 10.0 Windows Server 2012
  • Table 1   The performance of lighting and shadow rendering algorithms in different devices
    makecell[c]Rendering algorithm for lighting and shadow makecell[c]Device configuration makecell[c]Frame rate(FPS) makecell[c]CPU load(%) makecell[c]GPU load(%) makecell[c]Memory load (M)
    Ambient Web front-end with low-power 46 32 71 931
    Blinn phong Web front-end with low-power 24 43 92.4 1358
    Blinn phong Web front-end with high-power 176 44.2 50.3 1019
    Blinn phong Cloud back-end 355 4.1 23.5 881
    Shadow map Web front-end with low-power 18 37 96.8 1021
    Shadow map Web front-end with high-power 65 43 87.6 1434
    Variance shadow map Web front-end with high-power 22 55 97.1 1056
    Variance shadow map Cloud back-end 210 5.9 47.3 925
    Screen space ambient occlusion Web front-end with low-power 23 47.1 96.5 994
    Screen space ambient occlusion Web front-end with high-power 89 39.1 88.2 1033
    Voxel accelerate ambient occlusion Web front-end with high-power 17 69 98.1 2143
    Voxel accelerate ambient occlusion Cloud back-end 217 24.7 61.4 1142
    Voxel cone tracing Cloud back-end 192 11.3 54.1 1071
  • Table 2   3D scenes used for testing
    Sponza Gallery Conference Sibenik Fireroom
    Num of faces 262267 998941 331179 75284 143173
    Sizes in MB 103.0 71.4 19.9 11.5 20.9
  • Table 2   The lighting and shadow algorithm selection based on device configuration
    makecell[c]Device configuration makecell[c]Direct lightingalgorithm makecell[c]Shadow algorithm makecell[c]Ambient occlusion algorithm makecell[c]Indirect lighting algorithm
    Web front-end with
    low-power
    Ambient No No No
    Web front-end with
    high-power
    Blinn phong Shadow map Screen space ambient occlusion No
    Cloud back-end Blinn phong Variance shadow map Voxel accelerate ambient occlusion Voxel cone tracing
  • Table 3   Rendering efficiency comparison of cloud rendering system with low-power front-end
    makecell[c]Device configuration makecell[c]Test item makecell[c]Sponza makecell[c]Gallery makecell[c]Conference makecell[c]Sibenik makecell[c]Fireroom
    Web front-end with low-power Frame rate (FPS) 48 39 66 84 57
    CPU load (%) 34 67 51.2 57.4 43.1
    GPU load (%) 69 85.2 71.3 64.1 81.4
    Memory load (M) 972 1371 1013 1014 1131
    Cloud back-end Frame rate (FPS) 177 117 311 274 213
    CPU load (%) 37 43 39.2 41.1 44
    GPU load (%) 25.4 37.1 24.8 33.8 35.7
    Memory load (M) 892 993 741 912 714