Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

A Comprehensive Survey on Generative AI for Metaverse: Enabling Immersive Experience

  • Review
  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Generative Artificial Intelligence models are Artificial Intelligence models that generate new content based on a prompt or input. The output content can be in various forms, including text, images, and video. Metaverse refers to a virtual world where users can interact with each other, objects and events in an immersive, realistic, and dynamic manner. A critical and foremost step in realizing the Metaverse is content creation for its different realms. Given Metaverse’s need for enormous content, Generative AI is a perfect technology for content creation. This paper explores how Generative AI models can help fulfil the potential of the Metaverse by assisting in the design and production of various aspects of the Metaverse and attracting users not just by creating dynamic, interactive, and personalised content at scale but also by producing various revenue-generating opportunities for users and organisations in the Metaverse. The paper analyses the Generative AI models by grouping them according to the type of content they generate, namely text, image, video, 3D visual, audio, and gaming. Various use cases in the Metaverse are explored and listed according to each type of AI Generated Content (AIGC). This paper also presents several applications and scenarios where the mixture of different Generative AI (GAI) models benefits the Metaverse. Further, this paper also enumerates the limitations and challenges of Generative AI models and the areas of future work. Despite the obstacles, Generative AI can realise the potential of the Metaverse by making it much more functional and interactive owing to the vast use cases of different types of AIGC in the Metaverse, and the age of virtual reality may not be too distant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

Data sharing does not apply to this article as no datasets were generated or analysed during the current study.

References

  1. Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, Sun L. A comprehensive survey of AI-generated content (AIGC): a history of generative AI from GAN to ChatGPT. 2023. arXiv preprint arXiv:2303.04226

  2. Buchanan BG. A (very) brief history of artificial intelligence. Ai Magazine. 2005;26(4):53–53.

    Google Scholar 

  3. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Communications of the ACM. 2020;63(11):139–44.

    MathSciNet  Google Scholar 

  4. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.

  5. OpenAI R. GPT-4 technical report. arXiv; 2023. pp. 2303–08 774.

  6. Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I. Zero-shot text-to-image generation. In: International Conference on Machine Learning. PMLR; 2021. pp. 8821–31.

  7. Pichai S. An important next step on our AI journey. The keyword: Google; 2023.

    Google Scholar 

  8. Hanna DM. The use of artificial intelligence art generator “midjourney’’ in artistic and advertising creativity. J Design Sci Appl Arts. 2023;4(2):42–58.

    Google Scholar 

  9. Rephrase.ai: convert text into engaging AI videos in minutes. https://www.rephrase.ai/. Accessed 02 Dec 2023

  10. Nichol A, Jun H, Dhariwal P, Mishkin P, Chen M. Point-E: a system for generating 3D point clouds from complex prompts. 2022. arXiv preprint arXiv:2212.08751

  11. Podell D, English Z, Lacey K, Blattmann A, Dockhorn T, Müller J, Penna J, Rombach R. SDXL: improving latent diffusion models for high-resolution image synthesis. 2023. arXiv preprint arXiv:2307.01952

  12. Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava P, Bhosale S et al. Llama 2: open foundation and fine-tuned chat models. 2023. arXiv preprint arXiv:2307.09288

  13. Gozalo-Brizuela R, Garrido-Merchán EC. A survey of generative AI applications. 2023. arXiv preprint arXiv:2306.02781

  14. Park S-M, Kim Y-G. A metaverse: taxonomy, components, applications, and open challenges. IEEE access. 2022;10:4209–51.

    Google Scholar 

  15. Azuma RT. A survey of augmented reality. Presence: teleoperators & virtual environments. 1997;6(4):355–85.

  16. Rokhsaritalemi S, Sadeghi-Niaraki A, Choi S-M. A review on mixed reality: current trends, challenges and prospects. Appl Sci. 2020;10(2):636.

    Google Scholar 

  17. Zheng J, Chan K, Gibson I. Virtual reality. Ieee Potentials. 1998;17(2):20–3.

    Google Scholar 

  18. Davis A, Murphy J, Owens D, Khazanchi D, Zigurs I. Avatars, people, and virtual worlds: foundations for research in metaverses. J Assoc Inf Syst. 2009;10(2):1.

    Google Scholar 

  19. Abbate S, Centobelli P, Cerchione R, Oropallo E, Riccio EA. first bibliometric literature review on metaverse. In: IEEE Technology and Engineering Management Conference (TEMSCON EUROPE). IEEE. 2022;2022:254–60.

  20. The Facebook company is now meta | meta. https://about.fb.com/news/2021/10/facebook-company-is-now-meta/. Accessed 02 Dec 2023.

  21. Official site | second life - virtual worlds, virtual reality, VR, avatars, and free 3D chat. https://secondlife.com/. Accessed 02 Dec 2023.

  22. History of second life - second life wiki. https://wiki.secondlife.com/wiki/History_of_Second_Life. Accessed 02 Dec 2023.

  23. Greenwold S. Spatial computing. Master: Massachusetts Institute of Technology; 2003.

    Google Scholar 

  24. Qian L, Luo Z, Du Y, Guo L. Cloud computing: an overview. In: Cloud Computing: First International Conference, CloudCom 2009, Beijing, China, December 1–4, 2009. Proceedings 1. Springer; 2009. pp. 626–31.

  25. Madakam S, Lake V, Lake V, Lake V, et al. Internet of things (IoT): a literature review. J Comput Commun. 2015;3(05):164.

    Google Scholar 

  26. Shi W, Cao J, Zhang Q, Li Y, Xu L. Edge computing: vision and challenges. IEEE Internet Things J. 2016;3(5):637–46.

    Google Scholar 

  27. Evans A, Romeo M, Bahrehmand A, Agenjo J, Blat J. 3D graphics on the web: a survey. Comput Graph. 2014;41:43–61.

    Google Scholar 

  28. Xu M, Ng WC, Lim WYB, Kang J, Xiong Z, Niyato D, Yang Q, Shen XS, Miao C. A full dive into realizing the edge-enabled metaverse: visions, enabling technologies, and challenges. IEEE Commun Surv Tutorials. 2022.

  29. Bale AS, Ghorpade N, Hashim MF, Vaishnav J, Almaspoor Z. A comprehensive study on metaverse and its impacts on humans. Adv Hum Comput Interact. 2022;2022.

  30. Pallavicini F, Pepe A, Minissi ME. Gaming in virtual reality: what changes in terms of usability, emotional response and sense of presence compared to non-immersive video games? Simulation & Gaming. 2019;50(2):136–59.

    Google Scholar 

  31. Bourlakis M, Papagiannidis S, Li F. Retail spatial evolution: paving the way from traditional to metaverse retailing. Electron Commer Res. 2009;9:135–48.

    Google Scholar 

  32. Wang G, Badal A, Jia X, Maltz JS, Mueller K, Myers KJ, Niu C, Vannier M, Yan P, Yu Z, et al. Development of metaverse for intelligent healthcare. Nat Mach Intell. 2022;4(11):922–9.

    Google Scholar 

  33. Tasa UB, Görgülü T. Meta-art: art of the 3-d user-created virtual worlds. Digital creativity. 2010;21(2):100–11.

    Google Scholar 

  34. Asara C. Real estate in the metaverse. 2022.

  35. Moneta A. Architecture, heritage, and the metaverse. Tradit Dwellings Settlements Rev. 2020;32(1):37–49.

    Google Scholar 

  36. Gursoy D, Malodia S, Dhir A. The metaverse in the hospitality and tourism industry: an overview of current trends and future research directions. J Hosp Mark Manag. 2022;31(5):527–34.

    Google Scholar 

  37. Bibri SE, Allam Z. The metaverse as a virtual form of data-driven smart urbanism: on post-pandemic governance through the prism of the logic of surveillance capitalism. Smart Cities. 2022;5(2).

  38. Hwang G-J, Chien S-Y. Definition, roles, and potential research issues of the metaverse in education: an artificial intelligence perspective. Comput Educ Artif Intell. 2022;3:100082.

    Google Scholar 

  39. Popescu GH, Ciurlău CF, Stan CI, Băcănoiu C, Tănase A. Virtual workplaces in the metaverse: immersive remote collaboration tools, behavioral predictive analytics, and extended reality technologies. Psychosociological Issues Hum Resour Manag. 2022;10(1):21–34.

    Google Scholar 

  40. Ning H, Wang H, Lin Y, Wang W, Dhelim S, Farha F, Ding J, Daneshmand M. A survey on the metaverse: the state-of-the-art, technologies, applications, and challenges. IEEE Internet Things J. 2023.

  41. Chamola V, Bansal G, Das TK, Hassija V, Reddy NSS, Wang J, Zeadally S, Hussain A, Yu FR, Guizani M et al. Beyond reality: the pivotal role of generative AI in the metaverse. 2023. arXiv preprint arXiv:2308.06272

  42. Qin HX, Hui P. Empowering the metaverse with generative AI: survey and future directions. In: 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW). IEEE; 2023. pp. 85–90.

  43. Huynh-The T, Pham Q-V, Pham X-Q, Nguyen TT, Han Z, Kim D-S. Artificial intelligence for the metaverse: a survey. Eng Appl Artif Intell. 2023;117:105581.

    Google Scholar 

  44. Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z et al. A survey of large language models. 2023. arXiv preprint arXiv:2303.18223

  45. Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, Lee YT, Li Y, Lundberg S et al. Sparks of artificial general intelligence: early experiments with GPT-4. 2023. arXiv preprint arXiv:2303.12712

  46. Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, de Las Casas D, Hendricks LA, Welbl J, Clark A, et al. An empirical analysis of compute-optimal large language model training. Adv Neural Inf Process Syst. 2022;35(30):016–30.

    Google Scholar 

  47. Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I. Zero-shot text-to-image generation. In: International Conference on Machine Learning. PMLR; 2021. pp. 8821–31.

  48. Midjourney. https://www.midjourney.com/home. Accessed 08 Feb 2024.

  49. Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. pp. 10 684–695.

  50. Craiyon - your free AI image generator tool: create AI art!. https://www.craiyon.com/. Accessed 09 Nov 2023.

  51. Liu M, Shi J, Cao K, Zhu J, Liu S. Analyzing the training processes of deep generative models. IEEE transactions on visualization and computer graphics. 2017;24(1):77–87.

    Google Scholar 

  52. Make-a-video. https://makeavideo.studio/. Accessed 12 Dec 2023.

  53. Imagen video. https://imagen.research.google/video/. Accessed 12 Dec 2023.

  54. Synthesia - #1 ai video generator. https://www.synthesia.io/. Accessed 12 Dec 2023.

  55. AI animation maker. https://www.krikey.ai/. Accessed 09 Nov 2023.

  56. Research. https://openai.com/research/overview. Accessed 09 Nov 2023 .

  57. AI voice generator: versatile text to speech software | murf ai. https://murf.ai/. Accessed 09 Nov 2023.

  58. AI music generator - royalty free music for creators | soundful. https://soundful.com/. Accessed 26 Nov 2023.

  59. Borsos Z, Marinier R, Vincent D, Kharitonov E, Pietquin O, Sharifi M, Roblek D, Teboul O, Grangier D, Tagliasacchi M, et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio: Speech, and Language Processing; 2023.

    Google Scholar 

  60. Wang T, Zhang B, Zhang T, Gu S, Bao J, Baltrusaitis T, Shen J, Chen D, Wen F, Chen Q et al. Rodin: a generative model for sculpting 3D digital avatars using diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. pp. 4563–73.

  61. Poole B, Jain A, Barron JT, Mildenhall B. Dreamfusion: text-to-3D using 2D diffusion. 2022. arXiv preprint arXiv:2209.14988

  62. Li C, Zhang C, Waghwase A, Lee L-H, Rameau F, Yang Y, Bae S-H, Hong CS. Generative AI meets 3D: a survey on text-to-3D in AIGC era. 2023. arXiv preprint arXiv:2305.06131

  63. Build more engaging games with ML agents | unity. https://unity.com/products/machine-learning-agents. Accessed 13 Dec 2023.

  64. Documentation | sidefx. https://www.sidefx.com/docs/. Accessed 09 Nov 2023.

  65. Tsugi studio | software for creatives. http://tsugi-studio.com/web/en/index.html. Accessed 09 Nov 2023.

  66. Nvidia gameworks documentation - nvidia gameworks documentation. https://docs.nvidia.com/gameworks/index.html. Accessed 13 Dec 2023.

  67. Nash C, Ganin Y, Eslami SA, Battaglia P. Polygen: an autoregressive generative model of 3D meshes. In: International conference on machine learning. PMLR; 2020, pp. 7220–7229.

  68. Scenario - AI-generated game assets. https://www.scenario.com/. Accessed 30 May 2024.

  69. AI dungeon. https://aidungeon.com/. Accessed 30 May 2024.

  70. Plut C, Pasquier P. Generative music in video games: state of the art, challenges, and prospects. Entertainment Computing. 2020;33:100337.

    Google Scholar 

  71. Salge C, Green MC, Canaan R, Togelius J. Generative design in minecraft (GDMC) settlement generation competition. In: Proceedings of the 13th International Conference on the Foundations of Digital Games. 2018. pp. 1–10.

  72. Jones D, Snider C, Nassehi A, Yon J, Hicks B. Characterising the digital twin: a systematic literature review. CIRP journal of manufacturing science and technology. 2020;29:36–52.

    Google Scholar 

  73. Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, Casas DdL, Hendricks LA, Welbl J, Clark A et al. Training compute-optimal large language models. 2022. arXiv preprint arXiv:2203.15556

  74. Thoppilan R, De Freitas D, Hall J, Shazeer N, Kulshreshtha A, Cheng H-T, Jin A, Bos T, Baker L, Du Y et al. LAMDA: language models for dialog applications. 2022. arXiv preprint arXiv:2201.08239

  75. Beattie C, Leibo JZ, Teplyashin D, Ward T, Wainwright M, Küttler H, Lefrancq A, Green S, Valdés V, Sadik A et al. Deepmind lab. 2016. arXiv preprint arXiv:1612.03801

  76. Gong J, Foo LG, He Y, Rahmani H, Liu J. LLMS are good sign language translators. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. pp. 18 362–72.

  77. Brown PF, Cocke J, Della Pietra SA, Della Pietra VJ, Jelinek F, Mercer RL, Roossin P. A statistical approach to language translation. In: Coling Budapest 1988 Volume 1: International Conference on Computational Linguistics. 1988.

  78. Razavi AH, Inkpen D, Uritsky S, Matwin S. Offensive language detection using multi-level classification. In: Advances in Artificial Intelligence: 23rd Canadian Conference on Artificial Intelligence, Canadian AI 2010, Ottawa, Canada, May 31–June 2, 2010. Proceedings 23. Springer; 2010. pp. 16–27.

  79. López-Gil J-M, Pereira J. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society. 2024. pp. 1–16.

  80. Metaverse-retail service quality: a future framework for retail service quality in the 3D internet. J Mark Manag. 29(13-14). https://www.tandfonline.com/doi/abs/10.1080/0267257X.2013.835742. Accessed 28 Jan 2024.

  81. Sitaram S, Choudhury M, Patra B, Chaudhary V, Ahuja K, Bali K. Everything you need to know about multilingual LLMS: towards fair, performant and reliable models for languages of the world. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts). 2023. pp. 21–6.

  82. Musical metaverse: vision, opportunities, and challenges | personal and ubiquitous computing. https://link.springer.com/article/10.1007/s00779-023-01708-1. Accessed 28 Jan 2024.

  83. Sun Y, Xu Y, Cheng C, Li Y, Lee CH, Asadipour A. Travel with wander in the metaverse: An ai chatbot to visit the future earth. In: IEEE 24th International Workshop on Multimedia Signal Processing (MMSP). IEEE. 2022;2022:1–6.

  84. Vu MD, Wang H, Li Z, Chen J, Zhao S, Xing Z, Chen C. GPTVoicetasker: LLM-powered virtual assistant for smartphone. 2024. arXiv preprint arXiv:2401.14268

  85. Interactive tools for education in automatic control | ieee journals & magazine | ieee xplore. https://ieeexplore.ieee.org/abstract/document/687617. Accessed 28 Jan 2024.

  86. Krauss C, Bassbouss L, Upravitelev M, An T-S, Altun D, Reray L, Balitzki E, El Tamimi T, Karagülle M. Opportunities and challenges in developing educational AI-assistants for the metaverse,” in International Conference on Human-Computer Interaction. Springer; 2024. pp. 219–238.

  87. International journal on artificial intelligence tools. https://www.worldscientific.com/doi/abs/10.1142/S0218213011000188. Accessed 28 Jan 2024.

  88. Alhawiti KM. Natural language processing and its use in education. Int J Adv Comput Sci Appl. 2014;5(12).

  89. Virtual reality therapy in mental health | annual review of clinical psychology. https://www.annualreviews.org/doi/abs/10.1146/annurev-clinpsy-081219-115923. Accessed 02 Feb 2024.

  90. King DR, Nanda G, Stoddard J, Dempsey A, Hergert S, Shore JH, Torous J. An introduction to generative artificial intelligence in mental health care: considerations and guidance. Current psychiatry reports. 2023;25(12):839–46.

    Google Scholar 

  91. Kholmogorova A, Tarhanova P, Shalygina O. Standards of physical beauty and mental health in children and young people in the era of the information revolution. Int J Cult Ment Health. 2018;11(1):87–98.

    Google Scholar 

  92. Therapy in virtual environments-clinical and ethical issues | telemedicine and e-health. https://www.liebertpub.com/doi/abs/10.1089/tmj.2011.0195. Accessed 02 Feb 2024.

  93. Soviero B, Kuhn D, Salle A, Moreira VP. ChatGPT goes shopping: LLMS can predict relevance in ecommerce search. In: European Conference on Information Retrieval. Springer; 2024. pp. 3–11.

  94. Hudson J. Virtual immersive shopping experiences in metaverse environments: predictive customer analytics, data visualization algorithms, and smart retailing technologies. Linguistic and Philosophical Investigations. 2022;21:236–51.

    Google Scholar 

  95. Liu Y, Shi D, Skaar SB, Tan J. Development and experiment of CSM-based industrial robot servoing control system. In: 2013 IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems. IEEE; 2013. pp. 108–113.

  96. Mirage. https://mirageml.com/. Accessed 09 Nov 2023.

  97. Wu J, Zhang C, Xue T, Freeman B, Tenenbaum J. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Adv Neural Inf Process Syst. 2016;29.

  98. Research. https://openai.com/research/overview. Accessed 09 Nov 2023.

  99. Lensa - prisma labs. https://prisma-ai.com/lensa. Accessed 10 Nov 2023.

  100. Research room - AI generated artwork - nightcafe creator. https://creator.nightcafe.studio/creation/0hVaw7Kw6AD3qhL460Av. Accessed 10 Nov 2023.

  101. AI art generator | create AI images and photos online free | openart. https://openart.ai/. Accessed 10 Nov 2023.

  102. Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR. Le QV. Xlnet: Generalized autoregressive pretraining for language understanding. Adv Neural Inf Process Syst; 2019. p. 32.

    Google Scholar 

  103. Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al. Palm: scaling language modeling with pathways. 2022. arXiv preprint arXiv:2204.02311

  104. Extensions. https://bard.google.com/extensions?utm_source=sem&utm_medium=paid-media &utm_campaign=q4enIN_sem1. Accessed 10 Nov 2023.

  105. What’s new - fliki. https://fliki.ai/resources/whats-new. Accessed 10 Nov 2023.

  106. About runway research. https://research.runwayml.com/about. Accessed 10 Nov 2023.

  107. Support hub. https://www.tavus.io/support. Accessed 10 Nov 2023.

  108. [2311.04205] Rephrase and respond: let large language models ask better questions for themselves. https://arxiv.org/abs/2311.04205. Accessed 10 Nov 2023.

  109. AI script generator | wondershare virbo. https://virbo.wondershare.com/ai-script.html. Accessed 19 Dec 2023.

  110. Donahue C, McAuley J, Puckette M. Adversarial audio synthesis. 2018. arXiv preprint arXiv:1802.04208

  111. Melodrive - itch.io. https://melodrive.itch.io/. Accessed 14 Dec 2023.

  112. Civit M, Civit-Masot J, Cuadrado F, Escalona MJ. A systematic review of artificial intelligence-based music generation: scope, applications, and future trends. Expert Syst Appl. 2022;118190.

  113. Cheong I, Xia K, Feng KK, Chen QZ, Zhang AX. (a) I am not a lawyer, but...: engaging legal experts towards responsible LLM policies for legal advice. In: The 2024 ACM Conference on Fairness, Accountability, and Transparency. 2024. pp. 2454–2469.

  114. Kostenko O, Furashev V, Zhuravlov D, Dniprov O. Genesis of legal regulation web and the model of the electronic jurisdiction of the metaverse. Bratislava Law Review. 2022;6(2):21–36.

    Google Scholar 

  115. Kalyvaki M. Navigating the metaverse business and legal challenges: intellectual property, privacy, and jurisdiction. Journal of Metaverse. 2023;3(1):87–92.

    Google Scholar 

  116. Dubourg E, Thouzeau V, Baumard N. A step-by-step method for cultural annotation by LLMS. Frontiers in Artificial Intelligence. 2024;7:1365508.

    Google Scholar 

  117. Gaafar AA. Metaverse in architectural heritage documentation & education. Advances in Ecological and Environmental Research. 2021;6(10):66–86.

    Google Scholar 

  118. Huggett J. Virtually real or really virtual: towards a heritage metaverse. Studies in digital heritage. 2020;4(1):1–15.

    Google Scholar 

  119. Ren Z, Zhan Y, Yu B, Ding L, Tao D. Healthcare copilot: eliciting the power of general LLMS for medical consultation. 2024. arXiv preprint. arXiv:2402.13408

  120. Bulla C, Parushetti C, Teli A, Aski S, Koppad S. A review of AI based medical assistant chatbot. Research and Applications of Web Development and Design. 2020;3(2):1–14.

    Google Scholar 

  121. Yotam S, Penarska Gabriela A, Randsalu Isa A, Christian AC, Takeo I. Mystoryknight: a character-drawing driven storytelling system using LLM hallucinations. 2024.

  122. Cavazza M, Charles F, Mead SJ. Characters in search of an author: AI-based virtual storytelling. In: International Conference on Virtual Storytelling. Springer; 2001. pp. 145–154.

  123. Sun Y, Wang H, Chan PM, Tabibi M, Zhang Y, Lu H, Chen Y, Lee CH, Asadipour A. Fictional worlds, real connections: developing community storytelling social chatbots through LLMS. 2023. arXiv preprint arXiv:2309.11478

  124. Shaker N, Yannakakis G, Togelius J. Towards automatic personalized content generation for platform games. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. 2010. vol. 6, no. 1, pp. 63–8.

  125. Cox SR, Ooi WT. Check for conversational interactions with NPCS in LLM-driven gaming: guidelines from a content analysis of player feedback. In: Chatbot Research and Design: 7th International Workshop, CONVERSATIONS 2023, Oslo, Norway, November 22-23, 2023, Revised Selected Papers, vol. 14524. Springer Nature; 2024. p. 167.

  126. Huang J-t, Li EJ, Lam MH, Liang T, Wang W, Yuan Y, Jiao W, Wang X, Tu Z, Lyu MR. How far are we on the decision-making of LLMS? Evaluating LLMS’ gaming ability in multi-agent environments. 2024. arXiv preprint arXiv:2403.11807

  127. Colton S, Goodwin J, Veale T. Full-face poetry generation. In: ICCC. 2012. pp. 95–102.

  128. Doh S, Choi K, Lee J, Nam J. Lp-musiccaps: LLM-based pseudo music captioning. 2023. arXiv preprint arXiv:2307.16372

  129. Ding Z, Smith-Renner A, Zhang W, Tetreault JR, Jaimes A. Harnessing the power of LLMS: evaluating human-AI text co-creation through the lens of news headline generation. 2023. arXiv preprint arXiv:2310.10706

  130. Tsourma M, Zamichos A, Efthymiadis E, Drosou A, Tzovaras D. An AI-enabled framework for real-time generation of news articles based on big EO data for disaster reporting. Future Internet. 2021;13(6):161.

    Google Scholar 

  131. Dang W, Cai L, Liu M, Li X, Yin Z, Liu X, Yin L, Zheng W. Increasing text filtering accuracy with improved LSTM. Computing and Informatics. 2023;42(6):1491–517.

    Google Scholar 

  132. Feizi S, Hajiaghayi M, Rezaei K, Shin S. Online advertisements with LLMS: opportunities and challenges. 2023. arXiv preprint arXiv:2311.07601

  133. Meguellati E, Han L, Bernstein A, Sadiq S, Demartini G. How good are LLMS in generating personalized advertisements? Companion Proceedings of the ACM on Web Conference. 2024;2024:826–9.

    Google Scholar 

  134. Kim J. Advertising in the metaverse: research agenda. Journal of Interactive Advertising. 2021;21(3):141–4.

    Google Scholar 

  135. Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. pp. 10 684–95.

  136. Li Y, Liu H, Wu Q, Mu F, Yang J, Gao J, Li C, Lee YJ. Gligen: open-set grounded text-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. pp. 22 511–21.

  137. Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M. Hierarchical text-conditional image generation with clip latents. 2022;1(2):3. arXiv preprint arXiv:2204.06125

  138. Qin HX, Hui P. Empowering the metaverse with generative AI: survey and future directions. In: 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW). 2023. pp. 85–90.

  139. Best AI video generator | deepbrain ai. https://www.deepbrain.io/. Accessed 13 Dec 2023.

  140. Park M, Cho Y, Na G, Kim J. Application of virtual avatar using motion capture in immersive virtual environment. Int J Human–Comp Interact. 2023;1–15.

  141. Simulation of the field of view in AR and VR headsets | springerlink. https://link.springer.com/chapter/10.1007/978-3-030-77599-5_21. Accessed 19 July 2024.

  142. Liu Y, Siau KL. Generative artificial intelligence and metaverse: future of work, future of society, and future of humanity. In: International Conference on AI-generated Content. Springer. 2023. pp. 118–27.

  143. Dreamstudio. https://beta.dreamstudio.ai/generate. Accessed 10 Nov 2023.

  144. Artbreeder. https://www.artbreeder.com/. Accessed 10 Nov 2023.

  145. Designify - turn any photo into awesome. https://www.designify.com/. Accessed 10 Nov 2023.

  146. Roberts A, Engel J, Mann Y, Gillick J, Kayacik C, Nørly S, Dinculescu M, Radebaugh C, Hawthorne C, Eck D. Magenta studio: augmenting creativity with deep learning in ableton live. 2019.

  147. Guo C, Dou Y, Bai T, Dai X, Wang C, Wen Y. Artverse: a paradigm for parallel human-machine collaborative painting creation in metaverses. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2023;53(4):2200–8.

    Google Scholar 

  148. Karras T, Laine S, Aila T. A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. pp. 4401–10.

  149. AI image generator | deep dream generator. https://deepdreamgenerator.com/. Accessed 19 Dec 2023

  150. Henry J, Natalie T, Madsen D. Pix2pix GAN for image-to-image translation. Research Gate Publication. 2021;1–5.

  151. Alasadi EA, Baiz CR. Generative AI in education and research: opportunities, concerns, and solutions. J Chem Educ. 2023;100(8):2965–71.

    Google Scholar 

  152. Rahman KR, Shitol SK, Islam MS, Iftekhar KT, Pranto S. Use of metaverse technology in education domain. J Metaverse. 2023;3(1):79–86.

  153. Zhu J, Dang P, Zhang J, Cao Y, Wu J, Li W, Hu Y, You J. The impact of spatial scale on layout learning and individual evacuation behavior in indoor fires: single-scale learning perspectives. Int J Geogr Inf Sci. 2024;38(1):77–99.

    Google Scholar 

  154. Nasyrov RR, Excell PS. Creation of interactive virtual reality scenarios as a training and education tool. Technol Des Arts-Oppor Challenges. 2020;353–69.

  155. Barráez-Herrera DP. Metaverse in a virtual education context. Metaverse. 2022;3(1):10.

    Google Scholar 

  156. Yang S. Storytelling and user experience in the cultural metaverse. Heliyon. 2023;9(4).

  157. Jang S-Y, Kim S-A. Automatic generation of virtual architecture using user activities in metaverse. Int J Human-Comput Studies. 2024;182: 103163.

    Google Scholar 

  158. Architechtures - AI-powered building design. https://architechtures.com/en. Accessed 25 Feb 2024.

  159. Zhang J, Liu X, Ye X, Zhao F, Zhang Y, Wu M, Zhang Y, Xu L, Yu J. Editable free-viewpoint video using a layered neural representation. ACM Trans Graph (TOG). 2021;40(4):1–18.

    Google Scholar 

  160. Ferraro C, Demsar V, Sands S, Restrepo M, Campbell C. The paradoxes of generative AI-enabled customer service: a guide for managers. Business Horizons, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0007681324000582

  161. de Ocáriz Borde HS. Presentations and fast content creation for video conferencing platforms and the metaverse using AI. 2022.

  162. Jiang Y, Yang S, Koh TL, Wu W, Loy CC, Liu Z. Text2performer: text-driven human video generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023. pp. 22 747–57.

  163. Baía Reis A, Ashmore M. From video streaming to virtual reality worlds: an academic, reflective, and creative study on live theatre and performance in the metaverse. Int J Perform Arts Digit Media. 2022;18(1):7–28.

    Google Scholar 

  164. Khan AA, Shao J. SPNET: a deep network for broadcast sports video highlight generation. Comput Electr Eng. 2022;99:107779. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0045790622000817

  165. Oja M, Karhu H. Enhanced sport event. 2023.

  166. Tavus | the most advanced AI video personalization platform. https://www.tavus.io/. Accessed 14 Dec 2023.

  167. Fliki - video creation made 10x simpler & faster with ai. https://fliki.ai/. Accessed 14 Dec 2023.

  168. AI video generator: from text to video in minutes - hour one. https://hourone.ai/. Accessed 14 Dec 2023.

  169. A deep learning approach for quality enhancement of surveillance video: Journal of intelligent transportation systems. 24(3). https://www.tandfonline.com/doi/abs/10.1080/15472450.2019.1670659. Accessed 19 July 2024.

  170. Burri-Nenova M. User created content in virtual worlds and cultural diversity. In: Governance of Digital Game Environments and Cultural Diversity. Edward Elgar Publishing. 2010.

  171. Li H, Cui C, Jiang S. Strategy for improving the football teaching quality by AI and metaverse-empowered in mobile internet environment. Wireless Networks. 2022. pp. 1–10.

  172. AI animation maker. https://www.krikey.ai/. Accessed 26 Nov 2023.

  173. Lam KY, Yang L, Alhilal A, Lee L-H, Tyson G, Hui P. Human-avatar interaction in metaverse: framework for full-body interaction. In: Proceedings of the 4th ACM International Conference on Multimedia in Asia. 2022. pp. 1–7.

  174. Chen S-C. Multimedia research toward the metaverse. IEEE MultiMedia. 2022;29(1):125–7.

    Google Scholar 

  175. Lee H, Woo D, Yu S. Virtual reality metaverse system supplementing remote education methods: based on aircraft maintenance simulation. Applied Sciences. 2022;12(5):2667.

    Google Scholar 

  176. Jabri A, Fleet D, Chen T. Scalable adaptive computation for iterative generation. 2022. arXiv preprint arXiv:2212.11972

  177. Shapenet. https://shapenet.org/. Accessed 14 Dec 2023.

  178. Lumirithmic | 3D appearance capture at scale. https://www.lumirithmic.com/. Accessed 14 Dec 2023.

  179. Fan H, Su H, Guibas LJ. A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. pp. 605–613.

  180. So C, Baciu G, Sun H. Reconstruction of 3D virtual buildings from 2D architectural floor plans. In: Proceedings of the ACM symposium on Virtual reality software and technology. 1998. pp. 17–23.

  181. Zhou P, Peng R, Xu M, Wu V, Navarro-Alarcon D. Path planning with automatic seam extraction over point cloud models for robotic arc welding. IEEE robotics and automation letters. 2021;6(3):5002–9.

    Google Scholar 

  182. Ducheneaut N, Wen M-H, Yee N, Wadley G. Body and mind: a study of avatar personalization in three virtual worlds. In: Proceedings of the SIGCHI conference on human factors in computing systems. 2009. pp. 1151–1160.

  183. Chao F, Kantorová V, Gonnella G, Bassarsky L, Zeifman L, Gerland P. Estimating age-specific fertility rate in the world population prospects: a Bayesian modelling approach. 2023.

  184. Lin H, Wang H. Avatar creation in virtual worlds: behaviors and motivations. Computers in Human Behavior. 2014;34:213–8.

    Google Scholar 

  185. Kim DY, Lee HK, Chung K. Avatar-mediated experience in the metaverse: the impact of avatar realism on user-avatar relationship. Journal of Retailing and Consumer Services. 2023;73:103382.

    Google Scholar 

  186. Ho J, Chan W, Saharia C, Whang J, Gao R, Gritsenko A, Kingma DP, Poole B, Norouzi M, Fleet DJ, et al. Imagen video: high definition video generation with diffusion models. 2022. arXiv preprint arXiv:2210.02303

  187. Kim J. Modeling and optimization of a tree based on virtual reality for immersive virtual landscape generation. Symmetry. 2016;8(9):93.

    MathSciNet  Google Scholar 

  188. Fan X, Jiang X, Deng N. Immersive technology: a meta-analysis of augmented/virtual reality applications and their impact on tourism experience. Tourism Management. 2022;91:104534.

    Google Scholar 

  189. Wei W, Baker MA, Onder I. All without leaving home: building a conceptual model of virtual tourism experiences. International Journal of Contemporary Hospitality Management. 2023;35(4):1284–303.

    Google Scholar 

  190. Wu J, Zhu J, Zhang J, Dang P, Li W, Guo Y, Fu L, Lai J, You J, Xie Y, et al. A dynamic holographic modelling method of digital twin scenes for bridge construction. International Journal of Digital Earth. 2023;16(1):2404–25.

    Google Scholar 

  191. Rauch U. Who owns this space anyway? The arts 3D VL metaverse as a network of imagination. In: EdMedia+ Innovate Learning. Association for the Advancement of Computing in Education (AACE). 2007. pp. 4249–53.

  192. Portman ME, Natapov A, Fisher-Gewirtzman D. To go where no man has gone before: virtual reality in architecture, landscape architecture and environmental planning. Computers, Environment and Urban Systems. 2015;54:376–84.

    Google Scholar 

  193. Blender - home of the blender project - free and open 3D creation software. https://www.blender.org/. Accessed 08 Feb 2024.

  194. Zhao Y, He R, Kersting N, Liu C, Agrawal S, Chetia C, Gu Y. ONNXExplainer: an ONNX based generic framework to explain neural networks using Shapley values. 2023. arXiv preprint arXiv:2309.16916

  195. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst. 2019;32.

  196. Waisberg E, Ong J, Masalkhi M, Kamran SA, Zaman N, Sarker P, Lee AG, Tavakkoli A. GPT-4: a new era of artificial intelligence in medicine. Irish Journal of Medical Science (1971-). 2023;1–4.

  197. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems. 2016. arXiv preprint arXiv:1603.04467

  198. Unreal engine: real-time 3D creation tool. https://www.unrealengine.com/en-US. Accessed 08 Feb 2024.

  199. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Scao TL, Gugger S, Drame M, Lhoest Q, Rush AM. Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Online: Association for Computational Linguistics. 2020. pp. 38–45. [Online]. Available: https://www.aclweb.org/anthology/2020.emnlp-demos.6

  200. ML agents | unity. https://unity.com/products/machine-learning-agents. Accessed 08 Feb 2024.

  201. Ayiter E. Embodied in a metaverse: anatomia and body parts. Technoetic Arts. 2010;8(2):181–8.

    Google Scholar 

  202. Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X. A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert review of medical devices. 2021;18(1):47–62.

    Google Scholar 

  203. Matwala K, Shakir T, Bhan C, Chand M. The surgical metaverse. Cirugía Española. 2024;102:S61–S65. cirugía Digital. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0009739X23002312

  204. Lee J, Eom S-Y, Lee J. Empowering game designers with generative AI. IADIS International Journal on Computer Science & Information Systems. 2023;18(2):213–30.

    Google Scholar 

  205. Ratican J, Hutson J, Wright A. A proposed meta-reality immersive development pipeline: generative AI models and extended reality (XR) content for the metaverse. J Intell Learn Syst Appl. 2023;15.

  206. Change the rules of the game: Nvidia omniverse brings an arsenal of RTX and AI-powered apps, extensions and DIY toolkits to accelerate game development pipelines | nvidia technical blog. https://developer.nvidia.com/blog/?p=34587. Accessed 28 Jan 2024.

  207. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA. Generative adversarial networks: an overview. IEEE Signal Proc Mag. 2018;35(1):53–65.

    Google Scholar 

  208. Brocchini M, Mameli M, Balloni E, Sciucca LD, Rossi L, Paolanti M, Frontoni E, Zingaretti P. Monster: a deep learning-based system for the automatic generation of gaming assets. In: International Conference on Image Analysis and Processing. Springer. 2022. pp. 280–90.

  209. Watkins R. Procedural content generation for unity game development. Packt Publishing Ltd. 2016.

  210. Croitoru F-A, Hondru V, Ionescu RT, Shah M. Diffusion models in vision: a survey. IEEE Trans Pattern Anal Mach Intell. 2023;45(9):10 850–69.

  211. Li H, Xia C, Wang T, Wang Z, Cui P, Li X. Grass: learning spatial–temporal properties from chainlike cascade data for microscopic diffusion prediction. IEEE Trans Neural Netw Learn Syst. 2023.

  212. Xu Y, Wang E, Yang Y, Chang Y. A unified collaborative representation learning for neural-network based recommender systems. IEEE Trans Knowl Data Eng. 2021;34(11):5126–39.

    Google Scholar 

  213. Ban Y, Liu Y, Yin Z, Liu X, Liu M, Yin L, Li X, Zheng W. Micro-directional propagation method based on user clustering. Comput Inform. 2023;42(6):1445–70.

    Google Scholar 

  214. Kings M, Täcklind S. Leveling up the playing field: exploring the strengths and weaknesses of AI-generated content in game development. 2023.

  215. Larsson T, Font J, Alvarez A. Towards AI as a creative colleague in game level design. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 18, no. 1, 2022, pp. 137–45.

  216. Shen Z. Effects of AI-generated content (AIGC) in the game development: from traditional PCG to AIGC. 2023.

  217. Huang F, Wang Z, Huang X, Qian Y, Li Z, Chen H. Aligning distillation for cold-start item recommendation. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023. pp. 1147–57.

  218. Bakkes S, Tan CT, Pisan Y. Personalised gaming: a motivation and overview of literature. In: Proceedings of the 8th Australasian Conference on Interactive Entertainment: Playing the System. 2012. pp. 1–10.

  219. Zhu C. Research on emotion recognition-based smart assistant system: emotional intelligence and personalized services. J Syst Manag Sci. 2023;13(5):227–42.

    Google Scholar 

  220. Göbel S, Wendel V. Personalization and adaptation. Serious games: foundations, concepts and practice. 2016. pp. 161–210.

  221. Liu Q, Chang C, Shen H, Cheng S, Li X, Zheng R. Research on artificial intelligence generated audio. In: Sixth International Conference on Computer Information Science and Application Technology (CISAT 2023), vol. 12800. SPIE, 2023, pp. 1206–12.

  222. Plans D, Morelli D. Experience-driven procedural music generation for games. IEEE Trans Comput Intell AI Games. 2012;4(3):192–8.

    Google Scholar 

  223. Karpov N. Artificial intelligence for music composing: future scenario analysis. 2020.

  224. Martinson F, Rangel D. A comprehensive analysis of game hacking through injectors: exploits, defenses and beyond. Int J Comput Appl. 2023;975:8887.

  225. Sai S, Sai R, Chamola V. Generative AI for industry 5.0: analyzing the impact of chatGPT, DALLE, and other models. IEEE Open J Commun Soc. 2024;1–1.

  226. Sai S, Yashvardhan U, Chamola V, Sikdar B. Generative AI for cyber security: analyzing the potential of chatGPT, DALL-E, and other models for enhancing the security space. IEEE Access. 2024;12(53):497–516.

    Google Scholar 

  227. Maario A, Shukla VK, Ambikapathy A, Sharma P. Redefining the risks of kernel-level anti-cheat in online gaming. In: 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN). IEEE. 2021. pp. 676–80.

  228. Sai S, Gaur A, Sai R, Chamola V, Guizani M, Rodrigues JJPC. Generative AI for transformative healthcare: a comprehensive study of emerging models, applications, case studies, and limitations. IEEE Access. 2024;12(31):078–106.

    Google Scholar 

  229. Sai S, Kanadia M, Chamola V. Empowering IoT with generative AI: applications, case studies, and limitations. IEEE Internet of Things Magazine. 2024;7(3):38–43.

    Google Scholar 

  230. Chamola V, Sai S, Sai R, Hussain A, Sikdar B. Generative AI for consumer electronics: enhancing user experience with cognitive and semantic computing. IEEE Consum Electron Mag. 2024;1–9.

  231. Liu H-I, Tang B-R. DACA: dynamic anti-cheating architecture for MMOGS. In: 2009 International Conference on Advanced Information Networking and Applications. IEEE. 2009. pp. 892–7.

  232. Tychsen A, Hitchens M, Brolund T, Kavakli M. The game master. ACM International Conference Proceeding Series. 2005;2005(123):215–22.

    Google Scholar 

  233. Mohammad Abedrabbu Alkhawaldeh MASK. Insights on the use of AI-powered game-based learning in translation education. J Southwest Jiaotong University. 2023;58(5).

  234. Choi S, Kim N, Kim J, Kang H. How does AI improve human decision-making? Evidence from the AI-powered go program. Evidence from the AI-Powered Go Program (April 2022). USC Marshall School of Business Research Paper Sponsored by iORB, No. Forthcoming. 2022.

  235. Huang Q, Park DS, Wang T, Denk TI, Ly A, Chen N, Zhang Z, Zhang Z, Yu J, Frank C, et al. Noise2music: text-conditioned music generation with diffusion models. 2023. arXiv preprint arXiv:2302.03917

  236. Sai S, Garg A, Chamola V. Navigating the metaverse: a comprehensive analysis of consumer electronics prospects and challenges. ACM Trans. Internet Technol. 2024. just Accepted. [Online]. Available: https://doi.org/10.1145/3680545

  237. Sai S, Prasad M, Garg A, Chamola V. Synergizing digital twins and metaverse for consumer health: a case study approach. IEEE Transactions on Consumer Electronics. 2024;70(1):2137–44.

    Google Scholar 

  238. AIVA, the AI music generation assistant. https://www.aiva.ai/. Accessed 19 Dec 2023.

  239. Royalty free music for video creators | epidemic sound. https://www.epidemicsound.com/. Accessed 19 Dec 2023.

  240. Melodrive - itch.io. https://melodrive.itch.io/. Accessed 19 Dec 2023.

  241. Amper music. https://ampermusic.zendesk.com/hc/en-us. Accessed 05 Feb 2024.

  242. Roberts A, Engel J, Raffel C, Simon I, Hawthorne C. Musicvae: creating a palette for musical scores with machine learning. Magenta. 2018.

  243. Abdrabuh EAA. AI-synthesized speech: generation and detection. State University of New York at Albany; 2022.

  244. Leongómez JD, Pisanski K, Reby D, Sauter D, Lavan N, Perlman M, Varella Valentova J. Voice modulation: from origin and mechanism to social impact. 2021. p. 20200386.

  245. Sai S, Prasad M, Upadhyay A, Chamola V, Herencsar N. Confluence of digital twins and metaverse for consumer electronics: real world case studies. IEEE Transactions on Consumer Electronics. 2024;70(1):3194–203.

    Google Scholar 

  246. Sai S, Goyal D, Chamola V, Sikdar B. Consumer electronics technologies for enabling an immersive metaverse experience. IEEE Consumer Electronics Magazine. 2024;13(3):16–24.

    Google Scholar 

  247. Liang C, Du H, Sun Y, Niyato D, Kang J, Zhao D, Imran MA. Generative AI-driven semantic communication networks: architecture, technologies and applications. 2023. arXiv preprint arXiv:2401.00124

  248. Li C. System design and platform implementation for AI-based metaverse music. In: Proceedings of the 2nd International Conference on Bigdata Blockchain and Economy Management, ICBBEM 2023, May 19–21, 2023. Hangzhou, China; 2023.

  249. Li B, Zhao Y, Zhelun S, Sheng L. Danceformer: music conditioned 3D dance generation with parametric motion transformer. Proceedings of the AAAI Conference on Artificial Intelligence. 2022;36(2):1272–9.

    Google Scholar 

  250. Höllein L, Cao A, Owens A, Johnson J, Nießner M. Text2room: extracting textured 3D meshes from 2D text-to-image models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. pp. 7909–20.

  251. Paliwal R. Generative artificial intelligence in metaverse era. Int Res J Modernization Eng Technol Sci. 2023;5(09).

  252. Kanematsu H, Fukumura Y, Barry DM, Sohn SY, Taguchi R. Multilingual discussion in metaverse among students from the usa, korea and japan. In,. Knowledge-Based and Intelligent Information and Engineering Systems: 14th International Conference, KES. Cardiff, UK, September 8–10, 2010, Proceedings, Part IV 14. Springer. 2010;2010:200–9.

  253. Jot J-M, Audfray R, Hertensteiner M, Schmidt B. Immersive and 3D audio: from architecture to automotive (I3DA). IEEE. 2021;2021:1–15.

    Google Scholar 

  254. Clancy M. Artificial intelligence and music ecosystem. CRC Press. 2022.

  255. Onderdijk KE, Bouckaert L, Van Dyck E, Maes P-J. Concert experiences in virtual reality environments. Virtual Reality. 2023;27(3):2383–96.

    Google Scholar 

  256. Tabak C. Intelligent music applications: innovative solutions for musicians and listeners. Uluslararası Anadolu Sosyal Bilimler Dergisi. 2024;7(3):752–73.

    Google Scholar 

  257. Chen C, Zhang KZ, Chu Z, Lee M. Augmented reality in the metaverse market: the role of multimodal sensory interaction. Internet Res. 2024;34(1):9–38.

    Google Scholar 

  258. Raghuvanshi N, Lin MC. Physically based sound synthesis for large-scale virtual environments. IEEE Comput Graph Appl. 2007;27(1):14–8.

    Google Scholar 

  259. Park B, Namkung K, Pan Y. Could you evaluate sounds in a virtual environment? Evaluation components of auditory experience in a metaverse environment. Appl Sci. 2023;13(19):10991.

    Google Scholar 

  260. Kulkarni C, Druga S, Chang M, Fiannaca A, Cai C, Terry M. A word is worth a thousand pictures: prompts as AI design material. 2023. arXiv preprint arXiv:2303.12647

  261. Zhang C, Zhang C, Zhang M, Kweon IS. Text-to-image diffusion models in generative AI: a survey. 2023. arXiv preprint arXiv:2303.07909

  262. Urban Davis J, Anderson F, Stroetzel M, Grossman T, Fitzmaurice G. Designing co-creative AI for virtual environments. In: Proceedings of the 13th Conference on Creativity and Cognition. 2021. pp. 1–11.

  263. Jeong Y, Lee Y, Byun G, Moon J. Navigating the creation of immersive learning environments in roblox: integrating generative AI for enhanced simulation-based learning. Immersive Learn Res Pract. 2024;16–19.

  264. (377) Youtube. https://www.youtube.com/. Accessed 19 Jul 2024.

  265. Díaz J, Saldaña C, Avila C. Virtual world as a resource for hybrid education. International Journal of Emerging Technologies in Learning (iJET). 2020;15(15):94–109.

    Google Scholar 

  266. Regenwetter L, Nobari AH, Ahmed F. Deep generative models in engineering design: a review. Journal of Mechanical Design. 2022;144(7):071704.

    Google Scholar 

  267. Introducing chatGPT | openai. https://openai.com/index/chatgpt/. Accessed 27 Jun 2024.

  268. Divya V, Mirza AU. Transforming content creation: the influence of generative AI on a new frontier. EXPLORING THE FRONTIERS OF ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING TECHNOLOGIES. 2024. p. 143.

  269. Niu X, Feng W. Immersive entertainment environments-from theme parks to metaverse. In: International Conference on Human-Computer Interaction. Springer; 2022. pp. 392–403.

  270. Mitra DS. Generative AI and metaverse: companionship and assisted living for elderly people. Available at SSRN 4843464. 2023.

  271. Shen B, Tan W, Guo J, Zhao L, Qin P. How to promote user purchase in metaverse? A systematic literature review on consumer behavior research and virtual commerce application design. Applied Sciences. 2021;11(23):11087.

    Google Scholar 

  272. Kliestik T, Novak A, Lăzăroiu G. Live shopping in the metaverse: visual and spatial analytics, cognitive artificial intelligence techniques and algorithms, and immersive digital simulations. Linguistic and Philosophical Investigations. 2022;21:187–202.

    Google Scholar 

  273. McEwan M. Taking motion controls to the next level: interactions in the metaverse. 2023.

  274. Wang Y, Wang L, Siau KL. Human-centered interaction in virtual worlds: a new era of generative artificial intelligence and metaverse. Int J Human–Comput Interact. 2024. pp. 1–43.

  275. Wang X, Wan Z, Hekmati A, Zong M, Alam S, Zhang M, Krishnamachari B. IoT in the era of generative AI: vision and challenges. 2024. arXiv preprint arXiv:2401.01923

  276. Di Pietro R, Cresci S. Metaverse: security and privacy issues. In: 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). IEEE. 2021. pp. 281–88.

  277. Jaber TA. Security risks of the metaverse world. Int J Interact Mobile Technol. 2022;16(13).

  278. Zhang M, Wei E, Berry R, Huang J. Age-dependent differential privacy. IEEE Trans Inf Theory. 2023.

  279. Xu H, Li Z, Li Z, Zhang X, Sun Y, Zhang L. Metaverse native communication: a blockchain and spectrum prospective. In: 2022 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE. 2022. pp. 7–12.

  280. Barreda-Ángeles M, Hartmann T. Hooked on the metaverse? Exploring the prevalence of addiction to virtual reality applications. Frontiers in Virtual Reality. 2022;3:1031697.

    Google Scholar 

  281. Illia L, Colleoni E, Zyglidopoulos S. Ethical implications of text generation in the age of artificial intelligence. Business Ethics, the Environment & Responsibility. 2023;32(1):201–10.

    Google Scholar 

  282. Wang P, Song W, Qi H, Zhou C, Li F, Wang Y, Sun P, Zhang Q. Server-initiated federated unlearning to eliminate impacts of low-quality data. IEEE Trans Serv Comput. 2024.

  283. Li M, Wan Z, Zou T, Shen Z, Li M, Wang C, Xiao X. Artificial intelligence enabled self-powered wireless sensing for smart industry. Chem Eng J. 2024;492: 152417.

    Google Scholar 

  284. Sun G, Xu Z, Yu H, Chang V. Dynamic network function provisioning to enable network in box for industrial applications. IEEE Trans Ind Inf. 2020;17(10):7155–64.

    Google Scholar 

  285. Sun G, Xu Z, Yu H, Chen X, Chang V, Vasilakos AV. Low-latency and resource-efficient service function chaining orchestration in network function virtualization. IEEE Internet Things J. 2019;7(7):5760–72.

    Google Scholar 

  286. Sun G, Zhu G, Liao D, Yu H, Du X, Guizani M. Cost-efficient service function chain orchestration for low-latency applications in NFV networks. IEEE Syst J. 2018;13(4):3877–88.

    Google Scholar 

  287. Ghimire P, Kim K, Acharya M. Generative AI in the construction industry: opportunities & challenges. 2023. arXiv preprint arXiv:2310.04427

  288. M. Zhou, V. Abhishek, T. Derdenger, J. Kim, and K. Srinivasan, “Bias in generative AI. 2024. arXiv preprint arXiv:2403.02726

  289. Wang P, Wei Z, Qi H, Wan S, Xiao Y, Sun G, Zhang Q. Mitigating poor data quality impact with federated unlearning for human-centric metaverse. IEEE J Selected Areas Commun. 2023.

  290. De Angelis L, Baglivo F, Arzilli G, Privitera GP, Ferragina P, Tozzi AE, Rizzo C. Chatgpt and the rise of large language models: the new AI-driven infodemic threat in public health. Frontiers in Public Health. 2023;11:1166120.

    Google Scholar 

  291. Kim JH, Kim J, Park J, Kim C, Jhang J, King B. When chatGPT gives incorrect answers: the impact of inaccurate information by generative AI on tourism decision-making. J Travel Res. 2023;00472875231212996.

  292. Menell PS, Scotchmer S. Intellectual property law. Handbook of law and economics. 2007;2:1473–570.

    Google Scholar 

  293. Hacker P, Engel A, Mauer M. Regulating chatGPT and other large generative AI models. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 2023, pp. 1112–23.

  294. Rajput RS, Shah S, Neema S. Content moderation framework for the LLM-based recommendation systems. Journal of Computer Engineering and Technology (IJCET). 2023;14(3):104–17.

    Google Scholar 

  295. Kilovaty I. Hacking generative AI. Loyola of Los Angeles Law Review, vol. 58. 2025.

Download references

Acknowledgements

This work was supported by CHANAKYA Fellowship Program of TIH Foundation for IoT & IoE (TIH-IoT) received by Dr. Vinay Chamola under Project Grant File CFP/2022/027.

Funding

No funding was obtained for this study.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Birla Institute of Technology & Science (BITS), Pilani, 333031, Rajasthan, India

    Vinay Chamola & Siva Sai

  2. Anuradha and Prashanth Palakurthi Centre for Artificial Intelligence Research (APPCAIR), Palakurthi, India

    Vinay Chamola

  3. Department of Computer Science and Engineering, BITS-Pilani, Pilani Campus, Pilani, India

    Animesh Bhargava

  4. Department of Computer Science and Engineering, Jorhat Engineering College, Jorhat, India

    Ashis Sahu

  5. Pillar of Information Systems Technology and Design, Singapore University of Technology and Design, Singapore, Singapore

    Wenchao Jiang & Zehui Xiong

  6. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Dusit Niyato

  7. School of Computing, Edinburgh Napier University, Scotland, UK

    Amir Hussain

Authors
  1. Vinay Chamola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siva Sai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Animesh Bhargava
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashis Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenchao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zehui Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dusit Niyato
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Vinay Chamola, Siva Sai, Animesh Bhargava, and Ashis Sahu wrote the main manuscript text. Wenchao Jiang, Zehui Xiong, Dusit Niyato, and Amir Hussain prepared figures and reviewed the document in multiple iterations. All authors reviewed the manuscript.

Corresponding author

Correspondence to Vinay Chamola.

Ethics declarations

Ethical Approval

This article contains no studies with human participants or animals performed by authors.

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chamola, V., Sai, S., Bhargava, A. et al. A Comprehensive Survey on Generative AI for Metaverse: Enabling Immersive Experience. Cogn Comput 16, 3286–3315 (2024). https://doi.org/10.1007/s12559-024-10342-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-024-10342-9

Keywords

Search

Navigation