Problem Statement
Our Problem Statement & Research Objectives
Last updated
Our Problem Statement & Research Objectives
Last updated
Virtual reality (VR) has been used to provide immersive experience in pedagogical contexts, enabling students to grasp complex concepts through experiential learning. University of Bristol developed the VR oracle project for its history students to walk through the streets of ancient Greece [20]. These experiences foster deeper connection with the subject [12].
However, contemporary VR worlds are static and predefined. These environments are built upon finite state machines (FSMs) and pre-rendered graphics. FSMs dictate deterministic behaviours in the environment while pre-rendered graphics restricts real-time adaptability [1]. Hence, resulting in explicit creation of dedicated VR modules tailored for specific purposes. This methodology is manpower and financially intensive, and limits personalised content.
Figure 1: Videos games that used GAI support [4]
Generative AI (GAI) can automate the creation of VR worlds. By training neural networks, GAI can generate limited assets (e.g., avatars, objects, etc.) based on user prompts. From Fig. 1, there is an increasing trend in implementing GAI in open-world-games. Skyrim, for instance, automates character interactions using GAI [15]. However, a research gap exists in the confluence of these two technologies to generate virtual worlds [23].
This research develops a multi-modal radiance fields (MMRF) model that generates a virtual world composing volumetric scenes using multimodal inputs. We will incorporate a physics engine embedded with text-driven physics customisation capable of zero-shot content property customisation.
We will focus on the generation of mass structures with customisable physical properties. This study will be scalable and extrapolated to other areas of education. We aim to evaluate GAI and VR’s impact on understanding and retaining structural concepts.
