[Reference] FMETP Unity asset implementation in The Scientific Computing Conference - May 12
- Publication: Development of an educational escape game for Virtual Reality, Augmented Reality and Mobile phone
5 Implementation
Escape YOUR brain was developed with Unity and C#. The AR Foundation framework was used for the initial implementation of the AR application [Zwi22]. The Mixed Reality Toolkit (MRTK) was used for porting this application to the HoloLens 2. The overall concept utilizes four main methods of data transmission. First the Photon Unity Network (PUN) 2 asset is used for synchronizing movement (e.g. head movement between AR and VR), game events (e.g. level completion) and voice (by using the Photon Voice extension) over a cloud based solution. Between the AR, VR and admin application (gamemaster) as well as the web dashboard (gamemaster) the WebSocket protocol is used to stream the player’s view (FMETP Unity asset). Furthermore, the WebSocket protocol is also used between the dashboard and a backend server for transmitting the current time and other game information. This information is obtained through REST calls originating from the mobile application to the backend server. This backend server forwards this information via the fourth way of data transmission, MQTT, to the admin application. By leveraging this protocol, the overall architecture supports the integration of IoT devices to further enhance the players’ immersion.