- As the internet shifts towards decentralisation and communal control, metaverse applications aim to let users perform everyday activities like socialising, working, and shopping within a collective virtual space.
- The metaverse will be shaped by applications allowing real-time user interaction through avatars, and extended reality (XR) technologies with real and virtual components, which both present significant design challenges.
- Dr Christian Stary from Johannes Kepler University Linz, Austria presents a dynamic digital process twin-based approach to metaverse/XR application design.
- This allows for functionality and sustainability optimisation before applications are brought to market and ensures that users are provided with sustainable products.
The eagerly anticipated next phase of the internet, or Web3, will shift away from centralised platforms to a decentralised user network, providing increased control over virtual activities and data. In this changing digital landscape, users would be placed in the cusp between the physical world and the ‘metaverse’, a vast virtual space that allows them to conduct everyday activities while connecting with virtual environments and digital objects.
Mixed-reality is a mixture of real-world and virtual content, such as Instagram filters and virtual furniture fitting applications whereas extended reality or XR is an umbrella term for augmented reality, in which digital elements are overlaid on the physical world, mixed reality, and virtual reality. Metaverse applications and the technologies supporting mixed-reality and XR are developed to let users buy goods, pay bills, socialise, and manage relationships with colleagues online using avatars. Head-mounted displays will be required to interact with these technologies and applications, and this duality of physical and virtual activity leads to complex design challenges.
A Digital Twin (DT) is a high-fidelity computational model of a physical or virtual ecosystem that uses real-time data to simulate behaviour and monitor operations. DTs provide optimisation using feedback between a system and its model. A Cyber-Physical System (CPS) is a complex adaptive system that combines physical and computational components to monitor and control processes, such as autonomous automobile systems, medical monitoring, and industrial robotics systems. A Digital Process Twin (DPT) can be used to model variants of CPS behaviour and run predictive analyses on changes that might impact a business process.
Sustainability and the metaverse
A sustainable system uses resources in a way that supports ecological, human, and economic health and vitality. With the virtual product landscape in continuous evolution and expansion, it is crucial to optimise the sustainability of metaverse/XR applications before they are brought to the market.
With the virtual product landscape in continuous evolution and expansion, it is crucial to optimise the sustainability of metaverse/XR applications.
Dr Christian Stary from Johannes Kepler University Linz, Austria states that the immersive and digital nature of CPS allows for sustainability analysis before processes and products are finalised. This is a key point for metaverse/XR applications, as accurately determining behaviour allows us to experience the precise environmental, social, and economic impacts of system variants and make appropriate choices at the design and analysis stages.
Simulating a virtual world
Researchers previously used DTs to highlight knowledge and abstraction gaps caused by ignoring the inherently dynamic nature of CPS. Stary discusses the interplay between CPS and DTs in his book, From digital twins to digital selves and beyond: Engineering and social models for a trans-humanist world, cowritten by Professor Franz Barachini.
The dynamic nature of a CPS must be addressed to accurately model a system and thus, gain a more precise idea of its behaviour. Stary aims to address the current lack of a user-centred digital design and engineering approach for metaverse/XR applications using DPTs, with a view to capturing their behaviour and optimising system sustainability. Stary’s methodical design approach acknowledges the revolutionary nature of metaverse/XR application development while enabling stakeholder participation in both design and engineering processes.
DPTs allow stakeholders to run models and validate systems before implementing corresponding metaverse/XR applications. These DPTs consider evolving stakeholder roles and technological capacity, system components and collaborations, providing a user-focused level of system abstraction, and enabling sustainability assessment, although ensuring model accuracy is key to optimising product sustainability.
Metaverse/XR application design
For metaverse/XR applications, Stary adopts the design science framework: an iterative, solution-orientated design approach that focuses on evolutionary information system design and facilitates stepwise system refinement. The design science methodology provides traceability and transparency at every developmental step; an ideal choice for dynamic applications in which interactions between components, users, and their XR representations must all be considered.
Stary’s design approach provides behaviour-centred application specification and handling, allowing designers to experience and evaluate systems before putting them into practice.
This framework involves design cycles for relevance (connecting the environment of metaverse/XR to its core activities), rigor (relating these activities to the knowledge base informing the research), and design (iterating between the core development activities of building and evaluation). This iterative process is used to produce a validated DPT.
Stary stresses that it is also important to consider the openness of design representations when embedding artificial intelligence components, allowing for either a black-box approach, or one in which processing details are captured to fully understand and interpret results.
Virtual rooms
Once the DPT for a metaverse/XR application is validated, the next step is to build an interactional application prototype controlled by that DPT. Stary introduced the concept of a ‘Room for the Interested Person’ for an NFT (non-functional-token)-business case, implemented using the spatial.io Creator toolkit v1.0. Stary’s ‘room’ is a 3D virtual social space in which users, such as social media contacts, can participate to accomplish a certain task. Users place their avatar in the ‘room’ and visually interact with the content represented there.
Metaverse applications allow for individualised immersion (which can be increased by representing access and control devices visually in the design space) and interfaces with other digital components (such as displaying social media threads). The concept of a ‘room’ allows users to easily engage with content from several perspectives and on various levels of detail, which enhances immersion. Designers and stakeholders can use this type of prototype to determine the effects of different behaviours on an application before it is finalised.
A mixed-reality future
In the future, Web3 will allow metaverse users to design their own mixed-reality environment, interacting with both the real and virtual world using devices such as mixed-reality glasses to perform everyday technical, social, and work-based tasks. Stary’s design approach provides behaviour-centred application specification and handling, allowing designers to experience and evaluate systems before putting them into practice.
Stary’s design architecture also allows people to control the metaverse lifecycle through dynamically adaptable DPT-based behaviour models. Enabling users to simulate how they behave and their environment behaves in common situations allows them to configure and optimise their experience while living sustainably in a mixed-reality world. In the long run, users will replace the developers by creating their ‘Metaverse-by-Design’.
What do you see as the main challenges to metaverse/XR application development?
One of the main hurdles is managing the complexity when adopting metaverse/XR applications and technologies along further digital transformation processes. Both (organisational) designers and tool developers are challenged here.
Which key issues could be addressed to optimise application sustainability?
Some of the key issues to address would include: (1) development of design guidelines ‘for all’ to inform users on technology capabilities; (2) transparent process designs in organisations, through explainable behaviour models, so that users know their roles and can work on their avatars’ appearance, activities, and social behaviour; and (3) exploring social spaces with Web3 capabilities to adopt 3D interaction.
Where do you see the Metaverse and XR in ten years?
Metaverse will be a diversified space for hybrid interaction in B2B and C2B contexts. Every user will be able to define dedicated roles through several avatars.
Metaverse will aid in better organising living in an increased variety of digital or digitised contexts.
How could the concept of ‘rooms’ be used in different applications in the metaverse?
Each activity performed in a room has a specific objective. For instance, informing users on the origin of product ingredients or components that can be shared with others should be located in a room.
