Construction of an Evaluation Index System for User Satisfaction with Immersive Virtual Reality Exergaming

Advances in virtual reality technology have unlocked the potential for immersive virtual reality exergaming (IVRE), presenting a significant avenue for increasing public physical activity levels and fostering the growth of recreational sports. A new study proposes a comprehensive framework designed to systematically evaluate user satisfaction (USAT) within this rapidly evolving domain. This framework includes detailed measurement instruments and decision-making support, aiming to guide the assessment and enhancement of user experience in IVRE.

The research, published in Frontiers in Psychology, employed a rigorous, multi-stage methodology. Initially, grounded theory, a qualitative approach, was utilized to conduct in-depth interviews with IVRE users. This process allowed researchers to identify the core factors that significantly influence user satisfaction. Subsequently, expert opinions were combined with a sophisticated statistical technique, the G1-CRITIC method, to quantitatively determine the relative importance of each identified factor. To validate the practical applicability of the developed framework, it was tested on eight popular IVRE platforms using the multi-criteria decision-making tool, the VIKOR method.

The findings reveal a robust USAT evaluation framework composed of five primary indices, fourteen secondary indices, and fifty-seven tertiary indices. Empirical validation demonstrated satisfactory reliability and validity, offering a crucial methodological and empirical foundation for the design and development of IVRE technologies with an enhanced focus on user experience. This research not only contributes to the existing theoretical perspectives and methodological approaches in IVRE but also provides a valuable reference for future studies in this burgeoning field.

The Growing Landscape of Immersive Virtual Reality Exergaming

Immersive Virtual Reality Exergaming (IVRE) has emerged as a significant force in the digital health and entertainment sectors, attracting considerable attention from both academic and industrial spheres. Its appeal lies in its unique blend of embodied interaction, minimal spatial constraints, and cost-effectiveness, making it an accessible and engaging form of physical activity. As the virtual reality market continues its exponential growth, projected to reach an estimated $120.74 billion by 2032 according to GLOBE NEWSWIRE, the IVRE segment is poised for substantial expansion.

Despite this potential, user acceptance of IVRE products remains varied. User satisfaction (USAT) is recognized as a pivotal factor influencing not only sustained usage intention but also the long-term viability of these technologies. Therefore, a systematic and scientific evaluation of USAT throughout the design and development phases of IVRE is not merely beneficial but critical for its widespread adoption and success.

Existing research has primarily focused on the intervention effectiveness of IVRE, its suitability for specific user groups, and various product development pathways. However, a notable gap exists in systematic investigations dedicated to establishing comprehensive evaluation index systems for IVRE user satisfaction. This study aims to fill that void by proposing and validating such a system.

Methodology: A Three-Stage Approach to Understanding User Satisfaction

The research team adopted a methodical three-stage approach to construct and validate their USAT evaluation framework:

• Stage 1: Indicator Identification: Grounded theory was employed to systematically explore user experiences and derive a preliminary list of satisfaction indicators directly from user narratives. This qualitative phase involved in-depth interviews to capture rich, authentic perspectives.
• Stage 2: Indicator Weighting: The G1-CRITIC method was applied to assign weights to the identified indicators. This hybrid approach balances subjective expert judgment (G1 method) with objective data analysis derived from user questionnaires (CRITIC method), providing a nuanced understanding of factor importance.
• Stage 3: Framework Validation: The complete framework was rigorously tested by evaluating eight popular IVRE platforms. The VIKOR method, a multi-criteria decision-making tool, was used to rank these platforms based on the established USAT indices, thereby confirming the framework’s effectiveness and applicability in a real-world context.

Unearthing User Satisfaction: The Grounded Theory Approach

The initial phase of the study focused on data collection through semi-structured interviews, a method chosen for its ability to elicit detailed and nuanced user experiences. Guided by an interview protocol developed from a thorough review of existing IVRE literature and internal team discussions, researchers engaged in open-ended, interactive conversations with participants. The protocol was designed to cover four key thematic areas: overall IVRE experience, specific aspects of the user journey (immersion, usability, interactivity, enjoyment), social dimensions for interactive platforms, and suggestions for improvement.

To ensure the quality and relevance of the questions, the protocol underwent a pilot test with two experienced IVRE users, with minor adjustments made based on their feedback. Interviews were conducted by trained researchers, offered in both in-person and online formats to accommodate participant preferences and geographical locations. Each session averaged approximately 55 minutes, with all interviews audio-recorded with explicit consent and transcribed verbatim. Rigorous accuracy checks were performed on a subset of transcripts against the original recordings.

Participant recruitment prioritized diversity in disciplinary backgrounds and age groups, adhering to strict inclusion criteria: experience with multiple IVRE types, ability to articulate views effectively, and provision of informed consent. The principle of theoretical saturation sampling was applied, where data was continuously analyzed, coded, and organized after each interview. When no new categories emerged after cross-validation with at least three additional textual sources, theoretical saturation was deemed achieved, confirming the final sample size of 16 participants.

The qualitative data was then subjected to a rigorous three-stage coding process:

• Open Coding: This initial stage involved a line-by-line examination of interview transcripts to identify core meanings and generate basic categories. This process yielded 57 initial categories, forming the bedrock for further analysis.
• Axial Coding: Building upon open coding, axial coding explored the logical relationships among categories. Basic categories were integrated to form higher-level abstract concepts. This led to the refinement of the 57 basic categories into fourteen distinct subcategories, organized under five main categories: Enjoyment, Effectiveness, Usability, Interactivity, and Sociality. These subcategories represent key facets of the user experience, such as gamified design, adaptive exercise guidance, customizable settings, user-friendly interfaces, and social interaction mechanisms.
• Selective Coding: This final qualitative stage synthesized and abstracted the findings from open and axial coding. It elucidated the relationships between the main categories and overall user satisfaction, culminating in the formation of a coherent conceptual framework. The study identified strong positive influences of Enjoyment, Effectiveness, Usability, and Interactivity on USAT, with Sociality playing a crucial regulatory role.

Quantifying Importance: The G1-CRITIC and VIKOR Methods

Following the qualitative exploration, the research transitioned to quantitative analysis to determine the relative importance of the identified indicators and validate the framework. The G1-CRITIC method was employed for weighting. The G1 method captured subjective expert judgments on the importance of each indicator, while the CRITIC method utilized objective data from user questionnaires to assess indicator weights based on contrast intensity and correlations. These two sets of weights were then combined using a multiplicative normalization approach to derive composite weights for each indicator across the three hierarchical levels: primary (main categories), secondary (subcategories), and tertiary (basic categories).

The results of the combined weighting revealed significant insights into what drives user satisfaction in IVRE. Enjoyment emerged as the most influential primary index, accounting for 32.10% of the total weight. This was followed by Sociality (19.60%), Usability (18.69%), Interactivity (15.18%), and Effectiveness (14.46%). At the secondary level, specific indicators like gamified experience design, user-friendly interface design, challenge-oriented design, effective and health-oriented exercise, and social interaction design demonstrated particularly prominent weightings, underscoring their importance in shaping user perception.

To quantitatively validate the constructed USAT evaluation index system, the VIKOR method was applied to eight representative IVRE platforms. These platforms were selected based on market recognition, user base, cross-platform compatibility, and language availability to ensure a diverse and representative sample. The VIKOR method, a multi-attribute decision-making tool, ranks alternatives based on their proximity to ideal and anti-ideal solutions.

The VIKOR analysis provided a clear ranking of the eight IVRE platforms. Notably, the platform ranked first (S2) in the VIKOR evaluation also received the highest user rating in the collected questionnaire data. This strong consistency between the VIKOR evaluation results and direct user-reported satisfaction ratings serves as compelling evidence for the effectiveness and validity of the developed USAT evaluation index system.

Key Drivers of User Satisfaction in IVRE

The comprehensive analysis of weighted indicators and the VIKOR validation have illuminated the primary drivers of user satisfaction in immersive virtual reality exergaming.

• Enjoyment (32.10% weight): This dimension stands out as the most critical factor influencing USAT. Its sub-indices, including gamified experience design, system rewards and achievements, challenge-oriented design, and personalized virtual configurations, collectively highlight the importance of creating engaging and intrinsically motivating experiences. The findings align with Flow Theory, suggesting that users are most satisfied when immersed in activities that offer clear goals, immediate feedback, and a balance between skill and challenge. Features like compelling game elements, rewarding progression systems, and personalized virtual environments are paramount for enhancing enjoyment.

• Sociality (19.60% weight): The capacity for social interaction and connection within IVRE environments significantly impacts user satisfaction, aligning with Self-Determination Theory’s emphasis on the need for relatedness. Effective social interaction design, competitive modes, and cooperative modes foster a sense of belonging and social fulfillment. Offering diverse social features, such as virtual social spaces and collaborative tasks, can transform solitary exercise into a shared and rewarding experience, boosting engagement and retention.

• Usability (18.69% weight): The ease with which users can learn, understand, and operate IVRE systems is crucial for reducing cognitive load and lowering participation barriers. Customizable system settings, accessibility-oriented design, and a reasonable usage cost contribute to a positive user experience. Prioritizing intuitive interfaces, adaptable settings for diverse needs, and clear instructions are essential for broader adoption.

• Interactivity (15.18% weight): The quality and extent of interaction between users and IVRE systems directly influence engagement and perceived experience. User-friendly interaction design, characterized by intuitive workflows, responsive feedback, and fault tolerance, coupled with aesthetically pleasing and clearly structured interfaces, enhances immersion and emotional resonance.

• Effectiveness (14.46% weight): The extent to which IVRE achieves its intended functional objectives, such as fitness enhancement or skill development, is a key differentiator from conventional games. Effective and health-oriented exercise features, along with appropriate and adaptive exercise guidance, are vital for users to perceive tangible benefits, thereby increasing their satisfaction.

Implications and Future Directions

The development of this comprehensive USAT evaluation index system offers significant implications for the IVRE industry. It provides developers and businesses with a structured, evidence-based roadmap for designing and improving IVRE products. By focusing on the identified key indices, companies can prioritize features that directly contribute to user satisfaction, leading to increased engagement, longer retention rates, and ultimately, greater market success.

The study’s findings underscore the need for a holistic approach to IVRE design, balancing engaging gameplay with tangible health and usability benefits, and fostering social connections. As the VR landscape continues to evolve, this framework serves as a vital tool for navigating the complexities of user experience and ensuring the sustained growth and positive impact of immersive virtual reality exergaming.

However, the study acknowledges several limitations. The demographic scope of the qualitative sample was restricted, and the quantitative validation sample was convenience-based, potentially limiting generalizability to broader populations, including older adults, children, or individuals with diverse educational backgrounds. Future research could benefit from stratified sampling and longitudinal designs to track satisfaction evolution over time. Furthermore, future studies could explore genre-specific satisfaction factors within IVRE, as different game types may prioritize distinct aspects of the user experience. Finally, cultural contexts and the rapid pace of VR technological advancements necessitate ongoing validation and adaptation of the framework.

In conclusion, this research provides a significant contribution to the field of immersive virtual reality exergaming by establishing a robust, validated evaluation framework for user satisfaction. The detailed indices and quantitative validation offer a clear path forward for developers aiming to create more engaging, effective, and satisfying IVRE experiences, paving the way for a healthier and more active future powered by immersive technologies.