Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions Without the Presence of Pertinent Adverse Symptomatology

Panagiotis Kourtesis, Simona Collina, Leonidas A A Doumas, Sarah E MacPherson, Panagiotis Kourtesis, Simona Collina, Leonidas A A Doumas, Sarah E MacPherson

Abstract

There are major concerns about the suitability of immersive virtual reality (VR) systems (i.e., head-mounted display; HMD) to be implemented in research and clinical settings, because of the presence of nausea, dizziness, disorientation, fatigue, and instability (i.e., VR induced symptoms and effects; VRISE). Research suggests that the duration of a VR session modulates the presence and intensity of VRISE, but there are no suggestions regarding the appropriate maximum duration of VR sessions. The implementation of high-end VR HMDs in conjunction with ergonomic VR software seems to mitigate the presence of VRISE substantially. However, a brief tool does not currently exist to appraise and report both the quality of software features and VRISE intensity quantitatively. The Virtual Reality Neuroscience Questionnaire (VRNQ) was developed to assess the quality of VR software in terms of user experience, game mechanics, in-game assistance, and VRISE. Forty participants aged between 28 and 43 years were recruited (18 gamers and 22 non-gamers) for the study. They participated in 3 different VR sessions until they felt weary or discomfort and subsequently filled in the VRNQ. Our results demonstrated that VRNQ is a valid tool for assessing VR software as it has good convergent, discriminant, and construct validity. The maximum duration of VR sessions should be between 55 and 70 min when the VR software meets or exceeds the parsimonious cut-offs of the VRNQ and the users are familiarized with the VR system. Also, the gaming experience does not seem to affect how long VR sessions should last. Also, while the quality of VR software substantially modulates the maximum duration of VR sessions, age and education do not. Finally, deeper immersion, better quality of graphics and sound, and more helpful in-game instructions and prompts were found to reduce VRISE intensity. The VRNQ facilitates the brief assessment and reporting of the quality of VR software features and/or the intensity of VRISE, while its minimum and parsimonious cut-offs may appraise the suitability of VR software for implementation in research and clinical settings. The findings of this study contribute to the establishment of rigorous VR methods that are crucial for the viability of immersive VR as a research and clinical tool in cognitive neuroscience and neuropsychology.

Keywords: VR sickness; VRISE; cybersickness; motion sickness; neuropsychology; neuroscience; psychology; virtual reality.

Copyright © 2019 Kourtesis, Collina, Doumas and MacPherson.

Figures

FIGURE 1
FIGURE 1
CFA: model’s path diagram. From left to right: the structural model illustrates the associations between VRNQ domains (paths with double headed arrow) and between each VRNQ domain and its items. At the right there are the error items (e) for each item; USER, user experience; GM, game mechanics; GA, in-game assistance; VR, VRISE.
FIGURE 2
FIGURE 2
VRISE intensity in VR sessions as measured by VRNQ. Median scores of VRISE items of VRNQ; VRNQ Minimum Cut-off (≥); VRNQ Parsimonious Cut-off (≥); 1, Extreme intense feeling; 2, Very intense feeling; 3, Intense feeling; 4, Moderate feeling; 5, Mild feeling; 6, Very mild feeling; 7, Absent feeling.
FIGURE 3
FIGURE 3
Variables’ prior inclusion Probabilities.

References

    1. Arafat I. M., Ferdous S. M. S., Quarles J. (2018). “Cybersickness-provoking virtual reality alters brain signals of persons with multiple sclerosis,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), (Reutlingen: IEEE; ), 1–120.
    1. Arbuckle J. L. (2014). Amos (Version 23.0). Chicago: IBM SPSS.
    1. Bohil C. J., Alicea B., Biocca F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12 752–762. 10.1038/nrn3122
    1. Brade J., Dudczig M., Klimant P. (2018). “Using virtual prototyping technologies to evaluate human-machine-interaction concepts,” in aw&I Conference, Vol. 3 (Chemnitz: ).
    1. Brennesholtz M. S. (2018). 3−1: invited paper: vr standards and guidelines. SID Symposium Dig. Tech. Pap. 49 1–4. 10.1002/sdtp.12476
    1. Button K. S., Ioannidis J. P., Mokrysz C., Nosek B. A., Flint J., Robinson E. S., et al. (2013). Confidence and precision increase with high statistical power. Nat. Rev. Neurosci. 14:585. 10.1038/nrn3475-c4
    1. Cohen J. (2013). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge.
    1. Cole D. A. (1987). Utility of confirmatory factor analysis in test validation research. J. Consul. Clin. Psychol. 55 584–594. 10.1037//0022-006x.55.4.584
    1. de França A. C. P., Soares M. M. (2017). “Review of virtual reality technology: an ergonomic approach and current challenges,” in International Conference on Applied Human Factors and Ergonomics, (Cham: Springer; ), 52–61. 10.1007/978-3-319-60582-1_6
    1. Dennison M., Jr., D’Zmura M., Harrison A., Lee M., Raglin A. (2019). “Improving motion sickness severity classification through multi-modal data fusion,” in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Vol. 11006 (Baltimore, MA.), 110060T.
    1. Dennison M. S., Wisti A. Z., D’Zmura M. (2016). Use of physiological signals to predict cybersickness. Displays 44 42–52. 10.1016/j.displa.2016.07.002
    1. Ferrand S., Alouges F., Aussal M. (2017). “Binaural spatialization methods for indoor navigation,” in Audio Engineering Society Convention, Vol. 142 (New York, NY: Audio Engineering Society; ).
    1. Gavgani A. M., Wong R. H., Howe P. R., Hodgson D. M., Walker F. R., Nalivaiko E. (2018). Cybersickness-related changes in brain hemodynamics: a pilot study comparing transcranial Doppler and near-infrared spectroscopy assessments during a virtual ride on a roller coaster. Physiol. Behav. 191 56–64. 10.1016/j.physbeh.2018.04.007
    1. Harpe S. E. (2015). How to analyze likert and other rating scale data. Curr. Pharm. Teach. Learn. 7 836–850. 10.1016/j.cptl.2015.08.001
    1. Hopwood C. J., Donnellan M. B. (2010). How should the internal structure of personality inventories be evaluated? Personal. Soc. Psychol. Rev. 14 332–346. 10.1177/1088868310361240
    1. Hu L. T., Bentler P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struc. Equ. Model. 6 1–55. 10.1080/10705519909540118
    1. Ibm Corp, (2016). IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp.
    1. Jackson D. L., Gillaspy J. A., Jr., Purc-Stephenson R. (2009). Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol. Methods 14 6–23. 10.1037/a0014694
    1. Jasp Team, (2017). JASP (Version 0.8.1.2).
    1. Kennedy R. S., Lane N. E., Berbaum K. S., Lilienthal M. G. (1993). Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3 203–220. 10.1207/s15327108ijap0303_3
    1. Kim H. K., Park J., Choi Y., Choe M. (2018). Virtual reality sickness questionnaire (VRSQ): motion sickness measurement index in a virtual reality environment. Appl. Ergon. 69 66–73. 10.1016/j.apergo.2017.12.016
    1. Kim Y. Y., Kim H. J., Kim E. N., Ko H. D., Kim H. T. (2005). Characteristic changes in the physiological components of cybersickness. Psychophysiology 42 616–625.
    1. Kobayashi M., Ueno K., Ise S. (2015). The effects of spatialized sounds on the sense of presence in auditory virtual environments: a psychological and physiological study. Presence 24 163–174. 10.1162/pres_a_00226
    1. Kortum P., Peres S. C. (2014). The relationship between system effectiveness and subjective usability scores using the system usability scale. Int. J. Hum. Comput. Int. 30 575–584. 10.1080/10447318.2014.904177
    1. Kourtesis P., Collina S., Doumas L. A. A., MacPherson S. E. (2019). Technological competence is a precondition for effective implementation of virtual reality head mounted displays in human neuroscience: a technological review and meta-analysis. Front. Hum. Neurosci. 13:342.
    1. LaViola J. J., Jr. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bull. 32 47–56. 10.1145/333329.333344
    1. Lavoué G., Mantiuk R. (2015). “Quality assessment in computer graphics,” in Visual Signal Quality Assessment, (Cham: Springer; ), 243–286. 10.1007/978-3-319-10368-6_9
    1. Liang F., Paulo R., Molina G., Clyde M. A., Berger J. O. (2008). Mixtures of g priors for bayesian variable selection. J. Am. Stat. Assoc. 103 410–423.
    1. Lum H. C., Greatbatch R., Waldfogle G., Benedict J. (2018). “How Immersion, Presence, Emotion, & Workload Differ in Virtual Reality and Traditional Game Mediums,” in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 62 (Los Angeles, CA: SAGE Publications; ), 1474–1478. 10.1177/1541931218621334
    1. Marsman M., Wagenmakers E. J. (2017). Bayesian benefits with JASP. Eur. J. Dev. Psychol. 14 545–555. 10.1080/17405629.2016.1259614
    1. McCauley M. E., Sharkey T. J. (1992). Cybersickness: perception of self-motion in virtual environments. Presence 1 311–318. 10.1162/pres.1992.1.3.311
    1. Milleville-Pennel I., Charron C. (2015). Do mental workload and presence experienced when driving a real car predispose drivers to simulator sickness? An exploratory study. Accid. Anal. Prev. 74 192–202. 10.1016/j.aap.2014.10.021
    1. Mittelstaedt J., Wacker J., Stelling D. (2018). Effects of display type and motion control on cybersickness in a virtual bike simulator. Displays 51 43–50. 10.1016/j.displa.2018.01.002
    1. Mittelstaedt J. M., Wacker J., Stelling D. (2019). VR aftereffect and the relation of cybersickness and cognitive performance. Vir. Real. 23 143–154. 10.1007/s10055-018-0370-3
    1. Nalivaiko E., Davis S. L., Blackmore K. L., Vakulin A., Nesbitt K. V. (2015). Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Physiol. Behav. 151 583–590. 10.1016/j.physbeh.2015.08.043
    1. Nesbitt K., Davis S., Blackmore K., Nalivaiko E. (2017). Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays 48 1–8. 10.1016/j.displa.2017.01.002
    1. Nunally J. C., Bernstein I. H. (1994). Psychometric Theory, 3rd Edn New Yokr, NY: Mcgraw-Hill.
    1. Palmisano S., Mursic R., Kim J. (2017). Vection and cybersickness generated by head-and-display motion in the oculus rift. Displays 46 1–8. 10.1016/j.displa.2016.11.001
    1. Parsons T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9:660. 10.3389/fnhum.2015.00660
    1. Parsons T. D., McMahan T., Kane R. (2018). Practice parameters facilitating adoption of advanced technologies for enhancing neuropsychological assessment paradigms. Clin. Neuropsychol. 32 16–41. 10.1080/13854046.2017.1337932
    1. Riecke B. E., Feuereissen D., Rieser J. J., McNamara T. P. (2011). “Spatialized sound enhances biomechanically-induced self-motion illusion (vection),” in Proceedings of the SIGCssHI Conference on Human Factors in Computing Systems, (New York, NY: ACM.), 2799–2802.
    1. Rouder J. N., Morey R. D. (2012). Default bayes factors for model selection in regression. Multivar. Behav. Res. 47 877–903. 10.1080/00273171.2012.734737
    1. Sharples S., Cobb S., Moody A., Wilson J. R. (2008). Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 29 58–69. 10.1016/j.displa.2007.09.005
    1. Soper D. S. (2019a). A-priori Sample Size Calculator for Structural Equation Models. Available at (accessed July 10, 2019).
    1. Soper D. S. (2019b). Post-hoc Statistical Power Calculator for Multiple Regression. Available at (accessed July 10, 2019).
    1. Stanney K. M., Kennedy R. S., Drexler J. M. (1997). “Cybersickness is not simulator sickness,” In Proceedings of the Human Factors and Ergonomics Society annual meeting, (Los Angeles, CA: SAGE Publications; ). 41 1138–1142.
    1. Toschi N., Kim J., Sclocco R., Duggento A., Barbieri R., Kuo B., et al. (2017). Motion sickness increases functional connectivity between visual motion and nausea-associated brain regions. Auton. Neurosci. 202 108–113. 10.1016/j.autneu.2016.10.003
    1. Weech S., Kenny S., Barnett-Cowan M. (2019). Presence and cybersickness in virtual reality are negatively related: a review. Front. Psychol. 10:158. 10.3389/fpsyg.2019.00158
    1. Wetzels R., Wagenmakers E. J. (2012). A default bayesian hypothesis test for correlations and partial correlations. Psychon. Bull. Rev. 19 1057–1064. 10.3758/s13423-012-0295-x
    1. Zarour M., Abran A., Desharnais J. M., Alarifi A. (2015). An investigation into the best practices for the successful design and implementation of lightweight software process assessment methods: a systematic literature review. J. Syst. Softw. 101 180–192. 10.1016/j.jss.2014.11.041

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever