Biofeedback in Virtual Reality

Virtual reality (VR) interfaces is an influential medium to trigger emotional changes in humans. However, there is little research on making users of VR interfaces aware of their own and in collaborative interfaces, one another’s emotional state.

In this project, through a series of system development and user evaluations, we are investigating how physiological data such as heart rate, galvanic skin response, pupil dilation, and EEG can be used as a medium to communicate emotional states either to self (single user interfaces) or the collaborator (collaborative interfaces). The overarching goal is to make VR environments more empathetic and collaborators more aware of each other’s emotional state by providing biofeedback.

Publications

  • Effects of Manipulating Physiological Feedback in Immersive Virtual Environments
    Arindam Dey, Hao Chen, Mark Billinghurst, Robert W Lindeman

    Dey, A., Chen, H., Billinghurst, M. and Lindeman, R.W., 2018, October. Effects of Manipulating Physiological Feedback in Immersive Virtual Environments. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (pp. 101-111). ACM.

    @inproceedings{dey2018effects,
    title={Effects of Manipulating Physiological Feedback in Immersive Virtual Environments},
    author={Dey, Arindam and Chen, Hao and Billinghurst, Mark and Lindeman, Robert W},
    booktitle={Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play},
    pages={101--111},
    year={2018},
    organization={ACM}
    }
    Virtual environments have been proven to be effective in evoking emotions. Earlier research has found that physiological data is a valid measurement of the emotional state of the user. Being able to see one's physiological feedback in a virtual environment has proven to make the application more enjoyable. In this paper, we have investigated the effects of manipulating heart rate feedback provided to the participants in a single user immersive virtual environment. Our results show that providing slightly faster or slower real-time heart rate feedback can alter participants' emotions more than providing unmodified feedback. However, altering the feedback does not alter real physiological signals.
  • Effects of Sharing Real-Time Multi-Sensory Heart Rate Feedback in Different Immersive Collaborative Virtual Environments
    Arindam Dey, Hao Chen, Chang Zhuang, Mark Billinghurst, Robert W Lindeman

    Dey, A., Chen, H., Zhuang, C., Billinghurst, M. and Lindeman, R.W., 2018, October. Effects of Sharing Real-Time Multi-Sensory Heart Rate Feedback in Different Immersive Collaborative Virtual Environments. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 165-173). IEEE.

    @INPROCEEDINGS{8613762,
    author={A. {Dey} and H. {Chen} and C. {Zhuang} and M. {Billinghurst} and R. W. {Lindeman}},
    booktitle={2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
    title={Effects of Sharing Real-Time Multi-Sensory Heart Rate Feedback in Different Immersive Collaborative Virtual Environments},
    year={2018},
    volume={},
    number={},
    pages={165-173},
    keywords={feedback;groupware;virtual reality;immersive collaborative virtual environments;real-time multisensory heart rate feedback;collaborative VR environments;real-time heart rate feedback participants;providing heart rate feedback;single user environments;providing physiological feedback;Heart rate;Collaboration;Real-time systems;Physiology;Avatars;Task analysis;Virtual environments},
    doi={10.1109/ISMAR.2018.00052},
    ISSN={1554-7868},
    month={Oct},}
    Collaboration is an important application area for virtual reality (VR). However, unlike in the real world, collaboration in VR misses important empathetic cues that can make collaborators aware of each other's emotional states. Providing physiological feedback, such as heart rate or respiration rate, to users in VR has been shown to create a positive impact in single user environments. In this paper, through a rigorous mixed-factorial user experiment, we evaluated how providing heart rate feedback to collaborators influences their collaboration in three different environments requiring different kinds of collaboration. We have found that when provided with real-time heart rate feedback participants felt the presence of the collaborator more and felt that they understood their collaborator's emotional state more. Heart rate feedback also made participants feel more dominant when performing the task. We discuss the implication of this research for collaborative VR environments, provide design guidelines, and directions for future research.