Enhancing VR Gaming Experience using Computational Attention Models and Eye-Tracking

Ennadifi, Elias; Ravet, Thierry; Mancas, Matei; Mokhtari, Mohammed El Amine; Gosselin, Bernard

Request a copy

Paper published in a journal (Scientific congresses and symposiums)

Enhancing VR Gaming Experience using Computational Attention Models and Eye-Tracking

Ennadifi, Elias; Ravet, Thierry; Mancas, Matei et al.

2023 • In ACM International Conference on Interactive Media Experiences (IMX)

Peer reviewed

Permalink
https://hdl.handle.net/20.500.12907/46056

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Enhancing VR Gaming Experience using Computational Attention Models and Eye-Tracking.pdf

Author postprint (9.53 MB)

Request a copy

All documents in ORBi UMONS are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Virtual reality; Eye-tracking; Computational attention models; Salience map; Difficulty adjustment

Abstract :

[en] This study explores the potential of enhancing interaction experiences, such as virtual reality (VR) games, through the use of computational attention models. Our proposed approach utilizes a saliency map generated by attention models to dynamically adjust game difficulty levels and to help in the game level design, resulting in a more immersive and engaging experience for users. To inform the development of this approach, we present an experimental setup that is able to collect data in a VR environment and intends to be able to validate the adaptation of attention models to this domain. Through this work, we aim to create a framework for VR game design that leverages attention models to offer a new level of immersion and engagement for users. We believe our contributions have significant potential to enhance VR experiences and advance the field of game design.

Disciplines :

Computer science

Author, co-author :

Ennadifi, Elias ; Université de Mons - UMONS

Ravet, Thierry ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle

Mancas, Matei ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle

Mokhtari, Mohammed El Amine ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle

Gosselin, Bernard ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle

Language :

English

Title :

Enhancing VR Gaming Experience using Computational Attention Models and Eye-Tracking

Publication date :

15 June 2023

Event name :

ACM International Conference on Interactive Media Experiences (IMX)

Event place :

nantes, France

Event date :

June 12– 15, 2023

Audience :

International

Journal title :

ACM International Conference on Interactive Media Experiences (IMX)

Peer review/Selection committee :

Peer reviewed

Research unit :

F105 - Information, Signal et Intelligence artificielle

Research institute :

R450 - Institut NUMEDIART pour les Technologies des Arts Numériques

Available on ORBi UMONS :

since 19 June 2023

Statistics

Number of views

121 (13 by UMONS)

Number of downloads

3 (3 by UMONS)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

Bibliography

Isayas Berhe Adhanom, Paul MacNeilage, and Eelke Folmer. 2023. Eye Tracking in Virtual Reality: A Broad Review of Applications and Challenges. Virtual Reality (2023), 1-24.
Bridget K Behe, Mikyeung Bae, Patricia T Huddleston, and Lynnell Sage. 2015. The effect of involvement on visual attention and product choice. Journal of Retailing and Consumer Services 24 (2015), 10-21.
Ali Borji, Ming-Ming Cheng, Qibin Hou, Huaizu Jiang, and Jia Li. 2019. Salient object detection: A survey. Computational visual media 5 (2019), 117-150.
N. Bruce and J. Tsotsos. 2010. Attention based on information maximization. Journal of Vision-J VISION 7 (06 2010), 950-950. https://doi.org/10.1167/7.9.950
Anthony Cardinal. 2023. UNREAL ENGINE 5 : Guide complet développeur de jeux 2023. https://www.udemy.com/user/cardinale-anthony/
Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. 2017. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5659-5667.
Viviane Clay, Peter König, and Sabine Koenig. 2019. Eye tracking in virtual reality. Journal of eye movement research 12, 1 (2019).
Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, and Rita Cucchiara. 2017. A Deep Multi-Level Network for Saliency Prediction. arXiv:1609.01064 [cs.CV]
M. Cornia, L.o Baraldi, G. Serra, and R. Cucchiara. 2018. Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model. IEEE Transactions on Image Processing 27, 10 (2018), 5142-5154.
Richard Droste, Jianbo Jiao, and J Alison Noble. 2020. Unified image and video saliency modeling. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16. Springer, 419-435.
Epic Game. 2023. UNREAL ENGINE. https://www.unrealengine.com
A. Garcia-Diaz, V. Leboran, X. Fdez-Vidal, and X. Pardo. 2012. On the relationship between optical variability, visual saliency, and eye fixations: A computational approach. Journal of vision 12 (06 2012), 17. https://doi.org/10.1167/12.6.17
Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming-Ming Cheng, and Shi-Min Hu. 2022. Visual attention network. arXiv preprint arXiv:2202.09741 (2022).
J. Han, D. Zhang, X. Hu, K. Li, J. Ren, and F. Wu. 2014. Background Prior Based Salient Object Detection via Deep Reconstruction Residual. IEEE Transactions on Circuits and Systems for Video Technology 25 (01 2014), 1-1. https://doi.org/10.1109/TCSVT.2014.2381471
J. Harel, C. Koch, and P. Perona. 2007. Graph-based visual saliency. In Advances in neural information processing systems. 545-552.
HTC. 2023. Vive. https://www.vive.com
L. Itti and C. Koch. 2000. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40 (06 2000).
Ming Jiang, Shengsheng Huang, Juanyong Duan, and Qi Zhao. 2015. SALICON: Saliency in Context. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1072-1080. https://doi.org/10.1109/CVPR.2015.7298710
Esther Jun, Steven Landry, and Gavriel Salvendy. 2011. A visual information processing model to characterize interactive visualization environments. Intl. Journal of Human-Computer Interaction 27, 4 (2011), 348-363.
Zhaoyang Niu, Guoqiang Zhong, and Hui Yu. 2021. A review on the attention mechanism of deep learning. Neurocomputing 452 (2021), 48-62.
Cagri Ozcinar and Aljosa Smolic. 2018. Visual attention in omnidirectional video for virtual reality applications. In 2018 Tenth international conference on quality of multimedia experience (QoMEX). IEEE, 1-6.
Jella Pfeiffer, Thies Pfeiffer, Martin Meißner, and Elisa Weiß. 2020. Eye-trackingbased classification of information search behavior using machine learning: Evidence from experiments in physical shops and virtual reality shopping environments. Information Systems Research 31, 3 (2020), 675-691.
Pico. 2023. Neo. https://www.picoxr.com/global/products/neo3-pro-eye
J. Qin and L. Xu. 2015. Data Acquisition and digital Instrumentation Engineering Modelling for Intelligent Learning and Recognition. Biosensors Journal 4 (03 2015), 1-4. https://doi.org/10.4172/2090-4967.1000e103
N. Riche, M. Mancas, M. Duvinage, M. Mibulumukini, B. Gosselin, and T. Dutoit. 2013. RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Processing Image Communication 28 (03 2013). https://doi.org/10.1016/j.image.2013.03.009.
R. Rosenholtz. 1999. A simple saliency model predicts a number of motion popout phenomena. Vision research 39 (09 1999), 3157-63. https://doi.org/10.1016/S0042-6989(99)00077-2
Helmut Schrom-Feiertag, Volker Settgast, and Stefan Seer. 2017. Evaluation of indoor guidance systems using eye tracking in an immersive virtual environment. Spatial Cognition & Computation 17, 1-2 (2017), 163-183.
Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and GordonWetzstein. 2018. Saliency in VR: How do people explore virtual environments IEEE transactions on visualization and computer graphics 24, 4 (2018), 1633-1642.
Pengfei Sun and Jun Qin. 2016. Enhanced factored three-way restricted boltzmann machines for speech detection. arXiv preprint arXiv:1611.00326 (2016).
Pengfei Sun and Jun Qin. 2017. Neural networks based EEG-Speech Models. arXiv:1612.05369 [cs.SD]
Evgeniy Upenik and Touradj Ebrahimi. 2017. A simple method to obtain visual attention data in head mounted virtual reality. In 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 73-78.
varjo. 2023. VR-3. https://varjo.com/products/vr-3/
Ziqiang Wang, Zhi Liu, Gongyang Li, Yang Wang, Tianhong Zhang, Lihua Xu, and Jijun Wang. 2021. Spatio-temporal self-attention network for video saliency prediction. IEEE Transactions on Multimedia (2021).
J. Zhang and S. Sclaroff. 2013. Saliency Detection: A Boolean Map Approach. In 2013 IEEE International Conference on Computer Vision. 153-160.
Kao Zhang and Zhenzhong Chen. 2018. Video saliency prediction based on spatial-temporal two-stream network. IEEE Transactions on Circuits and Systems for Video Technology 29, 12 (2018), 3544-3557.
Rui Zhao, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. 2015. Saliency detection by multi-context deep learning. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1265-1274. https://doi.org/10.1109/CVPR. 2015.7298731