Abstract :
[en] Technology and Trainer Support in Simulation: The Case of ARC Sim'Pro and the Use of ZED2 Cameras for Detailed Participant Monitoring
SUMMATIVE STATEMENT: This presentation discusses the development of technological tools dedicated to behavioral analysis to support trainers in simulation-based training, particularly in the fields of education and civil defense.
PROBLEM STATEMENT: Even for experienced trainers, collecting data during simulations presents challenges with traditional observation methods, whether direct or recorded, due to their complexity (Bocquillon et al., 2018) and validity issues (Porter, 2011). Technological advancements now allow the use of detailed recording and automatic annotation tools (Laurent et al., 2022), especially through signal analysis and machine learning. These technologies can enrich trainers' observations by automatically capturing difficult-to-obtain visuospatial information. However, developing and adapting these tools to the specific needs of training requires time, given the diversity of simulation environments.
OBJECTIVE: In this context, the primary goal of the ARC Sim’Pro project is to develop an assistance tool for trainers with three main objectives: 1) to maintain a training environment that closely reflects real working conditions, without disruptive elements that could affect the natural flow of activities; 2) to ensure that this tool is economically accessible, both in terms of initial cost and ease of use and operation; and 3) to create a flexible tool that can be easily adapted to different training contexts. Within the ARC Sim’Pro project, two training contexts with different environmental conditions were selected: teacher training in a controlled environment and firefighter training in an open environment.
METHODOLOGY: Actions were taken, first resulting in the identification of challenges inherent in the training fields and key information to capture during the simulated exercise. In the field of teacher training, the environment is a micro-teaching classroom equipped with four cameras at each corner, microphones, and a control room with multiple recording and observation software (Noldus). This specifically concerns active circulation within the micro-teaching room, considered an effective supervision and management strategy for students (Bissonnette & Gauthier, 2017). In the field of firefighter training, the actions also include observing movements, but in this case, it involves the movements of the trainee firefighters and the trainer. Secondly, the selection of specific equipment was made: Stereolabs' ZED 2 cameras. Chosen for their ability to accurately capture space in three dimensions, three stereoscopic cameras were installed in the micro-teaching room due to the more controlled nature of this environment. Thirdly, a "test" data collection period was undertaken. The data collection relied on multimodal recordings, including video and audio. The analysis methods were based on data from the fusion of tracked skeletons.
RESULTS/OUTCOMES: To simplify the visualization and interpretation of results, the simulation space was modeled in 2D in an interface, facilitating the analysis and understanding of group dynamics and individual behaviors within the simulation. The interface includes features such as heat maps, trajectory tracking, calculation of interpersonal distances, and identification of frequently traversed zones.
DISCUSSION: The analysis conducted by the ZED2 cameras reveals issues with accurately identifying individuals in close proximity, leading to errors in tracking. This limitation persists even when the trainee teacher remains within a relatively defined area during the lesson, indicating the need for technological optimization for close interactions. Additionally, an interface was designed to improve data visualization, particularly using heat and trajectory maps to better understand space usage. Although basic, this interface lays the foundation for more advanced features, such as accelerated movement analysis and automatic identification of the teacher. Regarding the adaptation of this tool to the firefighter training context, initial observations highlight significant challenges, particularly related to participant positioning, as firefighters move more compared to a teacher in a classroom.
CONCLUSIONS: This experience underscores the potential for pedagogical improvement offered by new technologies, while recognizing the challenges to achieving full effectiveness, especially in "open" training environments. The results suggest a promising integration of these tools into training methods, with adjustments needed for increased precision.
