Statechart modelling of interactive gesture-based applications

Developing intuitive interactive applications that are easy to maintain by developers is quite challenging, due to the com- plexity and the many technical aspects involved in such ap- plications. In this article, we tackle the problem in two com- plementary ways. First, we propose a gestural interface to improve the user experience when interacting with applica- tions that require the manipulation of 3D graphical scenes. Second, we reduce the complexity of developing such appli- cations by modeling their executable behaviour using stat- echarts. We validate our approach by creating a modular and extensible Java framework for the development of in- teractive gesture-based applications. We developed a proof- of-concept application using this framework, that allows the user to construct and manipulate 3D scenes in OpenGL by using hand gestures only. These hand gestures are captured by the Kinect sensor, and translated into events and actions that are interpreted and executed by communicating state- charts that model the main behaviour of the interactive ap- plication.