Heterogeneous Modeling of Gesture-Based 3D Applications

Model-driven software engineering (MDE) is a well-known approach for developing software. It reduces complexity, facilitates maintenance and allows for the simulation, ver- ification, validation and execution of software models.
In this article, we show how MDE and model execution can be leveraged in the context of human-computer interaction (HCI). We claim that in this application domain it is benefi- cial to use heterogeneous models, combining different models of computation for different components of the system. We report on a case study that we have carried out to develop an executable model of a gesture-based application for manipu- lating 3D objects, using the Kinect sensor as input device, and the OGRE graphical engine as output device for real- time rendering.
The interaction part of this application is fully specified as an executable heterogeneous model with the ModHel'X modeling environment. We exploit the seman- tic adaptation between different models of computation to implement a layered application using the most appropriate models of computation for each layer.