Congress contribution
Software framework for sensor-based therapy games
Introduction
Performing specific physical exercises regularly is important for patient recovery after severe events such as stroke [1] or surgery on the musculoskeletal system [2]. To ensure patient compliance, these rehabilitation exercises are usually monitored and motivated by physiotherapists.
In order to improve outcome further, promote self-motivation of patients and reduce costs, previous work has targeted encouraging and supervising therapeutic exercises with dedicated computer games using state-of-the-art human/computer interaction devices such as Nintendo Wii [3], Kinect [4], Rayzor Hydra [5]. Furthermore, new input devices are specifically designed for therapy and the market is changing rapidly [6, 7].
Designing therapeutic games is challenging because it requires an interdisciplinary team with clinical, engineering and game design backgrounds. The aim of this project was to simplify the design process of new therapeutic games by developing a software framework to unify tracking of skeletal motion features by external sensors with a game engine providing components for game development.
Methods
Software framework
The proposed framework is composed of sensor plugins, user datagram protocol (UDP) data transport and game layer components. Human computer interaction sensors track motion of body parts and determine joint centres and orientations of skeletal parts accessible through manufacturer software application program interfaces (APIs). The proposed software layer defines sensor plugins to integrate these software APIs to provide a unified view of sensors and tracking data to a game layer. Information of joint centres and orientations are extracted from native APIs and transferred to the game layer over UDP. The game layer further provides standard scripts and actors/game components to represent motion of body parts.
Game design methodology
Along with the software framework, a game design cycle is proposed. In this development cycle, a target therapy is selected and interviews with therapy experts are conducted to determine desired movements and expected clinical outcomes. Based on desired movements, possible input devices are selected and games fulfilling therapy requirements are designed. In addition to the visual components, data logging, difficulty and automatic adaptation to different sensors (if suitable) are considered. The game is subsequently refined with expert feedback.
Results
The proposed software framework was implemented in C# and sensor plugins for Kinect 2.0 and Rayzor Hydra were developed to interface with the corresponding manufacturer software development kit (SDK). Performance of UDP transport with the custom data transport protocol showed no limitations with respect to network transport and game interaction. Unity3D was selected as game engine and game layer components were implemented as prefabs. Standard visual components for limb segments for arm and hands were created, together with a range-of-motion assessment prefab.
Currently, the proposed game development cycle is applied to the area of upper arm exercises after shoulder injury. It is expected that a demonstration game for this therapy will be available in the coming weeks.
Conclusions
We expect that the proposed framework and game development cycle will simplify interdisciplinary therapy game development by providing a unified platform for team members to contribute to and routinely bring state-of-the-art game technology to therapeutic games.
Correspondence
Correspondence:
Yves Pauchard
Institute of Applied Information Technology, School of Engineering
Zurich University of Applied Sciences
Steinberggasse 13, Postfach, CH-8401 Winterthur
yves.pauchard[at]zhaw.ch
References
1 Cooke EV1, Mares K, Clark A, Tallis RC, Pomeroy VM. The effects of increased dose of exercise-based therapies to enhance motor recovery after stroke: a systematic review and meta-analysis. BMC Med. 2010;8:60.
2 Brander V, Stulberg SD. Rehabilitation after hip- and knee-joint replacement. An experience- and evidence-based approach to care. Am J Phys Med Rehabil. 2006;85(11 Suppl):S98–118.6.
3 Saposnik G, Teasell R, Mamdani M, Hall J, McIlroy W, Cheung D, et. al. Effectiveness of virtual reality using Wii gaming technology in stroke rehabilitation: a pilot randomized clinical trial and proof of principle. Stroke. 2010;41(7):1477–84.
4 Webster D, Celik. O.Systematic review of Kinect applications in elderly care and stroke rehabilitation. J Neuroeng Rehabil. 2014;11:108.
5 Burdea GC1, Cioi D, Kale A, Janes WE, Ross SA, Engsberg JR. Robotics and gaming to improve ankle strength, motor control, and function in children with cerebral palsy – a case study series. IEEE Trans Neural Syst Rehabil Eng. 2013 Mar;21(2):165–73.
6 YouGrabber YouRehab Ltd., Schlieren, Switzerland.
7 Valedo, Hocoma AG, Volketswil, Switzerland.
Copyright
Published under the copyright license
“Attribution – Non-Commercial – NoDerivatives 4.0”.
No commercial reuse without permission.
See: emh.ch/en/emh/rights-and-licences/
