Engineering User Interfaces with Beat Gestures

Beat gestures are biphasic up-and-down or back-and-forth movements of hand(s) that are associated with a specific meaning, such as in speech, or without, such as in rhythmic commands. Incorporating beat gesture recognition into user interface engineering involves dynamic recognition of hand pose, identification of movement direction, and calculation of beat number and frequency. We demonstrate a game that uses beat gestures for musical rhythm learning. We aim to understand the impact of real-time embodiment and guidance visualizations synthesizing user hands and gestures and considering the distance between virtual and real worlds.