Multimodality with HTML5

  • John Mannard

Student thesis: Master typesMaster in Computer science


Multimodality is a way of improving human-machine interaction. Studies have shown that it could enhance user productivity and working memory and that humans have a natural way of communicating that is multimodal. Since Bolt’s “put-that-there” experiment in 1979, several multimodal frameworks and findings have emerged but none seem to be designed for WEB applications. Considering the predominance of WEB applications and the advances in WEB technology following the arrival of HTML5, multimodality support seems to be the next natural step for WEB applications. This thesis lists a set of existing technologies fit for multimodality, introduces a new WEB multimodal framework based on these technologies and finally illustrates its usage through a set of examples.
Date of Award19 Jun 2015
Original languageEnglish
Awarding Institution
  • University of Namur
SupervisorBruno Dumas (Supervisor)


  • multimodal
  • WEB
  • EMMA
  • multimodal framework
  • put that there
  • speech
  • gesture
  • lattices
  • HTML5
  • canvas
  • WebRTC
  • WEB video
  • WEB audio
  • ink

Cite this