Modelling fusion of modalities in multimodal interactive systems with MMMM

Bruno Dumas, Jonathan Pirau, Denis Lalanne

Research output: Contribution to conferencePaper

Abstract

Several models and design spaces have been defined and are regularly used to describe how modalities can be fused together in an interactive multimodal system. However, models such as CASE, the CARE properties or TYCOON have all been defined more than two decades ago. In this paper, we start with a critical review of these models, which notably highlighted a confusion between how the user and the system side of a multimodal system were described. Based on this critical review, we define MMMM v1, an improved model for the description of multimodal fusion in interactive systems targeting completeness. A first user evaluation comparing the models revealed that MMMM v1 was indeed complete, but at the cost of user friendliness. Based on the results of this first evaluation, an improved version of MMMM, called MMMM v2 was defined. A second user evaluation highlighted that this model achieved a good balance between complexity, consistency and completeness compared to the state of the art.
Original languageEnglish
Pages288-296
Number of pages9
DOIs
Publication statusPublished - 13 Nov 2017

Fingerprint

Fusion reactions

Keywords

  • Care
  • Case
  • MMMM
  • Modelling
  • Multimodal interfaces
  • Tools for design

Cite this

@conference{bdf32f51302a4ede8f345028f06972dc,
title = "Modelling fusion of modalities in multimodal interactive systems with MMMM",
abstract = "Several models and design spaces have been defined and are regularly used to describe how modalities can be fused together in an interactive multimodal system. However, models such as CASE, the CARE properties or TYCOON have all been defined more than two decades ago. In this paper, we start with a critical review of these models, which notably highlighted a confusion between how the user and the system side of a multimodal system were described. Based on this critical review, we define MMMM v1, an improved model for the description of multimodal fusion in interactive systems targeting completeness. A first user evaluation comparing the models revealed that MMMM v1 was indeed complete, but at the cost of user friendliness. Based on the results of this first evaluation, an improved version of MMMM, called MMMM v2 was defined. A second user evaluation highlighted that this model achieved a good balance between complexity, consistency and completeness compared to the state of the art.",
keywords = "Care, Case, MMMM, Modelling, Multimodal interfaces, Tools for design",
author = "Bruno Dumas and Jonathan Pirau and Denis Lalanne",
year = "2017",
month = "11",
day = "13",
doi = "10.1145/3136755.3136768",
language = "English",
pages = "288--296",

}

Modelling fusion of modalities in multimodal interactive systems with MMMM. / Dumas, Bruno; Pirau, Jonathan; Lalanne, Denis.

2017. 288-296.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Modelling fusion of modalities in multimodal interactive systems with MMMM

AU - Dumas, Bruno

AU - Pirau, Jonathan

AU - Lalanne, Denis

PY - 2017/11/13

Y1 - 2017/11/13

N2 - Several models and design spaces have been defined and are regularly used to describe how modalities can be fused together in an interactive multimodal system. However, models such as CASE, the CARE properties or TYCOON have all been defined more than two decades ago. In this paper, we start with a critical review of these models, which notably highlighted a confusion between how the user and the system side of a multimodal system were described. Based on this critical review, we define MMMM v1, an improved model for the description of multimodal fusion in interactive systems targeting completeness. A first user evaluation comparing the models revealed that MMMM v1 was indeed complete, but at the cost of user friendliness. Based on the results of this first evaluation, an improved version of MMMM, called MMMM v2 was defined. A second user evaluation highlighted that this model achieved a good balance between complexity, consistency and completeness compared to the state of the art.

AB - Several models and design spaces have been defined and are regularly used to describe how modalities can be fused together in an interactive multimodal system. However, models such as CASE, the CARE properties or TYCOON have all been defined more than two decades ago. In this paper, we start with a critical review of these models, which notably highlighted a confusion between how the user and the system side of a multimodal system were described. Based on this critical review, we define MMMM v1, an improved model for the description of multimodal fusion in interactive systems targeting completeness. A first user evaluation comparing the models revealed that MMMM v1 was indeed complete, but at the cost of user friendliness. Based on the results of this first evaluation, an improved version of MMMM, called MMMM v2 was defined. A second user evaluation highlighted that this model achieved a good balance between complexity, consistency and completeness compared to the state of the art.

KW - Care

KW - Case

KW - MMMM

KW - Modelling

KW - Multimodal interfaces

KW - Tools for design

UR - http://www.scopus.com/inward/record.url?scp=85046649911&partnerID=8YFLogxK

U2 - 10.1145/3136755.3136768

DO - 10.1145/3136755.3136768

M3 - Paper

SP - 288

EP - 296

ER -