Designing Multimodal Mobile interaction for a Text Messaging application for Visually impaired Users

Carlos Duarte, Simon Desart, David Costa, Bruno Dumas

Research output: Contribution to journalArticle

3 Downloads (Pure)

Abstract

While mobile devices have experienced important accessibility advances in the past years, people with visual impairments still face important barriers, especially in specific contexts when both their hands are not free to hold the mobile device, like when walking outside. By resorting to a multimodal combination of body based gestures and voice, we aim to achieve full hands and vision free interaction with mobile devices. In this article, we describe this vision and present the design of a prototype, inspired by that vision, of a text messaging application. The article also presents a user study where the suitability of the proposed approach was assessed, and a performance comparison between our prototype and existing SMS applications was conducted. Study participants received positively the prototype, which also supported better performance in tasks that involved text editing.
Original languageEnglish
Article number26
Number of pages11
JournalFrontiers in ICT
Volume4
DOIs
Publication statusPublished - 6 Dec 2017

Fingerprint

Text messaging
Mobile devices

Keywords

  • non-visual interaction
  • body-based interaction
  • gestures
  • voice
  • multimodal interaction
  • accessibility

Cite this

@article{cd5c0deb4aea4a54ae595127276de5f9,
title = "Designing Multimodal Mobile interaction for a Text Messaging application for Visually impaired Users",
abstract = "While mobile devices have experienced important accessibility advances in the past years, people with visual impairments still face important barriers, especially in specific contexts when both their hands are not free to hold the mobile device, like when walking outside. By resorting to a multimodal combination of body based gestures and voice, we aim to achieve full hands and vision free interaction with mobile devices. In this article, we describe this vision and present the design of a prototype, inspired by that vision, of a text messaging application. The article also presents a user study where the suitability of the proposed approach was assessed, and a performance comparison between our prototype and existing SMS applications was conducted. Study participants received positively the prototype, which also supported better performance in tasks that involved text editing.",
keywords = "non-visual interaction, body-based interaction, gestures, voice, multimodal interaction, accessibility",
author = "Carlos Duarte and Simon Desart and David Costa and Bruno Dumas",
year = "2017",
month = "12",
day = "6",
doi = "10.3389/fict.2017.00026",
language = "English",
volume = "4",
journal = "Frontiers in ICT",
issn = "2297-198X",
publisher = "Frontiers Research Foundation",

}

Designing Multimodal Mobile interaction for a Text Messaging application for Visually impaired Users. / Duarte, Carlos; Desart, Simon; Costa, David; Dumas, Bruno.

In: Frontiers in ICT, Vol. 4, 26, 06.12.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Designing Multimodal Mobile interaction for a Text Messaging application for Visually impaired Users

AU - Duarte, Carlos

AU - Desart, Simon

AU - Costa, David

AU - Dumas, Bruno

PY - 2017/12/6

Y1 - 2017/12/6

N2 - While mobile devices have experienced important accessibility advances in the past years, people with visual impairments still face important barriers, especially in specific contexts when both their hands are not free to hold the mobile device, like when walking outside. By resorting to a multimodal combination of body based gestures and voice, we aim to achieve full hands and vision free interaction with mobile devices. In this article, we describe this vision and present the design of a prototype, inspired by that vision, of a text messaging application. The article also presents a user study where the suitability of the proposed approach was assessed, and a performance comparison between our prototype and existing SMS applications was conducted. Study participants received positively the prototype, which also supported better performance in tasks that involved text editing.

AB - While mobile devices have experienced important accessibility advances in the past years, people with visual impairments still face important barriers, especially in specific contexts when both their hands are not free to hold the mobile device, like when walking outside. By resorting to a multimodal combination of body based gestures and voice, we aim to achieve full hands and vision free interaction with mobile devices. In this article, we describe this vision and present the design of a prototype, inspired by that vision, of a text messaging application. The article also presents a user study where the suitability of the proposed approach was assessed, and a performance comparison between our prototype and existing SMS applications was conducted. Study participants received positively the prototype, which also supported better performance in tasks that involved text editing.

KW - non-visual interaction

KW - body-based interaction

KW - gestures

KW - voice

KW - multimodal interaction

KW - accessibility

U2 - 10.3389/fict.2017.00026

DO - 10.3389/fict.2017.00026

M3 - Article

VL - 4

JO - Frontiers in ICT

JF - Frontiers in ICT

SN - 2297-198X

M1 - 26

ER -