Multimodal turn-taking: motivations, methodological challenges, and novel approaches

Katharina Rohlfing; Giuseppe Leonardi; Iris Nomikou; Joanna Rączaszek-Leonardi; Eyke Hüllermeier

doi:10.1109/TCDS.2019.2892991

Multimodal turn-taking: motivations, methodological challenges, and novel approaches

Katharina Rohlfing, Giuseppe Leonardi, Iris Nomikou, Joanna Rączaszek-Leonardi, Eyke Hüllermeier

Research output: Contribution to journal › Article › peer-review

955 Downloads (Pure)

Abstract

In this article, we note that despite being a multimodal phenomenon, turn-taking has still been investigated mostly as being unimodal. Based on theoretical positions emphasizing that communication is organized jointly by interaction partners, we identify the challenge of assessing human sequential behavior that is (a) spread across different modalities and (b) co-constructed with a partner. By analyzing a corpus of mother-child dyads with Cross Recurrence Quantification Analysis and Frequent Pattern Mining, we offer novel steps toward understanding multimodal turn-taking.

Original language	English
Journal	IEEE Transactions on Cognitive and Developmental Systems
Early online date	15 Jan 2019
DOIs	https://doi.org/10.1109/TCDS.2019.2892991
Publication status	Early online - 15 Jan 2019

Access to Document

10.1109/TCDS.2019.2892991

MultimodalTurntaking_accepted
© © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript (Post-print), 1.01 MB

EASE: Early semantic development: Linking language development to emerging participation in social events (EASE)
Rohlfing, K. (PI), Rączaszek-Leonardi, J. (CoI) & Nomikou, I. (Team Member)
1/02/16 → 31/01/19
Project: Research

Cite this

@article{9ee8455db63840d584c29f54004f51d3,

title = "Multimodal turn-taking: motivations, methodological challenges, and novel approaches",

abstract = "In this article, we note that despite being a multimodal phenomenon, turn-taking has still been investigated mostly as being unimodal. Based on theoretical positions emphasizing that communication is organized jointly by interaction partners, we identify the challenge of assessing human sequential behavior that is (a) spread across different modalities and (b) co-constructed with a partner. By analyzing a corpus of mother-child dyads with Cross Recurrence Quantification Analysis and Frequent Pattern Mining, we offer novel steps toward understanding multimodal turn-taking.",

author = "Katharina Rohlfing and Giuseppe Leonardi and Iris Nomikou and Joanna R{\c a}czaszek-Leonardi and Eyke H{\"u}llermeier",

year = "2019",

month = jan,

day = "15",

doi = "10.1109/TCDS.2019.2892991",

language = "English",

journal = "IEEE Transactions on Cognitive and Developmental Systems",

issn = "2379-8920",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Multimodal turn-taking: motivations, methodological challenges, and novel approaches

AU - Rohlfing, Katharina

AU - Leonardi, Giuseppe

AU - Nomikou, Iris

AU - Rączaszek-Leonardi, Joanna

AU - Hüllermeier, Eyke

PY - 2019/1/15

Y1 - 2019/1/15

N2 - In this article, we note that despite being a multimodal phenomenon, turn-taking has still been investigated mostly as being unimodal. Based on theoretical positions emphasizing that communication is organized jointly by interaction partners, we identify the challenge of assessing human sequential behavior that is (a) spread across different modalities and (b) co-constructed with a partner. By analyzing a corpus of mother-child dyads with Cross Recurrence Quantification Analysis and Frequent Pattern Mining, we offer novel steps toward understanding multimodal turn-taking.

AB - In this article, we note that despite being a multimodal phenomenon, turn-taking has still been investigated mostly as being unimodal. Based on theoretical positions emphasizing that communication is organized jointly by interaction partners, we identify the challenge of assessing human sequential behavior that is (a) spread across different modalities and (b) co-constructed with a partner. By analyzing a corpus of mother-child dyads with Cross Recurrence Quantification Analysis and Frequent Pattern Mining, we offer novel steps toward understanding multimodal turn-taking.

U2 - 10.1109/TCDS.2019.2892991

DO - 10.1109/TCDS.2019.2892991

M3 - Article

SN - 2379-8920

JO - IEEE Transactions on Cognitive and Developmental Systems

JF - IEEE Transactions on Cognitive and Developmental Systems

ER -

Multimodal turn-taking: motivations, methodological challenges, and novel approaches

Abstract

Access to Document

Fingerprint

Projects

EASE: Early semantic development: Linking language development to emerging participation in social events (EASE)

Cite this