Identifying the Function of Hand Gestures from their Form in Political Speech
Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review
Standard
Identifying the Function of Hand Gestures from their Form in Political Speech. / Navarretta, Costanza.
Accentuated Innovations in Cognitive Info-Communication . ed. / Ryszard Klempous; Jan Nikodem; Péter Zoltán Baranyi. Springer, 2022. p. 203-216 (Topics in Intelligent Engineering and Informatics, Vol. 16).Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - CHAP
T1 - Identifying the Function of Hand Gestures from their Form in Political Speech
AU - Navarretta, Costanza
PY - 2022
Y1 - 2022
N2 - This paper addresses the automatic classification of the semiotic type of hand gestures produced by six English speaking politicians in different contexts from the annotations of the gesture form and shape. It builds upon and extends to more data the work proposed in [24]. Gestures contribute to the successful delivery of a message in face-to-face communication by reinforcing what is conveyed by speech or adding new information to it. Gestures are multi-functional and can have different meanings depending on the context. The identification of the semiotic type of gestures is the first step towards their automatic interpretation. Moreover, exploiting the relation between gesture form and function, described in the literature, on many types of data, contributes to the automatic generation of gestures e.g. in infocommunication systems. In the present work we annotated the semiotic types of hand gestures produced by Boris Johnson in two question/answer sessions at the British House of Common and added it to the annotations of the gestures of five other politicians in different contexts. We trained various classifiers to identify the semiotic type of the hand gestures. The F1 score obtained by the best performing algorithm on the classification of four semiotic types is 0.68. This result confirms on a larger data set the results obtained in the preceding pilot study indicating that it is possible to identify the semiotic types of hand gestures from their coarse-grained form features in nearly one third of the cases.
AB - This paper addresses the automatic classification of the semiotic type of hand gestures produced by six English speaking politicians in different contexts from the annotations of the gesture form and shape. It builds upon and extends to more data the work proposed in [24]. Gestures contribute to the successful delivery of a message in face-to-face communication by reinforcing what is conveyed by speech or adding new information to it. Gestures are multi-functional and can have different meanings depending on the context. The identification of the semiotic type of gestures is the first step towards their automatic interpretation. Moreover, exploiting the relation between gesture form and function, described in the literature, on many types of data, contributes to the automatic generation of gestures e.g. in infocommunication systems. In the present work we annotated the semiotic types of hand gestures produced by Boris Johnson in two question/answer sessions at the British House of Common and added it to the annotations of the gestures of five other politicians in different contexts. We trained various classifiers to identify the semiotic type of the hand gestures. The F1 score obtained by the best performing algorithm on the classification of four semiotic types is 0.68. This result confirms on a larger data set the results obtained in the preceding pilot study indicating that it is possible to identify the semiotic types of hand gestures from their coarse-grained form features in nearly one third of the cases.
U2 - 10.1007/978-3-031-10956-0_10
DO - 10.1007/978-3-031-10956-0_10
M3 - Book chapter
SN - 978-3-031-10955-3
T3 - Topics in Intelligent Engineering and Informatics
SP - 203
EP - 216
BT - Accentuated Innovations in Cognitive Info-Communication
A2 - Klempous, Ryszard
A2 - Nikodem, Jan
A2 - Zoltán Baranyi, Péter
PB - Springer
ER -
ID: 322283097