Giovanni Parodi & Tomás Moreno-de León
Pontificia Universidad Católica de Valparaíso | Valparaíso, Chile
Integrating multiple multimodal professional written static texts: reading purposes in an eye-tracking study with Economics word-graph texts
How is information integrated from multiple disciplinary multimodal genres, how does the presence/absence of a specific reading purpose (‘reading to integrate’) affect the processing of static economics texts composed of words and graphs? How do multimodal corpus studies nurture research focused on the cognitive processing of specialized professional genres? These questions guide research that addresses the study of the reading of multiple multimodal static texts (Balslev, Jarodzka, Holmqvist, de Grave, Muijtjens, Eika, … Scherpbier, 2012; Jarodzka, Scheiter, Gerjets & van Gog, 2010) as a mechanism of accessing and constructing disciplinary knowledge (e.g., International Monetary Fund Annual Report and Monthly Report on Monetary and Financial Statistics). Specifically, this research aimed to determine the effect of the presence/absence of a specific reading objective on the online processing of two textual segments (two particular rhetorical moves) belonging to two multimodal genres of Economics (Parodi, Julio & Vásquez-Rocca, 2015; Parodi, Julio & Recio, 2018). We were interested in exploring the way in which ocular behavior can display evidence of the processes of integration of multiple multisemiotic sources (2 word-graph texts) when faced with a specific reading task. In this eye-tracking experiment (N = 63 Economics professionals), we tested the effect of integrating multiple multimodal texts from two genres (e.g., words-graphs interactions), when confronted with the presence/absence of a specific reading purpose. We found different reading routes of Economics professionals in integrating multimodal texts, particularly connected to integrative transitions between one text and the other and also from one semiotic system to the other. This suggests that reading and integrating verbal and graph systems from different genres is affected by the presence of a specific reading task.
Balslev, T., Jarodzka, H., Holmqvist, K., de Grave, W., Muijtjens, A., Eika, B., … Scherpbier, A. (2012). Visual expertise in paediatric neurology. European Journal of Paediatric Neurology, 16(2), 161–166. doi:10.1016/j.ejpn.2011.07.004
Jarodzka, H., Scheiter, K., Gerjets, P. & van Gog, T. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction, 20(2), 146– 154. doi:10.1016/j.learninstruc.2009.02.019
Kaakinen, J. K., Hyönä, J. & Keenan, J. M. (2002). Perspective Effects on Online Text Processing. Discourse Processes, 33(2), 159–173. doi:10.1207/s15326950dp3302_03
Parodi, G., Julio, C. & Vásquez-Rocca, L. (2015). Los géneros del Corpus PUCV-UCSC 2013 del discurso académico de la economía: el caso del Informe de Política Monetaria. Revista ALED, 15(3), 179-200.
Parodi, G., Julio, C. & Recio, I. (2018). When words and graphs move the eyes: the processing of multimodal causal relations. Journal of Eye Movement Research, 11(1), 118. doi: 10.16910/jemr.11.1.5
Giovanni Parodi, PhD., is full professor and researcher at Pontificia Universidad Católica de Valparaíso, Chile. The central focus of his research is to advance and to deepen knowledge of psycholinguistic processes involved in the comprehension of written multisemiotic discourse genres and in the construction of disciplinary knowledge from multiple sources.
Tomás Moreno-de León is currently pursuing doctoral studies in linguistics at Pontificia Universidad Catolica de Valparaíso, Chile. Since 2017, he has been participating in several research projects focusing in discourse genres and multimodality in professional contexts. He has also contributed as coauthor in a number of papers on topics such as the processing of multimodal written texts with the use of eye-tracking.