University of Münster | Münster, Germany
Multimodality as Challenge for the Quantitative Analysis of Media Content
Most media content is multimodal. The interplay of different modalities is crucial in terms of production, content and effects of mass media. The social science oriented branch of communication science ignored this fact for a long time and focused primarily on the analysis of verbal communication. Accordingly, there are only a few researchers from the field of communication science who researched multimodality in depth. However, as a result of digitalization and ongoing media development, the interest in multimodality is growing, especially in the field of visual communication. Particularly from the perspective of framing theory, multimodality has recently come to the fore.
For the analysis of smaller corpora, the discipline applies successfully methods of multimodality research from other disciplines (e.g., semiotics, linguistics, etc.). However, many questions of communication science require a quantitative and/or hypothesis-testing approach. For example,
questions of frequency or comparisons over time, between cultures or media services require a larger number of cases to identify statistically significant differences. For this kind of questions, quantitative (respectively standardized) content analysis has been established in communication science. The challenge is to reconcile the typical requirements of quantitative content analysis – such as intersubjective reproducibility and intercoder reliability – with the process of multimodal meaning making. With regard to media content, the interplay of text (written and spoken) and images (moving and still) is of particular interest. This paper presents the research design of a multimodal framing study on inclusion in education (author’s dissertation project that will be completed in 2019). Using a mainly standardized content analysis, 720 articles and 695 images from online journalistic media were analyzed. The aim was to show how image and text together create multimodal media frames and how these frames manifest themselves as patterns in media coverage. The content analysis was carried out on the basis of four levels of analysis (cf. figure 1): First, the images were analyzed separately (level 1). Subsequently, both the images in context with the captions (level 2) and the texts of the articles (level 3) were coded in a standardized way. Finally, these levels of analysis were brought together in a multimodal analysis (level 4). This combination was carried out using both qualitative interpretation and statistical methods (cluster analysis).
Based on this example, the paper discusses methodological challenges and problems in carrying out multimodal quantitative content analysis. Questions that arise in this context are, for instance:
– How far is it possible to capture the context-dependent meaning of multimodal media content through a standardized content analysis?
– Can segmentation into different levels of analysis be used to adequately describe the process of meaning making of multimodal media content?
– Which methods can be used to recombine the different levels of analysis to identify the overall meaning of media content?
– How big is the effort of such a coding process and which case numbers can be handled with it?
– Is it possible to apply typical evaluation criteria of quantitative research (e.g., reliability) in multimodal content analyses?
– What are the challenges in transferring this approach to other modalities and more complex media content (e.g., moving images and audio)?
BATEMAN, John A. (2014): Text and Image: A Critical Introduction to the Visual/Verbal Divide. Oxon/New York: Routledge.
BOCK, Annekatrin/ISERMANN, Holger/KNIEPER, Thomas (2011): Quantitativ Content Analysis of the Visual. In: MARGOLIS, Eric/PAUWELS, Luc (Hrsg.): The Sage Handbook of Visual Research Methods. London et al.: Sage, S. 265–282.
HERBERS, Martin R./VOLPERS, Anna-Maria (2013): Visuelles Framing – Anforderungen an die empirische Forschung und methodologische Konsequenzen. In: GEISE, Stephanie/LOBINGER, Katharina (Hrsg.): Visual Framing. Perspektiven und Herausforderungen der Visuellen Kommunikationsforschung. Köln: Halem, S. 77–94.
LOBINGER, Katharina/GEISE, Stephanie (2013): ›Does the frame remain the same?‹ Visual Framing und Multimodalität als theoretische und methodische Herausforderungen der Visuellen Kommunikationsforschung. In: GEISE, Stephanie/LOBINGER, Katharina (Hrsg.): Visual Framing. Perspektiven und Herausforderungen der Visuellen Kommunikationsforschung. Köln: Halem, S. 332–357.
WESSLER, Hartmut/WOZNIAK, Antal/HOFER, Lutz/LÜCK, Julia (2016): Global Multimodal News Frames on Climate Change: A Comparison of Five Democracies around the World. In: The International Journal of Press/Politics, 21. Jg., Nr. 4, S. 423–445.
Anna-Maria Volpers, M.A., is a research assistant at the University of Münster (Department of Communication). Her dissertation project focuses on multimodal framing and image-textrelations. She studied Communication Science, Art History, and Public Law at the University of Münster. Anna-Maria was a research assistant in a project funded by the German Federal Ministry of Education and Research on the role of journalistic media in German research policy (2010–2013) and in the area of applied market and opinion research (2014–2016). Her fields of research are visual and multimodal communication, science communication, and journalism studies.