Bremen University | Bremen, Germany
An Empirical Exploration of Gameplay as Affective Semiotic Operations
This presentation illustrates a mixed-method empirical approach for theorizing dynamics among nonconscious affects, meaning-making, and behavior in the experience of digital gameplay. Digital gameplay unfolds as emergent realizations across technical materiality, bodies, and worlds (Keogh 2018; Dunér and Sonesson 2016). It is crucial to (re)consider affects in relation to meaning-making and core game studies debates (Hayles 2016; Anable 2018), anchored by a critical methodology grounded in fine-grained, empiricism-driven multimodal analyses (Bateman, Wildfeuer, and Hippala 2017). My work coheres body-text-assemblages that intersect corpus-based multimodal analyses of digital games, players’ biophysical measurements, and player-produced texts regarding gameplay experience, to integrate qualitative, quantitative, and embodied data to understand meaning-making in digital games. From an ongoing research that examines engagement of mainstream and experimental games with such phenomena as grief, war, mental illness, and human-robot relationships, the presentation discusses on the micro-level correlations among multimodal configurations and affective experiences at narrative-crucial gameplay moments; and how such patterns dialectically inform gameplay-contextualizing mechanisms and design strategies on the macro level. It draws on selected sequences from two games: Nevermind (Flying Mollusk 2016), a thriller that features motifs of psychological trauma and biofeedback sensitivity to foster player mindfulness and mood management; and Detroit: Become Human (Cage and Wasselin 2018), which facilitates inquiries into changing conceptualizations of subjectivity, agency, and intimacies; concerning sociopolitical issues such as gender, race, and robot ethics. Discussion explores ways that games from the collated corpora shape gameplay behaviors through complex affects (e.g. boredom, alienation, unease), multimodal materiality, and game design strategies, including subverting player agency, oscillating between player witnessing and complicity, ludonarrative dissonance, and incorporating unreliable narrators. Attuned to gamic materiality as simultaneously mediating and experiential, the presentation addresses key issues identified by both the conference and multimodality as a discipline. These range from tackling challenges in conceptualizing and conducting multimodal corpora research, in light of evolving semiotic phenomena and practices; to enhancing endeavors in the humanities and social sciences. Ultimately, my inquiry into digital games aims to contribute to critical awareness of multimodality; social impact game designs that foster empathy and mental health; and further empirical research on gameplay, affect, and meaning.
Anable, A. (2018). Playing with Feelings: Video Games and Affect. University of Minnesota Press.
Bateman, J. A.; Wildfeuer, J.; and Hiippala, T. (2017). Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Berlin and Boston: Walter de Gruyter.
Dunér, D. and Sonesson, G. (2016). Human Lifeworlds: The Cognitive Semiotics of Cultural Revolution. Frankfurt am Main: Peter Lang.
Hayles, N. K. (2016). The Cognitive Nonconscious: Enlarging the Mind of the Humanities. Critical Inquiry, 42(4), 783-808.
Keogh, B. (2018). A Play of Bodies: How We Perceive Videogames. Cambridge, MA: The MIT Press.
Ludography: Cage, D. (Director) and Wasselin, S. (Designer). (2018). Detroit: Become Human. Quantic Dream.
Flying Mullosk. (2016). Nevermind. Steam.
Carman Ng is Postdoctoral Fellow at the University of Bremen, with research sojourn experiences in the United States (Fulbright) and Germany (Erasmus Mundus, DAAD). Her current research intersects game studies, transmedia, and affective sciences, with the aim to theorize multimodal semiotics for social impact game designs that engage with empathy and mental health. She is interested in examining multimodal semiotics and ideologies of popular media, including digital games, anime, comics, and graphic novels; as well as experimental forms of media art. Outside the academia, she has participated in the Hong Kong community theater as a performer, writer, and techie.