Aline Frederico

University of Cambridge

Multimodal transcription as a method for understanding the dynamics of children’s meaning-making with digital narratives

The contemporary easiness for recording young children’s meaning-making, often in naturally-occurring contexts, comes with the challenge of making sense of this rich set of data in ways that accordingly capture and represent the dynamism of the multimodal configuration of these processes. Multimodal transcription has emerged as a significant method of analysis of video data, which can assume an important epistemological function in the research process (Bezemer, 2014; Cowan, 2014). Multimodal transcription can generate insights on the role of the individual modes as well as the intermodal relationships that occur in the iterative process of meaning-making (Norris, 2004). The dynamism of literacy and meaning-making was highlighted by Potter and McDougall (2017), so one significant challenge of multimodal transcription is to represent such dynamism and the changes through time of the multimodal configuration of the phenomenon being investigated.

In this presentation, I will show and discuss different forms of multimodal transcription and their role as methodological tools for making sense of video-data of preschool children reading literary apps with a parent. Literary apps are multimodal, interactive narratives for children that present a complex multimodal configuration. Such configuration is dynamic and changes through time, sometimes automatically, but often as controlled by the reader through interaction. Children also dynamically orchestrate their multimodal meaning-making when reading these texts. Multimodal transcription, therefore, was an essential method for knowing the data and knowing through the data and various formats of transcriptions were developed for understanding the flow of meaning that emerges in the relationship between reader and semiotic artefact.


Bezemer, J. (2014). Multimodal transcription: A case study. In S. Norris & C. D. Maier (Eds.), Interactions, Images and Texts: A Reader in Multimodality (pp. 155–170). Boston; Berlin: De Gruyter Mouton.

Cowan, K. (2014). Multimodal transcription of video: examining interaction in Early Years classrooms. Classroom Discourse, 5(1), 6–21.

Norris, S. (2004). Analyzing multimodal interaction: a methodological framework. New York: Routledge.

Potter, J., & McDougall, J. (2017). Digital media, culture and education: theorising third space literacies. London: Palgrave Macmillan.



Aline Frederico (@aline_frederico) is a PhD candidate at the Faculty of Education of the University of Cambridge, UK. Her current research investigates the dynamics of meaning in children’s encounters with literary apps, with a special focus on embodiment and agency in digital reading. Aline also has a MA in Children’s Literature from the University of British Columbia, Canada, and studied Communication and Publishing at the University of São Paulo, Brazil. She has worked in publishing for over a decade as a book designer, editor and translator of publications for children in print and digital formats. Aline has published journal articles in children’s literature and is currently working on the manuscript for the book Digital reading and embodied meaning-making: Young children’s transactions with literary apps, accepted for publication by Bloomsbury (2019). For further information check

Signe Kjær Jensen

Linnaeus University, Sweden

Exploring children’s understanding and interpretation of music in animated film: a multimodal framework

This paper will present a specific problematic of a broader PhD project on children’s reception of music and characters in animated features which, along with textual analyses, aims to gather and analyze an empirical ‘child perspective’ on selected films. In order to do so, video-recorded screenings and interviews have been carried out with small groups of children aged 7 and 11 years old respectively. In order to fully understand children’s reactions and responses to film, a multimodal approach needs to be taken when transcribing and analyzing these kinds of video-recorded data; a multimodal approach which seeks to, as a minimum, account for the children’s use of facial expressions and body language as well as their use of the verbal mode. This focus on gesture might seem particularly important when working with relatively young children and with the content of non-verbal modes such as music, which tends to appeal to people in a highly embodied and intuitive way often escaping clear verbal descriptions. At this preliminary stage of the project, the data seems to suggest a very high dependence for the children on using semiotic resources outside of the verbal mode, e.g. by singing, humming, tapping rhythms, dancing, and imitating the playing of instruments, in order to elicit experiences with music for which a suitable vocabulary might be out of reach. In this paper, I want to present a model for a multimodal transcription of interviews based on the possibilities afforded by the software program Multimodal Analysis Video, developed by Kay O’Halloran and her team. In doing so, I will open up a discussion of how the multimodal meaning-making practices of children can be captured, analyzed and understood in academic research.



Signe Jensen is a PhD student at the Centre for Intermedial and Multimodal Studies at Linnaeus University in Sweden. She has a background in Musicology from Aarhus University in Denmark, and her research interests centre on music and sound as parts of intermedial and multimodal media constellations. In her ongoing PhD project, she focuses on music in children’s animated features, exploring the musical potential for meaning through a qualitative audience study. The project aims to analyze a selected range of popular animation features from a ‘textual’ perspective as well as through children’s multimodal reactions and verbalizations of their experiences of the films.

Ben Burbank

 Oxford Brookes University, UK

Multi-modal research tools that enable and inform a posthuman analysis of young children’s museum experiences.

This paper begins by outlining methodological insights derived from an ethnographic study designed to explore young children’s (4/5 years) perspectives of their everyday visit experiences within three contrasting museums.

The data collection was participatory (N=21) and multi-modal. The children used hand-held cameras during their visits. Interviews were audio-recorded immediately after their visits using the children’s photographs as prompts. The visits were videoed using chest-mounted GoPros worn by the children during their time in the museums. Children in the correct age bracket were recruited purposively on the door and no contact was made prior to their arrival.

The GoPro video, although not without limitations, provides inti{Barad, 2007, Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning}mate insights into young children’s visits, including privileged access to self-talk, movement, perspectives, social interactions, and intra-actions (Barad, 2007) with the material world, all of which contribute to shaping children’s museum experiences.

The utility of the video captured by the GoPro is particularly apparent when analysed in the context of post-visit interview data. Although the photo-elicitation interviews do shed light on some important details about the children’s experiences, the video footage reveals a more nuanced and rich texture to the children’s time in the museums’ spaces. In addition, it allows fly-on-the-wall access to the intricacies of the children’s photography. The video also highlights how easy it is to misinterpret children when relying on their verbal recollections in an interview situation.

Using these data, the paper critiques the use of a posthuman lens to interpret children’s unfolding and emerging multimodal communicative practices within museum spaces. This is in response to a feeling experienced when viewing these data, echoed by Hackett and Somerville (2017), that a focus on verbal representations provides a partial and skewed understanding of the children’s intra-actions during their visits.



Barad, K. (2007) Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning. duke university Press.

Hackett, A. and Somerville, M. (2017) ‘Posthuman literacies: young children moving in time, place and more-than-human worlds’, Journal of Early Childhood Literacy.



Ben Burbank has spent his working life in education (in the UK and abroad) before beginning a PhD in the School of Education at Oxford Brookes University in 2015. His research seeks to foreground young children’s perspectives of their unique and ephemeral museum visits. He holds a Masters in the Psychology of Education from the University of Bristol, a PGCE from the University of Oxford, and a BSc from the University of Southampton. If not working on his thesis, (or visiting museums!), he is most often found looking after his two young children, on a squash court, or kayaking on the Thames.

Xi Chen

Macau University of Science and Technology, Macau SAR

Translating the Chinese Classic Mulan into Contemporary Children’s Picturebooks: A Multimodal Perspective

Mulan is a Chinese legendary heroine who impersonates a man to takes her father’s place in the army. “The Ballad of Mulan” dates back to the Northern Dynasties (386-581 C.E.) and gradually becomes a significant part of Chinese classical literature. The Chinese American writer, Kingston, introduced Mulan to Western readers in 1976. Since the 1990s, a number of adaptations of the ballad in children’s picturebooks have been published in America, and Disney’s animated films Mulan (1998) and Mulan II (2005) made Mulan a heroine in the West.

This paper investigates different translations of Mulan in children’s picturebooks via a multimodal approach, with a special focus on the cultural transplantation of Mulan’s images in the book covers. Based on the theories of multimodality (Kress and van Leeuwen, 1996, 2006; O’Toole, 1994; Painter, Martin and Unsworth, 2013), a framework is proposed for the visual analysis of images in picturebooks. Five bilingual picturebooks are selected as the data: The Legend of Mulan: A Heroine of Ancient China (1992), China’s Bravest Girl: The Legend of Hua Mu Lan (1993), The Ballad of Mulan (1998), Song of Mulan (2010) and Mulan (2012). It first makes a textual analysis to explore how Mulan’s legend is built or rebuilt in the five picturebooks, and then conducts a visual analysis on the five book covers to examine how Mulan’s images are presented or represented in these contemporary children’s picturebooks. It is expected that this research can shed light on the translation of picturebooks via a multimodal approach.



Xi Chen is an assistant professor of University International College at Macau University of Science and Technology. She has gained her PhD in English Linguistics at the University of Macau. Her research interests include multimodal discourse analysis, translation studies and intercultural studies. Her recent publication is “Representing cultures through language and image: a multimodal approach to translations of the Chinese classic Mulan” in Perspectives: Studies in Translatology.

Nathaly Gonzalez-Acevedo

Universitat Autònoma de Barcelona, Spain

Rethinking Dynamic Preschoolers’ Learning: A view into Autonomous and Collaborative Technology Supported Tasks/Working Spaces 

This talk will present Preschoolers as dynamic and agentic learners. It is through the exploration of 5-6-year-olds’ interactions during an autonomous and collaborative task, supported by Beebots and iPads, that evidence of their agentivity is made visible. A social semiotic multimodal micro analysis of the preschoolers’ interaction makes relevant on the one hand that a view into modes in interaction allows evidence to be accounted for, and on the other hand that preschoolers are agentic and dynamic autonomous learners. The micro analysis of video data offers relevant and fine-grained information about the rich classroom context and the highly dynamic and vibrant interaction among preschoolers and makes visible subtle transformative engagement in interaction that remains unseen if not micro-analyzed. In this talk, learning is understood as a transformative engagement in interaction. Consequently, autonomous and collaborative technology-enhanced task designs are presented as tasks that allow preschoolers’ agency to arise by providing spaces in which preschoolers´ orientation, towards collaboration and autonomous task accomplishment, stimulates preschoolers’ orchestration of modes to manage own and others’ knowledge through transformative engagement. Thus, presenting such designs as rich and dynamic learning spaces that trigger meaningful learning as well as social and management skills in preschoolers. Hence, providing a context in which to rethink preschooler’s autonomous learning as powerful and resourceful, at the same time as suggesting that acknowledging adult power relations in the classroom is key. It is suggested that by recognizing children’s agentivity in autonomous learning situations and the power relation that an adult exerts, technology supported tasks can be designed to offer meaningful working spaces for transformative engagement to take place in autonomous and collaborative group work.



Nathaly Gonzalez-Acevedo is interested in very young learners’ agency and in the use of technology in the teaching and learning of EFL.  Her research focuses, in the early years, on collaborative and autonomous task designs and working spaces supported by technology and the language learning triggers and transformative engagement that can arouse in such spaces.  She is interested in social semiotic multimodal analysis as a lens to approach data.  Nathaly is a full time preschool teacher and a part-time assistant teacher at the Universitat Autònoma de Barcelona.  She is currently a PhD candidate at the Universitat Autònoma de Barcelona where she has recently won a YERUN Research Mobility Award.


Research Gate:

Kate Cowan

UCL Institute of Education

Multimodal Perspectives on Play in LEGO House

Young children’s everyday lives are increasingly permeated by an array of digital technologies that are rapidly changing their experiences of play and the forms in which they make meaning. Children’s playworlds are therefore a complex interweaving of modes, with the border areas between the physical and digital becoming increasingly blurred. As a result, contemporary play often moves across boundaries of space and time in ways that were not possible in the pre-digital era (Marsh, Plowman, Yamada-Rice, Bishop, & Scott, 2016). Despite suggestions that new technologies might enhance children’s play in new ways, the complexity of play in such hybrid spaces has been largely under-examined (Marsh & Yamada-Rice, 2016).

LEGO House resists easy definition, combining elements of a museum, gallery, studio and playground, highlighting the close connections between activities such as playing, tinkering, designing, making and learning. Opened in Denmark in 2017 with the slogan ‘Home of the Brick’, LEGO House embodies the company’s learning-through-play ethos and is distinctive in its use of digital technologies such as cameras, sensors, scanners, projectors and programmable robots alongside traditional LEGO bricks. LEGO House therefore provides a rich context for exploring the liminal border-areas where physical and digital play are increasingly mixed.

A multimodal perspective offers a balanced, evaluative approach to researching such hybrid play by supporting detailed insights into the design and use of toys and spaces, both physical and digital. This presentation will share a multimodal perspective on LEGO House by analysing the design of its play experiences. Using the concept of ‘affordance’ (Kress, 2005), the examples will demonstrate ways in which LEGO House invites combination and movement between physical and digital forms, highlighting both the gains and losses that such transduction entails. The findings suggest that spaces playfully combining physical and digital dimensions can offer new opportunities for meaning-making, and add to a growing body of work recognising that deeply significant learning can happen in informal, playful contexts. 


Kress, G. (2005). Gains and losses: New forms of texts, knowledge, and learning. Computers and Composition, 22(1), 5–22.

Marsh, J., Plowman, L., Yamada-Rice, D., Bishop, J., & Scott, F. (2016). Digital play: a new classification. Early Years, 36(3), 242–253.

Marsh, Jackie, & Yamada-Rice, D. (2016). Bringing Pudsey to life: Young children’s use of augmented reality apps. In N. Kucirkova & G. Falloon (Eds.), Apps, Technology and Young Learners. London: Routledge.



Kate Cowan is a Research Associate at UCL Institute of Education, London, interested in early childhood education, play, multimodality and digital technologies. Kate is currently researching children’s archives, spaces and technologies of play through the Playing the Archive project, and is researching teacher observation and documentation of play in classrooms through a project funded by The Froebel Trust. Kate’s doctoral research was part of MODE, which developed multimodal methodologies for researching digital data, including the ethics of video-based research with young children. Before joining UCL IOE, Kate worked as a nursery teacher and she remains committed to connecting research and practice.

More information:

Twitter: @katecowan


About Odense

Odense is situated on the island of Funen, right in the heart of Denmark, and is the country’s third largest city with 200,000 inhabitants. The city is one of Denmark’s oldest, granted town rights in the year 988, and was originally named Odins Vé, as a place of worship for the Nordic god Odin. It was once the site of a Viking ring fortress on the banks of the Odense River. Due to this rich history, it has for centuries been a favorite destination for visitors. Today, it is first and foremost known as the hometown of the celebrated author Hans Christian Andersen.

For more than a 100 years children of all ages around the world have enjoyed his fairytales like ”The Emperor’s New Clothes”, ”The Little Mermaid”, ”The Nightingale”, ”The Snow Queen”, ”The Ugly Duckling”, ”Thumbelina” and many more. His spirit lives on in Odense where fairytales come to life as you follow H.C. Andersen’s footsteps around the city. This inspiring city invites guests to wander its cobblestone streets with its many shops and rows of cafés. Odense is easy to reach, easy to navigate, and easy to feel at home in. Everything is within comfortable walking distance, creating an intimate infrastructure and feel around your meetings.

Funen is known as the garden of Denmark due to its long agricultural history. This is reflected in Odense’s wide selection of excellent restaurants, ranging from new nordic cuisine with local and seasonal ingredients to international kitchens of the highest standard. The city of Odense is a mix of new environmental-friendly city development and old historic sites, making the best of both worlds.

An ambitious urban development plan is currently reshaping major parts of the city, which are being connected by an environment-friendly tramway. Through this transformation we further consolidate Odense’s position as a centre of innovation, knowledge and education, where strong clusters in the fields of robotics, drones, health technology and IT are booming. Within the next few years the strong profile of Odense as a city for education and science will be further reaffirmed through the establishment of a brand-new university hospital. The university hospital will be linked to the University of Southern Denmark and a new science park.

Read the Guardian report on Odense as one of the most livable cities in Europe here.