VIDEO GAMES, READING , AND TRANSMEDIAL COMPREHENSION.

Games represent a high interest accessible medium to build comprehension, and in using games we can continue to engage in topics that are complex, provocative and motivating, and not often found in texts designed to be simplified for the sake of decoding. Games will also help to get these students to reconnect with reading and learning, and create a basis for developing and using comprehension strategies. With this in mind, this knowledge and experience of theory can provide an opportunity for educators to bootstrap traditional print-based literacy and engage students in comprehension development.

Brock Dubbels

The University of Minnesota

The Center for Cognitive Sciences

305 Elliott Hall

Minneapolis, MN 55408

(612) 747-0346

(612) 626-7253

Dubbe003@umn.edu

Abstract: In this qualitative study, literacy practices of “struggling” seventh and eighth graders were recorded on videotape as they engaged in both traditional and new literacies practices in an after school video games club. These recordings were analyzed in the context of building comprehension skills with video games. The students struggled with reading and are characterized as unmotivated and disengaged by the school, which may be at the root of their inability to use comprehension strategies. Playing video games is viewed here as a literate practice, and was seen to be more engaging than traditional activities (such as reading school text, writing journals, etc.). The conclusion of this observation makes connections to current research in comprehension and provides a basis for teachers to use games to develop comprehension and learning.

Key Terms: situation, event indexing model, causal integration model, ludic, interaction, comprehension, knowledge act, decoding, agency, engagement, identity, self-monitoring, metacognition, transmedial, walk-through, level-up, button-mashing

This selected excerpt comes from:

Dubbels, B.R. (2008) Video games, reading, and transmedial comprehension. In R. E. Ferdig (Ed.),Reference. Information ScienceHandbook of research on effective electronic gaming in education.

Why games?

Regardless of whether we are talking about comprehension in novels, articles, sculpture, or dance, comprehension comes from dialogue and interaction, where the learner gathers information and must act upon it in some way—feedback and reflection are essential.

Feedback and interaction were central to the work of Lev Vygotsky, as depicted here in Figure 1. (Tharp & Gallimore, 1988, p. 35).

In this model called the Zone of Proximal Development, the external dialogue created by interaction in stage 1, eventually becomes internalized, and through this internalization individuals go through a process where they learn to self-monitor, learn to observe, learn to practice, learn to receive feedback, and ultimately they learn to refine the process of comprehension through questioning and verifying what they know in what can be called a knowledge act.

Vygotsky (1978) maintained that children imitate what they see adults do, and this gradually develops into the ability to do certain tasks without help or assistance. In teaching something new, we need to start with where the student is at with their ability to solve problems, and then under the guidance of a more experienced problem solver, their ability is extended—this is called the Zone of Proximal Development. The mentor’s role, according to Herb Simon is to align a problem so that the solution can become transparent, and this can then be extended and deepened.

Games can provide this interaction, provide relevant feedback, help to create and align a problem, and demand mastery in the same way a competent instructor can scaffold a learner through the Zone of Proximal Development. In a classroom situation, where a teacher has thirty or more students, and fifty-five minutes to work with them–an interactive game can be quite effective, as it is often unlikely that teacher will be able to work one-on-one with each student in a meaningful way every day. But it is possible that games can provide the kind of accessibility, interaction, and instruction to develop comprehension that young people who struggle to decode might benefit from, but more importantly, games may give incentive to reengage in developing essential reading skills.

What makes the idea of a game so compelling as a method for developing comprehension is the amount of regular feedback and the necessity of mastery of complex situations that might offer a variety of solution sets from combinations and permutations of different tools, rules, cooperation, and knowledge of the narrative goals and character arcs. This complex interaction scaffolds the player through difficulties until they are able to perform the task with knowledge through the developing awareness that failure and recovery from it (learning) are part of developing knowledge, strategy, and competence.

What Teachers Need to Know

The big idea here is that games are built upon many of the same comprehension elements that are incorporated in reading printed text. There is a tradition of consideration of video games as new narratives; Laurel (1993), Murray (1997), Aarseth (1997), Frasca (2003) and Crawford (2005) agree that there are narrative elements, but that games are also designed to be interactive in the way that they tell stories. As Frasca (2003) describes, the aspects of simulation and problem solving components of the game are also very compelling, and are what he called the ludic aspects of the game: the elements that instigate play—see also Salen & Zimmerman (2004).

These elements create the feedback and demand knowledge acts, where the player must gather information, decide what is useful, and use it to act to get through a puzzle. This process of game play demands that the player acquire and act upon the information they gather. This view of games, play, and learning presents a highly flexible and powerful form of comprehension, and it is generally instinctual, but often needs mentoring, modeling, and practice to develop.

The table above to the right shows a direct and close relation to situation model research– typical literary categories often used in reading comprehension assessments; I am familiar and have worked on the Minnesota MCA2 for four years– and state and national standards. This alignment of the dimensions of a Situation Model from the Event Indexing Model allows for close proximity and correlation between what are often considered traditional and customary literary terms and an empirically validated comprehension models such as from Walter Kintsch (1968) and participants in the Discourse Processing tradition.

Although these connections have not been made yet on the level of the state and nationwide assessments, as I believe they should, this study assumed that connecting empirical comprehension research from the Discourse Processing tradition would be useful in connecting the culturally mediated competencies of the state assessments (that are often only validated through inter-rater reliability: selected teachers sitting around a table rating and compromising about what is what) and language arts standards to an empirically validated theory of reading comprehension. This enables a more regimented look at comprehension, narrative, discourse, and connection to problem solving and decision making as made popular by Kahneman and Tversky (2001).

In addition. this connection makes the case for The Compensation Model (Stanovich, 2000), and also connects to more recent work on the Indexical Hypothesis and Embodiment Theory (Borghi 2005; Glenburg, 1997; Zwann 2003; Barsalou 1999).

Learning and Play are Inseparable

With this connection to discourse processing and comprehension, play might be seen as an evolutionary adaptation (Sutton-Smith, 1997) useful in transferring culture, knowledge, and developing proficiency in tool use, as well as the creation of knowledge, culture, and tools, (Mumford, 1945). Above all, play represents an opportunity for exploration and inquiry in a context where the cost of failure is the opportunity to learn from a mistake. In play, knowledge is gathered and acted upon, a very powerful process to develop experience and comprehension. Play is visible evidence of comprehension. It is an expression of and toward comprehension. It is a knowledge act.

Play is one of the building blocks of comprehension, and games are the way that we ground cognition and pass on cultural knowledge and skills sets through off-loading cognitive work into the environment, thus making he environment part of the cognitive system through place-holders and computational devides like toys and tools–Vygotsky (176) called these pivots.

With play, children and adults seek and explore through observation, manipulation, and imitation. When we play, we take our prior knowledge and attempt to create order and demonstrate competence. With toys, children are interacting with a representation of something that adults use.

My own son likes to play with cars. He makes engine sounds and drives them around with screeching tires. He also likes to take on roles and imitate with toys that are representations of the tools we use for productivity. Above we saw him in his own office with the same tools mom and dad use. What he was doing was building a repertoire of how things work and what they are used for in his memory.

From an evolutionary perspective, he is using his imagination to prepare for a time when he will drive and also to share, communicate, and produce culture through knowledge of cultural artifacts like cars and computers. This process of visualization, belief systems and cultural roles are important aspects of learning. Williamson Shaffer (2006) in his description of Epistemic Games provides depth and detail on how these elements work in games (and this is exemplified and contextualized later in Figure 2).

Games are a structured form of play, and what teachers need to understand is that play is learning, and by structuring play, we can create a higher degree of engagement and effort. Games allow for the kind of learning that encourages mastery, builds resilience and tenacity, innovation and creativity, as well as practice solving problems. Play creates comprehension through knowledge acts, whether the learner is performing or investigating.

Games, Play, and Learning are Inseparable.

As children develop and grow, their play becomes more complex, and we begin to see imaginative play, with language and visualization, use of complex ideas, actions, tools, and language, and interaction and shared play with others.

If you look at a game, it generally is composed of a number of elements that are structured to create the interaction and experience that children and adults enact in play (such as imagination, role playing and identities, rules, branching and choice, as well as incorporating the use of probability and chance.

Games are tools for learning, but what is important for educators to know is how to use games that structure and extend play and create reflection. Often tools are useful in helping individuals to learn complex problem solving and pass along cultural knowledge, but what we often fail to account for is that tools are not as powerful until used in an activity with intention. Kids may have grown up in a digital age, but this does not make

them digital natives. They may just be strangers in a strange land that they want to know more about—we can provide the passport.

We need to consider play as the foundation in an approach to using video games, and we need to recognize that games may represent an activity that is complex and relevant enough to engage students in learning and developing comprehension skills.

But games are not enough. In education, we often talk about the tools, because the tools are often the measure of what we take to be progress in education. Often the tools we use such as tests and software are thought to be capable of definitive assessment. And if the software helps improve test performance, it must be improving learning over all. This is not always the case, and more often than not, it is the way the tool is used that is important.

We are foolish to think we can just give out the tools and easily replicate this (Cuban, 1986),. It is important to understand that tools are only as powerful as the way they are employed. The central role of the tool may be one of the major obstacles educators will need to overcome in the classroom if schools and teachers are going to remain relevant for preparation of young people for the new global labor force. Games should not be seen as a panacea, but they may be used as powerful tools.

Play and the way that we learn to use tools should be the focus. Play changes the discourse around learning. It implies making mistakes, trying new things, and creativity, and innovation— traits necessary for knowledge workers. It also takes the pressure off children who are already under tremendous pressure to perform in school settings that regularly insist on going faster and covering more ground because we are all so far behind.

Play can bring some more humanity into our classroom teaching, and it is also a proven method for learning and developing cultural competency—and games represent more formalized and structured approaches to this cultural learning often found in informal learning environments often known as the wilds away from school.

The take home message

Play is how we share and transfer physical and intellectual tools to our young and other community members. Johann Huizinga (1938), the author of Homo Ludens, stated that play is the basis of culture. According to Lewis Mumford (1954), it was imitation, role play, the creation of miniature environments and the symbolic fields of play where every function of life was modeled as a game to develop competency and advance what was known and will be known.

Perhaps play and representation are the factory of our conceptual abilities, and if play involves the creation of abstractions and models of the world, then sharing play necessitates complex communication as well as a means for innovation and production. This makes it the perfect method for developing comprehension.

A Model for Comprehension

In terms of understanding the elements of comprehension, the tradition of Discourse Processing provides ample opportunity to look at comprehension as transmedial. Kintsch (1988, 1998) in presenting the Construction Integration Model suggested that there were three levels of representation of text.

Two additional levels of discourse processing were later added by Zwann, Langston, & Graesser, (1995), and account for literary elements often associated with genre and the author’s purpose and style that are used for top-down processing. These levels are:

• The text genre level, which accounts for the kinds of information presented, how the information is presented, and the ways in which the information is to be interpreted,
• And the communication level, which involves the reader constructing a representation of the writer of the text.

The Event Indexing Model, which includes all five boxes from Figure 3, provides insight on events, or situations as portrayed in a variety of media, which are the made up of the interrelated elements. This level can be very useful for teachers to know how to develop comprehension in their students and create transfer between different media experiences; and it is the basis for the transmedial nature of comprehension.

If students have struggled to decode, games and film can provide a more accessible entry point at the situation model level, (level 3 experiences), where they can interact with complex texts and narratives without the cognitive load of decoding text and making meaning from it (bottom up processing).

With games, much of the action, the context, and the actors are depicted ( they are shown in pictures), and also described through the action and narrative text in the print and audio of character dialog and narration. This depiction with description allows for clarification for how we should view the context and understand the situation with the directing of our attention (Kress, 2003) and may lessen demand on the propositional level from the text to make a mental image representing what is being described in the text.

This contextualization of images with printed traditional text informs and focuses the player, and offers the experience and information to embody the reading and comprehension experimentally. Thus, the learner makes a more robust model of the situation to develop and build the representation as the narrative evolves. This might allow for the student to draw from prior experience and use top-down processes to make meaning of the events and comprehend without the taxation on processing from decoding, therefore utilizing and further developing comprehension.

But how does this help decoding?

When a reader has well-developed comprehension skills, they can recruit prior knowledge to bootstrap lower level processes (Stanovich, 2000) and this is an important idea for making a case for using more accessible texts that are relevant and interesting to the learner. Once again, the reader can use higher-level process in order to support lower level process (Stanovich, 2000).

What makes teaching with games to develop comprehension meaningful are the second and the third level, called the Propositional and Situation Levels. The Propositional Level is where we decode communications into the Situation Level. The Situation level is where the reader represents many of the elements and descriptions that go into making a mental model of a situation. On this level, the learner constructs a mental representation of the situation experienced, remembered, read, perceived, or described in a real or imaginary world, (Zwann, Langston, & Graesser, 1995; Zwann & Radavansky, 1998).

When working to teach reading with this information, it is important to connect to prior knowledge and build and compare the new information to prior situation models, or prior experience. Consider a story board, or a comic strip where each scene is defined and then the next event is framed. Readers need to learn to create these frames when comprehending text. Each event in a text should then be integrated and developed as an evolution of ideas presented, as each scene builds with new information, the model is updated and expanded.

“If the event that is currently being processed overlaps with the

events in working memory on a particular dimension, then a link

between those events is established, then a link between those

events is stored in long-term memory. Overlap is determined based

on two events share an index, (i.e., a time, place, protagonist, cause,

or goal).”

(Goldman, Graesser, & van den Broek, 1999, p. 94).

Thus, we must build our understanding. The attributes of a situation model can be made much more robust and much more easily accessible when prior knowledge is recruited and connected into what we already know, and less effort will be made of finding the familiar.

Two types of prior knowledge that support this in the Event Indexing Model are:

·         General world knowledge (pan-situational knowledge about concept types, e.g.,
scripts, schemas, categories, etc.) 

·         Referent specific knowledge (pan-situational knowledge about specific entities). 

These two categories represent experience in the world and literary elements used in defining genre
and style as described from the Event Indexing Model.  The theory posits that if a reader has more
experience with the world that can be tapped into, and also knowledge and experience about the
structure of stories, they are more likely to have a deeper understanding of the passage. 

Genre Expectation

It is possible that when we are familiar with “once upon a time”, we will be familiar with the structure
and purpose of fairy tale stories. Thus we begin to expect certain events, characters, and settings that
we may be able to predict, and can therefore be surprised and entertained when the story has a twist,
or an element that was unexpected is presented.

Further, van den Broek et al (2001) reports that readers engage in a different style of reading according to genre expectations, and, that reading purpose will inform the notion that readers differentially allocate their processing resources according to their expectations about the genre of the text—so when readers have familiarity, they generally have a tactic or strategy for engaging with the text. It would be expected that a reader with little or no familiarity would use a button mashing style of trial and error when playing a game just as a reader would cast about for meaning when trying to comprehend a passage in a challenging text.

To sum this up, comprehenders parse clauses of texts into events on five situational dimensions: time, space, causation, motivation, and the protagonist. There are four central assumptions this model is built upon:

1. Events are the central units of situation models ( the event-centrality assumption)

2. Events can be linked on five dimensions

3. The events are either related or not related on a particular dimension

4. All dimensions are equal.

With this in mind, the Event Indexing Model is an equal opportunity model of comprehension. It works across media to describe the transformation of sensory information and how it is processed to some extent — and it is just as applicable with pictures as it is with text–it is the study of discourse and the conveyance of information. It is transmedial — or perhaps sub-medial: it is the process that turns that pattern to meaning — working as a form of categorical organization to go from high fidelity, fine-tooth detail, to more generalized levels of organization — chunking and schema building.

We create memory based upon pattern recognition, and perhaps print-text is not the only way to develop reading comprehension. In this sense, reading is thinking queued by text, and that the key to being a successful reader is not automaticity with decoding, but the ability to match new information with prior experience. The creation of organizational categories to chunk larger patterns of information and connect to prior experience helps the individual to process faster.  So a movie buff who understands story arc and plot may be better at predicting the story and have more cognitive resource to focus on the details and connect new information to existing memory and experience a mental simulation of what is coded in the text–moving back and forth between online (attention to the symbols)  and off-line meaning making(memory simulation of dimensions of the situation model).

When, for example, a learner is presented with

Once upon a time . . .

They often expect other narrative features like

Happily ever after.

These elements of story structure and narrative are called Story Grammars,

and these exist based upon our prior experience with text and our ability in
pattern recognition.If you think about a narrative as a labyrinth or maze

you may want to consider the view from the top to accelerate the journey

This is what Daedalus did for Icarus, in helping him to escape the labyrinth– and this use of to-sight may be useful through improving the knowledge of novice narrative comprehenders through teaching with more accessible and high-interest media which offers feedback, interaction, and multi-modal sensory input, as well as inference making, resource management, and strategy generation. In addition, games can provide the social capital that will reinforce engagement and quality time on task. In addition, games provide a much more complex narrative.

Just consider as a branching example a game as simple as Chutes and Ladders (Snakes and Ladders to UK folks):

The game movement can influence the narrative attributes and remove the barriers of decoding to the generation and development of literary elements and pattern recognition for cognitive chunking and schema building reflected in the categories listed in the Event Indexing Model. It was this exploration of theory that led to the study of struggling readers using video games as methods for observing upper levels in student comprehension and the ideal of using high motivation narratives such as video games like Metroid Prime to examine the comprehension abilities of students who had been removed from traditional classroom instruction for reading comprehension issues,who were self-reported gaming enthusiasts to see how they did with a complex game. In this sense, if a novel is a linear experience where the choices of context lead from one scene to the next based upon the author’s order of choice, a game can be offered based upon player choices, and may offer multiple possible arcs of experience, thus:

More to come . . .