shut up frog

Play has been removed from schools by non-educators

Brock Dubbels, Ph.D.

Dept. Psychology, Neuroscience, & Behaviour

McMaster University

It was not psychologists, educators, or child development researchers that removed play from schools. According to McCombs & Miller (2007), the emphasis on performance testing and standardization was led by a campaign of politicians and corporate interests to influence what happened in the classroom. With government reports such as Nation At Risk (1983), the National Governors Association (1989) worked to create Goals 2000 (1994) and called for greater levels of accountability for student achievement and rigorous academic standards. They called for more focus on standardized content, standardized content delivery, and standardized tests. This campaign to standardize schools worked to change classroom curriculum, but it contradicted and ignored 100 years of psychological research about human learning (McCombs & Miller, 2007).

The new standards and assessments became mandated performance indicators on how schools were evaluated. For a school to be rated as competent, their students had to meet federal and state performance guidelines, and school funding was tied to student performance on standardized assessments. This situation became so desperate for some schools, that entire school districts (superintendents, principals, and teachers) committed fraud by falsifying assessment data (Dayen, 2015).

Political reasons for standardization over play

Elected officials and journalists reported that American students had fallen behind other industrialized nations in math and science, and the proof was in American student performance on international testing tests called PISA and TIMMS. They warned that without improvements in student performance in math and science, the USA would no longer be competitive on the world stage (US Committee on Prospering in the Global Economy of the 21st Century, Science, & (US), 2007).

Reports such as these were political in nature. When American student scores are compared to students of the same income level, students in the United States did significantly better than all other countries:

For every administration of PISA and TIMSS, when controlling for

poverty, U.S. public school students are not only competitive, they

downright lead the world. Even at home nationally, when controlling

for poverty, public school students compete with private school

students in Lutheran, Catholic, and Christian schools when analyzing

NAEP data (Ravitch, 2013).

Poverty plays a central role in student performance. Schools serving lower-income students tend to be organized and operated differently than those serving more affluent students. Poverty is the most significant impact on academic performance. It does not matter if these schools are big or small, private, or religious. Poverty is the most significant predictor of poor academic performance (McNeil & Valenzuela, 2000; Rumberger & Palardy, 2005). Students in poverty often come to school without the social and economic benefits held by many middle-to-high SES students, such as access to books, food, parental support with schoolwork, and financial stability (Sirin, 2005).

In wealthy schools, students are more likely experience playful activities and learner centered pedagogy (Anyon, 1980). Schools that serve children in poverty, not only struggle the most, but are also often the first to get the standardized education, reduction in play, and elimination of electives such as music, arts, and training. We may be compounding the problem, rather than offering a solution by removing these things from children in poverty.

Children in poverty also experience greater exposure to threat and violence, which contributes to play deprivation. Play deprivation has arisen as a medical diagnosis. It means that children do not experience the essential cognitive, social, and affective benefits of learning through play (Milteer, Ginsburg, Health, & Mulligan, 2012). Play is an essential element of learning and development. Removing play in favor of standardization is a mistake.

Standardization is profit-centered, not student-centered

If anything was learned from the standardization campaign, it was that the creation of standards and content has proven to be very financially lucrative to testing companies, and very destructive for school districts (Dayen, 2015). These policies have led to change of control, where classrooms are now legislated through national education standards, and this legislation is often influenced, if not written by, lobbyists that work for the companies that profit from selling tests and curriculum, rather than the people who have experience working with children and child development research (Leistyna, 2007).

The shift to standardized assessment and curriculum has also led to instability. It is very profitable to have standards change. When standards change, schools are required to meet those new standards, and this is often accomplished by paying for new tests and new curriculum. State-based initiatives on Common Core—the standards and assessments—change every 4 years (Porter, McMaken, Hwang, & Yang, 2011). Each shift in standards constitutes a form of educational whack-a-mole, where districts are forced to purchase new curriculum, and states must create new assessments. This is a lucrative market, over $2 billion annually (Strauss, 2015).

To cultivate financial opportunity, educational publishers have been very involved in this process; Pearson Education, ETS (Educational Testing Service), Houghton Mifflin Harcourt, and McGraw-Hill collectively spent more than $20 million lobbying in states and on Capitol Hill from 2009 to 2014 (ibid). In many ways, standardization and accountability initiatives have exacerbated the “problems” they set out to solve, and instead, created a lucrative market for pre-packaged curriculum and tests, the deprofessionalization of teachers, and significant cost to American taxpayers.

Standardized methods of assessment often lack the long view, and do not pass the tests of time, retention, and adaptation. According to Atkinson & Mayo, (2010) focus on subject matter and facts only serve to limit student motivation, learning and choice, and reduce the potential for innovation. Additionally, high stakes tests, and the practice of evaluation during instruction is an unreliable index of whether the long-term changes, which constitute learning, have actually taken place (for review, read Soderstrom & Bjork, 2015).

Parents opt-out of standardization

Interestingly, many parents and stakeholders have begun to embrace the long view, and begun to doubt the value of testing; they have begun to “opt-out”, which is now called the “opt-out parents movement” (Layton, 2013). The opt-out movement indicates a trend towards more play-based and learner-centered practices, advocated for by the American Psychological Association (APA) (Alexander & Murphy, 1998; Barbara, 2004; Cornelius-White, 2007; McCombs, 2001; McCombs & Miller, 2007; Weimer, 2013).

The benefits of play

kids play as doctorPlay is not only an imaginative activity of amusement. Play and games serve important roles in cognitive, social, and affective development (Dubbels, 2014; Fisher, 1992; Frost, 1998; Garvey, 1990). In pre-industrial times, pastoral and foraging societies, children did not learn sequestered away from adult contexts (Thomas, 1964). Instead, children participated in playful variations of adult activities, where they could observe adults at work, and were able to imitate and emulate these activities through play without the danger of failure and consequence (Bock, 2005; Rogoff, 1994).

Rubin, Fein, and Vandenberg provided a thorough psychological overview of the early role of play in their chapter in volume four of the Manual of Child Psychology (1983). They observed that humans play longer relative to other mammals that play. Lancaster and Lancaster (1987) built upon this position and state that this extended period of play is essential for development. Bjorklund, (2006) expands upon this view, and states that humans play longer because they are adaptive organisms, and, that extended play is essential, allowing humans the skills and knowledge to become independent in complex environments.

When children engage in complex peer play, they exhibit greater gains in levels of symbolic functional and oral language production, as compared to if they are interacting with an adult (Pellegrini, 1983). Additionally, when a learner experiences learning through play, where they can experience and role-play adult work, they report the activities are more meaningful, and that the activity did not feel like learning (Dubbels, 2010). This aligns with Winkielman & Cacioppo, (2001), who found that when learning new information is experienced as easy, processing is experienced as pleasant and effective.

Learning generated in the context of play, especially social play, can lead to greater engagement, improved recall, comprehension, and be more innovative. Juveniles can observe behaviors and strategies performed by adults but then recombine elements of these behaviors in novel routines in play (Bateson, 2005; Bruner, 1972; Fagen, 1981; Sutton-Smith, 1966). For example, the levels of children’s symbolic functional and oral language production are more varied and complex in peer play, relative to when they are interacting with an adult (Pellegrini, 1983). More importantly, play is a low-cost and low-risk way to learn new behaviors and acquire new skills and knowledge (ibid). Conversely, one could suggest that a limitation of direction instruction, observation, and imitating adults is that this kind of instruction will only transmit existing practices.

Offering activities to children in a playful mood can increase a willingness to take direction, and on-task behavior (Moore, Underwood, & Rosenhan, 1973; Rosenhan, Underwood, & Moore, 1974; Underwood, Froming, & Moore, 1977). To create a more playful mood, participants engage in playful communication, with emphasis on reducing or eliminating all commands, questions, and criticisms.

Play acts as an important organizing principle during developmental growth (Brown, 1998). Play is not only an imaginative activity; play also allows children to imitate and emulate adult work activities without the danger of failure. Children role-play activities from the adult world, and learn to use the tools, rules, and language of adult work. Play is an important part of academic learning. When children play, they develop new strategies and behaviors with minimal costs (Bateson, 2005; Burghardt, 2005; Spinka, Newbury, and Bekoff, 2001).

joyUsing a playful approach in the classroom represents a fundamental change in assessment, offering a philosophy of playful and data-informed assessment, as compared to standardized, data-driven assessment. To be data-informed, assessments are used to guide the way, not to indicate that learning is accomplished. In play-based assessment, one can inform and improve student learning, increase motivation and engagement, and improve our school’s programs by learning from our challenges, progress, and performance.

 

A playful structuring of assessment allows one to integrate play, and utilize assessment as a form of instructional communication, reducing threat, and emphasizing play. The value of such an approach is that it provides support for a range of students, including specialized support to educationally disadvantaged populations, including economically disadvantaged students, English Language Learners, students with disabilities, and students who are at risk of not meeting state academic standards.

References

Alexander, P. A., & Murphy, P. K. (1998). The research base for APA’s learner-centered psychological principles. Retrieved from http://psycnet.apa.org/books/10258/001

Anyon, J. (1980). Social class and the hidden curriculum of work. Sociology of Education: Major Themes, 162, 1250.

Atkinson, R. D., & Mayo, M. J. (2010). Refueling the US innovation economy: Fresh approaches to science, technology, engineering and mathematics (STEM) education. The Information Technology & Innovation Foundation, Forthcoming. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1722822

Banks, J., Carson, J. S., Nelson, B. L., & Nicol, D. M. (2001). Verification and validation of simulation models. Discrete-Event System Simulation, 3rd Edition, Prentice-Hall, Upper Saddle River (NJ), 367–397.

Barbara, L. (2004). The learner-centered psychological principles: A framework for balancing academic achievement and social-emotional learning outcomes. Building Academic Success on Social and Emotional Learning: What Does the Research Say?, 23.

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31.

Brown, S. (1998). Play as an organizing principle: clinical evidence and personal observations. Animal Play: Evolutionary, Comparative, and Ecological Perspectives, 242–251.

Committee, R. A. the G. S. (2010). Rising above the gathering storm, revisited: Rapidly approaching Category 5. National Academies Press, Washington DC.

Cornelius-White, J. (2007). Learner-centered teacher-student relationships are effective: A meta-analysis. Review of Educational Research, 77(1), 113–143.

Danielson, L. M. (2009). Fostering reflection. Educational Leadership, 66(5).

Dayen, D. (2015, April 3). The Biggest Outrage in Atlanta’s Crazy Teacher Cheating Case. Retrieved April 4, 2015, from http://finance.yahoo.com/news/biggest-outrage-atlanta-crazy-teacher-091500508.html

Deno, S. L., & Marston, D. (2006). Curriculum-based measurement of oral reading: An indicator of growth in fluency. What Research Has to Say about Fluency Instruction, 179–203.

Dubbels. (2014). Play: A Framework for Design, Development, & Gamification. Retrieved from http://www.yorku.ca/intent/issue7/articles/pdfs/brockrdubbelsarticle.pdf

Dubbels, B. (2008). Rhythm & Flow: Putting words to music as performance reading with garage band. In Professionalism in Practice (Vol. 1). The University of Minnesota: Minneapolis Public Schools.

Dubbels, B. (2010). Engineering curriculum and 21st century learning–improving academic performance with play and game design. In Games in Engineering & Computer Science GECS. National Science Foundation, Arlington, VA: NSF Course, Curriculum, and Laboratory Instruction program under Award No. 0938176. Retrieved from http://gecs.tamu.edu/index.php

Dubbels, B. (2013). Gamification, Serious Games, Ludic Simulation, and other Contentious Categories. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 5(2).

Dweck, C. (2006). Mindset: The new psychology of success. Random House. Retrieved from http://books.google.ca/books?hl=en&lr=&id=fdjqz0TPL2wC&oi=fnd&pg=PA1&dq=Mindset:+The+New+Psychology+of+Success&ots=Bh76YKyGPD&sig=qkBjC2j6J9chNBClclBomJEANwI

Dweck, C. S. (2007). The perils and promises of praise. Retrieved from http://books.google.com/books?hl=en&lr=&id=QDwqACGdf0IC&oi=fnd&pg=PA57&dq=dweck+growth+mindset+academic+&ots=9rif4lroXa&sig=eOdyOEMPi3kt7oFFQh-ctyXZ2bA

Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the organization of behavior. Psychological Review, 66(3), 183.

Elliot, A. J., & Covington, M. V. (2001). Approach and avoidance motivation. Educational Psychology Review, 13(2), 73–92.

Elliot, A. J., Gable, S. L., & Mapes, R. R. (2006). Approach and avoidance motivation in the social domain. Personality and Social Psychology Bulletin, 32(3), 378–391.

Fisher, D., & Frey, N. (2012). Close reading in elementary schools. The Reading Teacher, 66(3), 179–188.

Fisher, E. P. (1992). The impact of play on development: A meta-analysis. Play & Culture, 5(2), 159–181.

Fisher, M. (2013, April 15). Map: How 35 countries compare on child poverty (the U.S. is ranked 34th). The Washington Post. Retrieved from http://www.washingtonpost.com/blogs/worldviews/wp/2013/04/15/map-how-35-countries-compare-on-child-poverty-the-u-s-is-ranked-34th/

Forehand, R., & Scarboro, M. E. (1975). An analysis of children’s oppositional behavior. Journal of Abnormal Child Psychology, 3(1), 27–31.

Forgas, J. P., Burnham, D. K., & Trimboli, C. (1988). Mood, memory, and social judgments in children. Journal of Personality and Social Psychology, 54(4), 697–703. http://doi.org/10.1037/0022-3514.54.4.697

Forster, M. (2009). Informative Assessment—understanding and guiding learning. Retrieved from http://research.acer.edu.au/research_conference/RC2009/17august/11/

Fox, E. (2008). Emotion Science. Retrieved from http://www.palgrave.com%2Fpage%2Fdetail%2Femotion-science-elaine-fox%2F%3FK%3D9780230005174

Frost, J. L. (1998). Neuroscience, Play, and Child Development. Retrieved from http://files.eric.ed.gov/fulltext/ED427845.pdf

Gabe, T. (2015). Poverty in the United States: 2013 (Congressional Research Service No. RL33069). Retrieved from https://www.fas.org/sgp/crs/misc/RL33069.pdf

Gable, P. A., & Poole, B. D. (2012). Time Flies When You’re Having Approach-Motivated Fun Effects of Motivational Intensity on Time Perception. Psychological Science, 23(8), 879–886.

Garvey, C. (1990). Play. Harvard Univ Pr. Retrieved from http://books.google.com/books?hl=en&lr=&id=d711jR0AqvIC&oi=fnd&pg=PR9&dq=garvey+play+1990&ots=YxJSWlZ7_K&sig=xCIVNP5RLJvPG2eGD_hU4esXoaM

Gavenda, V. (2005). GarageBand 2 for Mac OS X (Visual QuickStart Guide). Peachpit Press. Retrieved from http://dl.acm.org/citation.cfm?id=1197774

Ginsburg, K. R. (2007). The importance of play in promoting healthy child development and maintaining strong parent-child bonds. Pediatrics, 119(1), 182–191.

Gunnar, M., & Quevedo, K. (2007). The neurobiology of stress and development. Annu. Rev. Psychol., 58, 145–173.

Johnson, S. M., & Lobitz, G. K. (1974). The personal and marital adjustment of parents as related to observed child deviance and parenting behaviors. Journal of Abnormal Child Psychology, 2(3), 193–207.

Kindergarten. (2015, December 10). In Wikipedia, the free encyclopedia. Retrieved from https://en.wikipedia.org/w/index.php?title=Kindergarten&oldid=694674848

Kohl, H. (1992). I won’t learn from you! Thoughts on the role of assent in learning. Rethinking Schools, 7(1), 16–17.

Kohl, H. (1994). I won’t learn from you. Confronting Student Resistance in Our Classrooms. Teaching for Equity and Social Justice, 134–135.

Lay, K.-L., Waters, E., & Park, K. A. (1989). Maternal Responsiveness and Child Compliance: The Role of Mood as a Mediator. Child Development, 60(6), 1405–1411. http://doi.org/10.2307/1130930

Layton, L. (2013). Bush, Obama focus on standardized testing leads to “opt-out” parents” movement. The Washington Post. Retrieved from http://216.78.200.159/RandD/Washington%20Post/Focus%20on%20Testing%20Leads%20to%20%E2%80%98Opt-Out%E2%80%99%20Movement%20-%20Post.pdf

Leistyna, P. (2007). Corporate testing: Standards, profits, and the demise of the public sphere. Teacher Education Quarterly, 59–84.

Lupien, S. J., McEwen, B. S., Gunnar, M. R., & Heim, C. (2009). Effects of stress throughout the lifespan on the brain, behaviour and cognition. Nature Reviews Neuroscience, 10(6), 434–445.

McCombs, B. L. (2001). What do we know about learners and learning? The learner-centered framework: Bringing the educational system into balance. Educational Horizons, 182–193.

McCombs, B. L., & Miller, L. (2007). Learner-Centered Classroom Practices and Assessments: Maximizing Student Motivation, Learning, and Achievement. Corwin Press.

McNeil, L., & Valenzuela, A. (2000). The harmful impact of the TAAS system of testing in Texas: Beneath the accountability rhetoric. Retrieved from http://eric.ed.gov/?id=ED443872

Milteer, R. M., Ginsburg, K. R., Health, C. on C. and M. C. on P. A. of C. and F., & Mulligan, D. A. (2012). The Importance of Play in Promoting Healthy Child Development and Maintaining Strong Parent-Child Bond: Focus on Children in Poverty. Pediatrics, 129(1), e204–e213. http://doi.org/10.1542/peds.2011-2953

Moore, B. S., Underwood, B., & Rosenhan, D. L. (1973). Affect and altruism. Developmental Psychology, 8(1), 99.

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc. Retrieved from http://dl.acm.org/citation.cfm?id=1095592

Parpal, M., & Maccoby, E. E. (1985). Maternal responsiveness and subsequent child compliance. Child Development, 1326–1334.

Peed, S., Roberts, M., & Forehand, R. (1977). Evaluation of the Effectiveness of a Standardized Parent Training Program in Altering the Interaction of Mothers and their Noncompliant Children. Behavior Modification, 1(3), 323–350. http://doi.org/10.1177/014544557713003

Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common core standards the new US intended curriculum. Educational Researcher, 40(3), 103–116.

Ravitch, D. (2013, December 5). Daniel Wydo Disaggregates PISA Scores by Income. Retrieved from http://dianeravitch.net/2013/12/05/daniel-wydo-disaggregates-pisa-scores-by-income/

Recess (break). (2015, December 7). In Wikipedia, the free encyclopedia. Retrieved from https://en.wikipedia.org/w/index.php?title=Recess_(break)&oldid=694207866

Reddy, L. A., Files-Hall, T. M., & Schaefer, C. E. (2005). Announcing empirically based play interventions for children. Empirically Based Play Interventions for Children, 3–10.

Resnick, M., & Silverman, B. (2005). Some reflections on designing construction kits for kids. In Proceedings of the 2005 conference on Interaction design and children (pp. 117–122). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=1109556

Rosenhan, D. L., Underwood, B., & Moore, B. (1974). Affect moderates self-gratification and altruism. Journal of Personality and Social Psychology, 30(4), 546.

Rumberger, R., & Palardy, G. (2005). Does segregation still matter? The impact of student composition on academic achievement in high school. The Teachers College Record, 107(9), 1999–2045.

Sackett, A. M., Meyvis, T., Nelson, L. D., Converse, B. A., & Sackett, A. L. (2010). You’re having fun when time flies the hedonic consequences of subjective time progression. Psychological Science, 21(1), 111–117.

Sahlberg, P. (2007). Education policies for raising student learning: The Finnish approach. Journal of Education Policy, 22(2), 147–171.

Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers & Education, 46(3), 223–234.

Shaffer, D. W. (2006). How computer games help children learn. Macmillan.

Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453.

Soderstrom, N. C., & Bjork, R. A. (2015). Learning Versus Performance An Integrative Review. Perspectives on Psychological Science, 10(2), 176–199.

Strauss, V. (2015, March 30). Report: Big education firms spend millions lobbying for pro-testing policies. The Washington Post. Retrieved from http://www.washingtonpost.com/blogs/answer-sheet/wp/2015/03/30/report-big-education-firms-spend-millions-lobbying-for-pro-testing-policies/

Sutton-Smith, B. (2001). The ambiguity of play. Harvard Univ Pr. Retrieved from http://books.google.com/books?hl=en&lr=&id=AgA8q0TCKeIC&oi=fnd&pg=PR5&dq=sutton-smith&ots=Cna4A0r15X&sig=bjS1AtbvJP8cohAj52Oi2lZB1RE

Tennie, C., Call, J., & Tomasello, M. (2006). Push or pull: Imitation vs. emulation in great apes and human children. Ethology, 112(12), 1159–1169.

Underwood, B., Froming, W. J., & Moore, B. S. (1977). Mood, attention, and altruism: A search for mediating variables. Developmental Psychology, 13(5), 541.

(US Committee on Prospering in the Global Economy of the 21st Century, Science, & (US), P. P. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. National Academies Press. Retrieved from http://books.google.ca/books?hl=en&lr=&id=KoQf1m-s10UC&oi=fnd&pg=PR1&dq=Rising+Above+the+Gathering+Storm,+Energizing+and+Employing+America+for+a+Brighter+Economic+Future&ots=8otcXacPcm&sig=DepAfNSa-sXWLGNOD0F9nFlVZtQ

Weimer, M. (2013). Learner-Centered Teaching: Five Key Changes to Practice (2 edition). San Francisco: Jossey-Bass.

White, K. R. (1982). The relation between socioeconomic status and academic achievement. Psychological Bulletin, 91(3), 461.

Whiten, A., McGuigan, N., Marshall-Pescini, S., & Hopper, L. M. (2009). Emulation, imitation, over-imitation and the scope of culture for child and chimpanzee. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1528), 2417–2428.

Wiliam, D. (2007). Changing classroom practice. Educational Leadership, 65(4), 36.

Wiliam, D., & Thompson, M. (2007). Integrating assessment with learning: what will it take to make it work? Retrieved from http://eprints.ioe.ac.uk/1162/

Winkielman, P., & Cacioppo, J. T. (2001). Mind at ease puts a smile on the face: psychophysiological evidence that processing facilitation elicits positive affect. Journal of Personality and Social Psychology, 81(6), 989.

Yeager, D. S., & Dweck, C. S. (2012). Mindsets that promote resilience: When students believe that personal characteristics can be developed. Educational Psychologist, 47(4), 302–314.

 

boy_playing200CHA P T E R 4

Pedagogy and Play: Creating a Playful Curriculum for

Academic Achievement and Engaged Learning

Brock R. Dubbels, PhD.,  Dept. Psychology, Neuroscience, & Behavior, McMaster University, Hamilton,

ON, Canada, Dubbels@McMaster.ca

Key Summary Points

Using instructional techniques based upon play can improve achievement. Standardization has created more problems than it solved. Three case studies are presented as demonstrations of the framework.  When children engage in complex peer play, they exhibit greater gains in levels of symbolic functional and oral language production.

Key Terms

Play, Assessment, Learner Centered Practices, Instructional Communication, Curriculum, Cognitive, Affect, Classroom, Instructional Design, Learning

The Benefits of Play

Play is not only an imaginative activity of amusement. Play and games serve important roles in cognitive, social, and affective development (Dubbels, 2014; Fisher, 1992; Frost, 1998; Garvey, 1990). In pre-industrial times, pastoral and foraging societies, children did not learn sequestered away from adult contexts (Thomas, 1964). Instead, children participated in playful variations of adult activities, where they could observe adults at work, and were able to imitate and emulate these activities through play without the danger of failure and consequence (Bock, 2005; Rogoff, 1994). Rubin, Fein, and Vandenberg provided a thorough psychological overview of the early role of play in their chapter in volume four of the Manual of Child Psychology (1983). They observed that humans play longer relative to other mammals that play. Lancaster and Lancaster (1987) built upon this position and state that this extended period of play is essential for development. Bjorklund, (2006) expands upon this view, and states that humans play longer because they are adaptive organisms, and, that extended play is essential, allowing humans the skills and knowledge to become independent in complex environments. When children engage in complex peer play, they exhibit greater gains in levels of symbolic functional and oral language production, as compared to if they are interacting with an adult (Pellegrini, 1983). Additionally, when a learner experiences learning through play, where they can experience and roleplay adult work, they report the activities are more meaningful, and that the activity did not feel like learning (Dubbels, 2010). This aligns with Winkielman & Cacioppo, (2001), who found that when learning new information is experienced as easy, processing is experienced as pleasant and effective.

Read more of this article, download, feel good

 

What we cannot know or do individually, we may be capable of collectively.

My research examines the transformation of perceptual knowledge into conceptual knowledge. Conceptual knowledge can be viewed as crystallized, which means that it has become abstracted and is often symbolized in ways that do not make the associated meaning obvious. Crystallized knowledge is the outcome of fluid intelligence, or the ability to think logically and solve problems in novel situations, independent of acquired knowledge. I investigate how groups and objects may assist in crystallization of knowledge, or the construction of conceptual understanding.

I am currently approaching this problem from the perspective that cognition is externalized and extended through objects and relationships.  This view posits that skill, competence, knowledge are learned through interaction aided with objects imbued with collective knowledge.

Groups make specialized information available through objects and relationships so that individual members can coordinate their actions and do things that would be hard or impossible for them to enact individually.  To examine this, I use a socio-cognitive approach, which views cognition as distributed, where information processing is imbued in objects and communities and aids learners in problem solving.

This socio-cognitive approach is commonly associated with cognitive ethnography and the study of social networks. In particular, I have special interest in how play, games, modeling, and simulations can be used to enhance comprehension and problem solving through providing interactive learning. In my initial observational studies, I have found that games are structured forms of play, which work on a continuum of complexity:

  • Pretense, imagery and visualization of micro worlds
  • Tools, rules, and roles
  • Branching / probability

Games hold communal knowledge, which can be learned through game play. An example of this comes from the board game Ticket to Ride. In this strategy game players take on the role of a railroad tycoon in the early 1900′s. The goal is to build an empire that spans the United States while making shrewd moves that block your opponents from being able to complete their freight and passenger runs to various cities. Game play scaffolds the learner in the history and implications of early transportation through taking on the role of an entrepreneur and learning the context and process of building up a railroad empire. In the course of the game, concept are introduced, with language, and value systems based upon the problem space created by the game mechanics (artifacts, scoring, rules, and language). The game can be analyzed as a cultural artifact containing historical information; a vehicle for content delivery as a curriculum tool; as well as an intervention for studying player knowledge and decision-making.

I have observed that learners interact with games with growing complexity of the game as a system. As the player gains top sight, a view of the whole system, they play with greater awareness of the economy of resources, and in some cases an aesthetic of play. For beginning players, I have observed the following progression:

  1. Trial and error – forming a mental representation, or situation model of how the roles, rules, tools, and contexts work for problem solving.
  2. Tactical trials – a successful tactic is generated to solve problems using the tools, rules, roles, and contexts.  This tactic may be modified for use in a variety of ways as goals and context change in the game play.
  3. Strategies—the range of tactics of resulted in strategies that come from a theory of how the game works. This approach to problem solving indicates a growing awareness of systems knowledge, the purpose or criteria for winning, and is a step towards top sight. They understand that there are decision branches, and each decision branch comes with risk reward they can evaluate in the context of economizing resources.
  4. Layered strategies—the player is now making choices based upon managing resources because they are now economizing resources and playing for optimal success with a well-developed mental representation of the games criteria for winning, and how to have a high score rather than just finish.
  5. Aesthetic of play—the player understands the system and has learned to use and exploit ambiguities in the rules and environment to play with an aesthetic that sets the player apart from others. The game play is characterized with surprising solutions to the problem space.

For me, games are a structured form of play. As an example, a game may playfully represent an action with associated knowledge, such as becoming a railroad tycoon, driving a high performance racecar, or even raising a family. Games always involve contingent decision-making, forcing the players to learn and interact with cultural knowledge simulated in the game.

Games currently take a significant investment of time and effort to collectively construct. These objects follow in a history of collective construction by groups and communities. Consider the cartography and the creation of a map as an example of collective distributed knowledge imbued in an object.  According to Hutchins (1996),

“A navigation chart represents the accumulation of more observations than any one person could make in a lifetime. It is an artifact that embodies generations of experience and measurement. No navigator has ever had,nor will one ever have, all the knowledge that is in the chart.”

A single individual can use a map to navigate an area with competence, if not expertise. Observing an individual learning to use a map, or even construct one is instructive for learning about comprehension and decision-making. Interestingly, games provide structure to play, just as maps and media appliances provide structure to data to create information. Objects such as maps and games are examples of collective knowledge, and are what Vygotsky termed a pivot.

The term pivot was initially conceptualized in describing children’s play, particularly as a toy. A toy is a representation used in aiding knowledge construction in early childhood development. This is the transition where children may move from recognitive play to symbolic and imaginative play, i.e. the child may play with a phone the way it is supposed to be used to show they can use it (recognitive), and in symbolic or imaginative play, they may pretend a banana is the phone.

This is an important step since representation and abstraction are essential in learning language, especially print and alphabetical systems for reading and other discourse. In this sense, play provides a transitional stage in this direction whenever an object (for example a stick) becomes a pivot for severing meaning of horse from a real horse. The child cannot yet detach thought from object. (Vygotsky, 1976, p 97). For Vygotsky, play represented a transition in comprehension and problem solving –where the child moved from external processing — imagination in action —to internal processing — imagination as play without action.

In my own work, I have studied the play of school children and adults as learning activities. This research has informed my work in classroom instruction and game design. Learning activities can be structured as a game, extending the opportunity to learn content, and extend the context of the game into other aspects of the learner’s life, providing performance data and allowing for self-improvement with feedback, and data collection that is assessed, measured and evaluated for policy.

My research and publications have been informed by my work as a tenured teacher and software developer. A key feature of my work is the importance of designing for learning transfer and construct validity. When I design a learning environment, I do so with research in mind. Action research allows for reflection and analysis of what I created, what the learners experienced, and an opportunity to build theory. What is unique about what I do is the systems approach and the way I reverse engineer play as a deep and effective learning tool into transformative learning, where pleasurable activities can be counted as learning.

Although I have published using a wide variety of methodologies, cognitive ethnography is a methodology typically associated with distributed cognition, and examines how communities contain varying levels of competence and expertise, and how they may imbue that knowledge in objects. I have used it specifically on game and play analysis (Dubbels 2008, 2011). This involves observation and analysis of Space or Context—specifically conceptual space, physical space, and social space. The cognitive ethnographer transforms observational data and interpretation of space into meaningful representations so that cognitive properties of the system become visible (Hutchins, 2010; 1995). Cognitive ethnography seeks to understand cognitive process and context—examining them together, thus, eliminating the false dichotomy between psychology and anthropology. This can be very effective for building theories of learning while being accessible to educators.

My current interest is in the use of cognitive ethnographic methodology with traditional form serves as an opportunity to move between inductive and deductive inquiry and observation to build a Nomological network (Cronbach & Meehl, 1955) using measures and quantified observations with the Multiple Trait and Multiple Method Matrix Analysis (Campbell & Fiske, 1959) for construct validity (Cook & Campbell, 1979; Campbell & Stanley, 1966) especially in relation to comprehension and problem solving based upon the Event Indexing Model (Zwaan & Radvansky, 1998).

We distribute knowledge because it is impossible for a single human being, or even a group to have mastery of all knowledge and all skills (Levy, 1997). For this reason I study access and quality of collective group relations and objects and the resulting comprehension and problem solving. The use of these objects and relations can scaffold learners and inform our understanding of how perceptual knowledge is internalized and transformed into conceptual knowledge through learning and experience.

Specialties

Educational research in cognitive psychology, social learning. identity, curriculum and instruction, game design, theories of play and learning, assessment, instructional design, and technology innovation.

Additionally:

The convergence of media technologies now allow for collection, display, creation, and broadcast of information as narrative, image, and data. This convergence of function makes two ideas important in the study of learning:

  • The ability to create of media communication through narrative, image, and data analysis and information graphics is becoming more accessible to non-experts through media appliances such as phones, tablets, game consoles and personal computers.
    • These media appliances have taken very complex behaviors such as film production, which in the past required teams of people with special skill and knowledge, and have imbued these skills and knowledge in hand-held devices that are easy to use, and are available to the general population.
    • This accessibility allows novices to learn complex media production, analysis, and broadcast, and allows for the study of these devices as object that has been imbued with the knowledge and skill, as externalized cognitionThrough the use of these devices, the general population may learn complex skills and knowledge that may have required years of specialized training in the past. Study of the interaction between of individuals learning to use these appliances and devices can be studied as a progression of internalizing knowledge and skill imbued in objects.
    • The convergence of media technologies into small, single–even handheld—devices emphasizes that technology for producing media may change, but the narrative has remained relatively consistent.
  • This consistency of media as narrative, imagery, and data analysis emphasizes the importance of the continued study of narrative comprehension and problem solving through the use of these media appliances.

 

I don’t think anyone would disagree — fostering creativity should be a goal of classroom learning.

However, the terms creativity and innovation are often misused. When used they typically imply that REAL learning cannot be measured. Fortunately, we know A LOT about learning and how it happens now. It is measurable and we can design learning environments that promote it. It is the same with creativity as with intelligence–we can promote growth in creativity and intelligence through creative approaches to pedagogy and assessment. Because data-driven instruction does not kill creativity, it should promote it.

One of the ways we might look at creativity and innovation is through the much maligned tradition of intelligence testing as described in the Wikipedia:

Fluid intelligence or fluid reasoning is the capacity to think logically and solve problems in novel situations, independent of acquired knowledge. It is the ability to analyze novel problems, identify patterns and relationships that underpin these problems and the extrapolation of these using logic. It is necessary for all logical problem solving, especially scientific, mathematical and technical problem solving. Fluid reasoning includes inductive reasoning and deductive reasoning, and is predictive of creativity and innovation.

Crystallized intelligence is indicated by a person’s depth and breadth of general knowledge, vocabulary, and the ability to reason using words and numbers. It is the product of educational and cultural experience in interaction with fluid intelligence and also predicts creativity and innovation.

The Myth of Opposites

Creativity and intelligence are not opposites. It takes both for innovation.

What we often lack are creative ways of measuring learning growth in assessments. When we choose to measure growth in summative evaluations and worksheets over and over , we nurture boredom and kill creativity.

To foster creativity, we need to adopt and implement pedagogy and curriculum that promotes creative problems solving, and also provides criteria that can measure creative problem solving.

What is needed are ways to help students learn content in creative ways through the use of creative assessments.

We often confuse the idea of  learning creatively with trial and error and play, free of any kind of assessment–that somehow the Mona Lisa was created through just free play and doodling. That somehow assessment kills creativity.  Assessment provide learning goals.

Without learning criteria, students are left to make sense of the problem put before them with questions like “what do I do now?” (ad infinitum).

The role of the educator is to design problems so that the solution becomes transparent. This is done through providing information about process, outcome, and quality criteria . . . assessment, is how it is to be judged. For example, “for your next assignment, I want a boat that is beautiful and  that is really fast. Here are some examples of boats that are really fast.  Look at the hull, the materials they are made with, etc. and design me a boat that goes very fast and tell me why it goes fast. Tell me why it is beautiful.” Now use the terms from the criteria. What is beautiful? Are you going to define it? How about fast? Fast compared to what? These open-ended, interest-driven, free play assignments might be motivating, but they lead to quick frustration and lots of “what do I do now?”

But play and self-interest arte not the problem here. The problem is the way we are approaching assessment.

Although play is described as a range of voluntary, intrinsically motivated activities normally associated with recreational pleasure and enjoyment; Pleasure and enjoyment still come from judgements about one’s work–just like assessment–whether finger painting or creating a differential equation. The key feature here is that play seems to involve self-evaluation and discovery of key concepts and patterns. Assessments can be constructed to scaffold and extend this, and this same process can be structured in classrooms through assessment criteria.

Every kind of creative play activity has evaluation and self-judgement: the individual is making judgements about pleasure, and often why it is pleasurable. This is often because they want to replicate this pleasure in the future, and oddly enough, learning is pleasurable. So when we teach a pleasurable activity, the learning may be pleasurable. This means chunking the learning and concepts into larger meaning units such as complex terms and concepts, which represent ideas, patterns, objects, and qualities. Thus, crystallized intelligence can be constructed through play as long as the play experience is linked and connected to help the learner to define and comprehend the terms (assessment criteria). So when the learner talks about their boat, perhaps they should be asked to sketch it first, and then use specific terms to explain their design:

Bow is the frontmost part of the hull

Stern is the rear-most part of the hull

Port is the left side of the boat when facing the Bow

Starboard is the right side of the boat when facing the Bow

Waterline is an imaginary line circumscribing the hull that matches the surface of the water when the hull is not moving.

Midships is the midpoint of the LWL (see below). It is half-way from the forwardmost point on the waterline to the rear-most point on the waterline.

Baseline an imaginary reference line used to measure vertical distances from. It is usually located at the bottom of the hull

Along with the learning activity and targeted learning criteria and content, the student should be asked a guiding question to help structure their description.

So, how do these parts affect the performance of the whole?

Additionally, the learner should be adopting the language (criteria) from the rubric to build comprehension. Taking perception, experience, similarities and contrasts to understand Bow and Stern, or even Beauty.

Experiential Learning for Fluidity and Crystallization

What the tradition of intelligence offers is an insight as to how an educator might support students. What we know is that intelligence is not innate. It can change through learning opportunities. The goal of the teacher should be to provide experiential learning that extends Fluid Intelligence, through developing problem solving, and link this process to crystallized concepts in vocabulary terms that encapsulate complex process, ideas, and description.

The real technology in a 21st Century Classroom is in the presentation and collection of information. It is the art of designing assessment for data-driven decision making. The role of the teacher should be in grounding crystallized academic concepts in experiential learning with assessments the provide structure for creative problem solving. The teacher creates assessments where the learning is the assessment. The learner is scaffolded through the activity with guidance of assessment criteria.

A rubric, which provides criteria for quality and excellence can scaffold creativity innovation, and content learning simultaneously. A well-conceived assessment guides students to understand descriptions of quality and help students to understand crystallized concepts.

An example of a criteria-driven assessment looks like this:

Purpose & Plan Isometric Sketch Vocabulary Explanation
Level up Has identified event and hull design with reasoning for appropriateness. Has drawn a sketch where length, width, and height are represented by lines 120 degrees apart, with all measurements in the same scale. Understanding is clear from the use of five key terms from the word wall to describe how and why the boat hull design will be successful for the chosen event. Clear connection between the hull design, event, sketch, and important terms from word wall and next steps for building a prototype and testing.
Approaching Has chosen a hull that is appropriate for event but cannot connect the two. Has drawn Has drawn a sketch where length, width, and height are represented. Uses five key terms but struggles to demonstrate understanding of the terms in usage. Describes design elements, but cannot make the connection of how they work together.
Do it again Has chosen a hull design but it may not be appropriate for the event. Has drawn a sketch but it does not have length, width, and height represented. Does not use five terms from word wall. Struggles to make a clear connection between design conceptual design stage elements.

What is important about this rubric is that it guides the learner in understanding quality and assessment. It also familiarizes the learner with key crystallized concepts as part of the assessment descriptions. In order to be successful in this playful, experiential activity (boat building),  the learner must learn to comprehend and demonstrate knowledge of the vocabulary scattered throughout the rubric such as: isometric, reasoning, etc. This connection to complex terminology grounded with experience is what builds knowledge and competence. When an educator can coach a student connecting their experiential learning with the assessment criteria, they construct crystallized intelligence through grounding the concept in experiential learning, and potentially expand fluid intelligence through awareness of new patterns in form and structure.

Play is Learning, Learning is Measurable

Just because someone plays, or explores does not mean this learning is immeasurable. The truth is, research on creative breakthroughs demonstrate that authors of great innovation learned through years of dedicated practice and were often judged, assessed, and evaluated.  This feedback from their teachers led them to new understanding and new heights. Great innovators often developed crystallized concepts that resulted from experience in developing fluid intelligence. This can come from copying the genius of others by replicating their breakthroughs; it comes from repetition and making basic skills automatic, so that they could explore the larger patterns resulting from their actions. It was the result of repetition and exploration, where they could reason, experiment, and experience without thinking about the mechanics of their actions.  This meant learning the content and skills from the knowledge domain and developing some level of automaticity. What sets an innovator apart it seems, is tenacity and being playful in their work, and working hard at their play.

According to Thomas Edison:

During all those years of experimentation and research, I never once made a discovery. All my work was deductive, and the results I achieved were those of invention, pure and simple. I would construct a theory and work on its lines until I found it was untenable. Then it would be discarded at once and another theory evolved. This was the only possible way for me to work out the problem. … I speak without exaggeration when I say that I have constructed 3,000 different theories in connection with the electric light, each one of them reasonable and apparently likely to be true. Yet only in two cases did my experiments prove the truth of my theory. My chief difficulty was in constructing the carbon filament. . . . Every quarter of the globe was ransacked by my agents, and all sorts of the queerest materials used, until finally the shred of bamboo, now utilized by us, was settled upon.

On his years of research in developing the electric light bulb, as quoted in “Talks with Edison” by George Parsons Lathrop in Harpers magazine, Vol. 80 (February 1890), p. 425

So when we encourage kids to be creative, we must also understand the importance of all the content and practice necessary to creatively breakthrough. Edison was taught how to be methodical, critical, and observant. He understood the known patterns and made variations. It is important to know the known forms to know the importance of breaking forms. This may inv0lve copying someone else’s design or ideas. Thomas Edison also speaks to this when he said:

Everyone steals in commerce and industry. I have stolen a lot myself. But at least I know how to steal.

Edison stole ideas from others, (just as Watson and Crick were accused of doing). The point Watson seems to be making here is that he knew how to steal, meaning, he saw how the parts fit together. He may have taken ideas from a variety of places, but he had the knowledge, skill, and vision to put them together. This synthesis of ideas took awareness of the problem, the outcome, and how things might work. Lots and lots of experience and practice.

To attain this level of knowledge and experience, perhaps stealing ideas, or copying and imitation are not a bad idea for classroom learning? However copying someone else in school is viewed as cheating rather than a starting point. Perhaps instead, we can take the criteria of examples and design classroom problems in ways that allow discovery and the replication of prior findings (the basis of scientific laws). It is often said that imitation is the greatest form of flattery. Imitation is also one of the ways we learn. In the tradition of play research, mimesis is imitation–Aristotle held that it was “simulated representation”.

The Role of Play and Games

In close, my hope is that we not use the terms “creativity” and “innovation” as suitcase words to diminish such things as minimum standards. We need minimum standards.

But when we talk about teaching for creativity and innovation, where we need to start is the way that we gather data for assessment. Often assessments are unimaginative in themselves. They are applied in ways that distract from learning, because they have become the learning. One of the worst outcomes of this practice is that students believe that they are knowledgeable after passing a minimum standards test. This is the soft-bigotry of low expectation. Assessment should be adaptive, criteria driven, and modeled as a continuous improvement cycle.

This does not mean that we must  drill and kill kids in grinding mindless repetition. Kids will grind towards a larger goal where they are offered feedback on their progress. They do it in games.

Games are structured forms of play. They are criteria driven, and by their very nature, games assess, measure, and evaluate. But they are only as good as their assessment criteria.

These concepts should be embedded in creative active inquiry that will allow the student to embody their learning and memory. However, many of the creative, inquiry-based lessons I have observed tend to ignore the focus of academic language–the crystallized concepts. Such as, “what is fast?”, “what is beauty”,  “what is balance?”, or “what is conflict?” The focus seems to be on interacting with content rather than building and chunking the concepts with experience. When Plato describes the world of forms, and wants us to understand the essence of the chair, i.e., “what is chairness?” We may have to look at a lot of chairs to understand chairness.  Bu this is how we build conceptual knowledge, and should be considered when constructing curriculum and assessment. A guiding curricular question should be:

How does the experience inform the concepts in the lesson?

There is a way to use data-driven instruction in very creative lessons, just like the very unimaginative drill and kill approach. Teachers and assessment coordinators need to take the leap and learn to use data collection in creative ways in constructive assignments that promote experiential learning with crystallized academic concepts.

If you have kids make a diorama of a story, have them use the concepts that are part of the standards and testing: Plot, Character, Theme, Setting, ETC. Make them demonstrate and explain. If you want kids to learn the physics have them make a boat and connect the terms through discovery. Use their inductive learning and guide them to conceptual understanding.This can be done through the use of informative assessments, such as with rubrics and scales for assessment.  Evaluation and creativity are not contradictory or mutually exclusive. These seeming opposites are complementary, and can be achieved through embedding the crystallized, higher order concepts into meaningful work.

When a serious game is commissioned, it is expected that in-game learning should transfer to the work place or a clinical setting, not just lead to improvements in game play.

Vegas effect

Evidence of transfer should be a priority in serious game development; there should be evidence that learning acquired in a game is applicable outside of the game.

The Vegas Effect is not unique to games; however, serious games will need to provide evidence that learning that happens in games, does not stay in games.

The tradition of psychometrics may provide methods for data collection and analysis so that serious games may eventually serve as empirically validated diagnostic tools and measures of learning—applicable inside and outside of the game. With tools for measuring training effectiveness from psychometrics, ROI analysis of training solutions and clinical tools can be conducted, and the risk associated with the costs of game development may be diminished.

Serious games and assessment

Serious games are very much like the tools used in psychological assessments and evaluations. Three types of assessments from psychometric methods:

  • Formative assessments –a measurement tool used to measure growth and progress in learning and activity and can be used in games to alter subsequent learning experiences in games. Formative assessments represent a tool external to the learning activity, and typically occur in leading up to a summative evaluation.
  • Summative assessments provide an evaluation or a final summarization of learning. Summative assessment is characterized as assessment of learning and is contrasted with formative assessment, which is assessment for learning. Summative assessments are also tools external to the learning activity, and typically occur at the end of the learning intervention to evaluate and summarize and is conducted with a tool that is external, not part of the training.
  • An informative assessment guides and facilitates learning as part of the assessment. The assessment is the intervention. Successful participation in the learning results in evidence that learning has taken place. The behaviors in the activity have been shown to verify that learning has taken place. No external measures have been added on for assessment.

Games are typically used in the definition of what is an informative assessment. This makes sense, as a game, by its very nature, provides an activity along with assessments, measures, and evaluation. What, why, and how a game measures learning is of primary importance—and this is why serious game designers must learn assessment methods from the field of psychometrics if serious games are to grow as diagnostic tools, assessments, and evaluations.

If a game is to act as an informative assessment, it will stress meaningful, timely, and continuous feedback about learning concepts and process that are accurately depicted. As in an informative assessment, feedback in a game can be a powerful part of the assessment process. As the learner acts in the context of the games rule environment, they may learn the rules and tools through trial and error—eventually developing tactical approaches, and potentially formulate strategies from the possibilities for action deduced from learning from the in-game assessment criteria. This can be powerful.

Evidence supports this powerful learning tool. Research findings from over 4,000 studies indicate that informative assessment has the most significant impact on achievement (Wiliam, 2007). When serious games are built with same care as an informative assessment using methods from psychometrics, serious games can be as effective as an informative assessment.

Currently, most games are not designed as informative assessments. This means that learning in a serious game might suffer from the Vegas Effect. For a game to act as informative assessment, the game must accurately measure the learning the concepts, and the concepts from the game must transfer to other performance contexts—beyond the game. In order to achieve this, the issue of construct validity must be addressed.

For a serious game to have construct validity, the training interventions that they present must have been designed with emphasis on the creation of internal and external validity—what we model, how we measure it, and how it is presented in a game:

  • External validity: the ability to generalize in-game learning to other contexts.  To what extent can a training effect from a game be generalized to other populations (population validity), other settings (ecological validity), other treatment variables, and other measurement variables?
  • Internal validity: examines whether the adequacy of the study design, or in this case of the game, that the intervention was the only possible cause of a change in the players learning.

To do this, serious game development requires valid concepts for modeling, implementation, and assessment of what is to be learned, as well as how it will be measured outside the game. This is essential for ROI (return on investment) analysis. Serious game development requires research and construct validity to conduct ROI and to avoid the Vegas Effect. Learning that happens in games should not stay in games.

 

Leaving Las Vegas:

I have come across few if any games that have been designed with the kind of careful attention to research methodology that would be expected when measuring learning, intelligence, personality, or depression. Methods that ensure construct validity are expected in the field of psychometrics and the learning sciences, and may soon emerge as standard practice in serious game design.

Games are often designed to have surface validity. This means that the game APPEARS to measure what it is supposed to measure. Surface level validity is a useful beginning, but should only be considered a step towards having a valid assessment. It should be considered a gamble to build a serious game on surface validity. Designing a serious game on surface validity increases the likelihood of the Vegas Effect.

To reduce the likelihood of the Vegas Effect, a serious game designer could take their game and correlate learning outcomes with validated tools external to the serious game, such as formative and summative assessments. This method of validation is called criterion validity. To do this, the game designer might correlate success in the game with other diagnostic measures with verified content validity. For example, a claim may be made that a game improves working memory. This claim may be validated using the Dual N-Back Test for measures of working memory. The game designer might choose to have a sample of individuals take the Dual N-Back Task, play the game, and then use the Dual N-Back Task after the serious game to measure changes in working memory using the Dual N-Back Task as criterion for measuring changes in working memory.

Criterion validity is a powerful way to claim effectiveness, and reduce the likelihood of a Vegas Effect. However, the research design is essential in using criterion validity. One cannot simply have someone play their serious game and then attribute changes in the Dual N-Back score by correlation with having played the serious game . . . correlation does not imply causation. To validate the serious game with improvements in working memory on the Dual N-Back Task, the serious game developer should recruit methods from psychometrics such as a Repeated Measures Design, with attention to Sampling.

To really avoid the Vegas Effect, the serious game developer should adopt the gold standard:  Construct Validity. Meaning that the learning designed into the game is measured with the same rigor as the diagnostic tools in psychometrics. Through designing games with construct validity, the game scenarios can be shown to be definitively delivering and measuring the theoretical construct.  Although this is the gold standard, it requires significant investment in time and money to develop. There are however, some methods from psychometrics that can be adopted in the design process of a serious game to reduce the probability of the Vegas Effect.

One methodological step that can be taken towards construct validity is to conduct a study of inter rater agreement on the game elements that deliver instruction. The inter-rater reliability method can be used to identify and score of how much agreement there is on whether the game content is what we say it is.  One way to do this is to individually present the game content to a number of sequestered subject matter experts and ask them to judge. For example, we might present judges with number of scenarios from a game about Decision Making Stages based upon B. Aubrey Fisher’s four stages of group decision making (Fisher, 1970). To do this, the game developer might present the game scenarios to an expert on this topic and ask them to judge, whether the scenario is an example of Fisher’s Orientation Stage in Group Decision Making? Here is the definition:

Orientation stage- this phase is where members meet for the first time and start to get to know each other.

When the expert judges the scenarios, the responses from all the judges can be gathered and inter-rater reliability can be calculated from the responses using Cohen’s Kappa. If the percentage of agreement is low, either the scale (game scenario) is defective or the raters need to be re-trained. If agreement is high, the game scenario is a step closer to construct validity.

Inter-rater agreement is a simple, low-cost method for increasing assessment and content validity. This is an example of how traditional research methods from psychometrics can be integrated as part of the design process from the beginning. As suggested here, an early step in the design process is to conduct tests of inter-rater agreement.

 

This is an excerpt from:

Dubbels, B.R. (in preparation) The Importance of Construct Validity in Designing Serious Games for Return on Investment.

Works cited:

Fisher, B. A. (1970). Decision emergence: Phases in group decision making. Speech Monographs, 37, 53-66.

 

 

What I like about what Katy did with this article is that she showed games in the classroom that do not have to pivot on the “digital natives” argument.

Kids like games, not all of them, but many kids are interested — it really does  take a facilitator to motivate people and to create transfer to academic “work” or formal learning even if there is a high-interest activity available.

Teachers are important to practice and modeling to create a successful habit of learning the process of creating transfer.

That is, the kids need to learn how to learn, and think about learning.

This is called metacognition.

We all need to learn to question quality, question, raise and praise standards, be particular, and really think through the cognitive, social and cultural processes of comparison, combination, contrast, and reduction.

Learning by Design

The curriculum surrounding the games must be designed like a game in many ways:

  • the directions are clear;
  • everyone knows they can win;
  • they can win in different ways;
  • there are support and resources;
  • there is clear criteria in the signal words for qualities and process.

What is the difference between learning with games and learning with text books?

Play as the learning mood (subjunctive mood) and therefore the social structures surrounding the games.

Also, kids have the ownership, and this puts the teacher in the role of facilitator who must be accepting of guiding the learner though the criteria–so if you have lousy criteria, and are not consistent with what your signal words mean in quality and process, expect lousy work,  if not riots.

But if you put forward bad games, no curriculum or bad learning objectives, it is just like reading out of those boring collections of literature texts, without merit, and without no street cred.

The games are not the technology, the curriculum design surrounding the games are the technology. The computers only mediate good instructional design that teachers have always done in cooperative learning– and this exists in board games as well as it has in apprenticeship organizations to learn trades– just like it does in live action role plays,  that in some genres,  are what evolved into video games.

Good instruction is just good instruction. Hopefully this article takes folks towards that direction.

Because we can take the games and go a little further than merely connect.

Just playing a game may increase some cognitive functioning in low functioning people, but that does not mean that they can take that increase  in in-game problem solving to in-life problem solving.

Being good at the game does not mean you will pass a reading test.

Does the reading test really matter?

Maybe not.

Should they be able to pass it?

Yes. They are not that hard.

The work I have done surrounding games has led to improvement in reading scores.

You will notice that between Seventh grade (top-left) and the Eighth grade the following year (bottom-right) there was significant improvement in the school column. During that eighth grade year, I had all of the eighth -graders–but more importantly, we had a significant gain in a year when we switched from the Minnesota Basic Skills Test/ Minnesota Comprehension Assessment to the MCA2, which is a significantly harder test. What you see in comparing my eighth-graders with the year before was the movement from the bottom (who are the hardest to move btw) , the increase in students taking the test (they were all coming to class because of the curriculum), and the amount of students who met or exceeded a higher standard. I must admit, I built my games unit on the standards and my background in reading comprehension made games an easy connection — games are another narrative with all the same literary elements and genre patterns.

The bottom line, they surpassed expectations. We were expected to go down 12%.

The majority of my curriculum that year was in studying video games as new narratives.

Here is the curriculum

Here is a story about what we were doing

Here is a class you can take I have been offering for the last seven years

There is a lot to learn in a game, but there is a whole lot more to learn outside of the game in documenting, listening, presenting ideas, and extending them, than just playing the games themselves.

If you want, there is a whole bunch of games curriculum on my teaching blog  for language arts, reading, engineering, computer science, etc.

Everything from board games to curriculum for analysis of a time line. You might notice that they are set up to be run like a game.

I am hoping that this article makes a start for teachers embracing a model where they consider Learning by Design.Interestingly, games are also involved in assessment, and kids like to know their scores. The scores are an indication of learning.

And the learning is the fun part, the content and the problems are hard–and learning is not always easy, but it can be desirable. Games are hard too, oddly enough, but when enough kids play them, and it creates enough buzz as social capital, there will be interest and some sacrifice to try and persevere in learning.

Tenacity and metacognition are learned traits. With games and play, we can teach them.