Downloadable PDF version click here.

Knowledge Area Module V:
A Framework for the Pedagogical Evaluation of
Video Game-Based Learning Environments

Richard D. Blunt

Applied Management and Decision Sciences

Student Mentor: Dr. Ruth Maurer

Faculty Assessor:

Walden University

March 7, 2005

Abstract

Currently, there is no framework for the pedagogical evaluation of video game-based learning. A host of research issues have emerged to create the next generation of games to support learning in math, science, and engineering. Yet little to no research has emerged in the area of game-based learning to improve the combat readiness of the armed forces. This paper explores, compares, contrasts, and synthesizes prevailing learning design theories and the use of games for leaning into a logical argument for the need to develop a new learning taxonomy to help root game-based learning in sound pedagogical theory. It adds definitive research in the badly needed area of military game-based learning that the Department of Defense needs that proves, or disproves, the idea that digital game-based learning can improve individual, or collective, performance in the field.

March 7, 2005

Copyright © 2005

Richard D. Blunt

CONTENTS

Table of Figures

List of Tables

INTRODUCTION

The term "edutainment" has been used to describe the idea that the commercial entertainment industry has much to teach educators about how learning happens, especially with reference to the allure of computer digital and video games. Games are seen by some educators as a useful and perhaps even necessary learning environment suitable for learners of all ages. However, there are obstacles to this marriage. One issue concerns the translation of "fun" elements in games to settings of institutional learning where intellectual content is king. Adolescent students often complain that they cannot see the relationship between school subjects and real life. Adult learners view the manipulation of teaching strategies for entertainment value as transparent and reject hybrid experiences as patronizing. Critics of educational game design say that products have erred too far in the direction of weightiness and away from the attraction of play. Indeed, "designers have been tempted to hold children’s play at arm’s length, by referring to games for education as ‘serious’ games and thus completely different from the idle pastimes of the young" (Corbeil, p. 163).

Carl Rogers

In a key 1983 treatise, Carl Rogers made the distinction between education and learning as "lifeless, sterile, futile, quickly forgotten stuff" on the one hand and "the insatiable curiosity that drives the adolescent boy to absorb everything he can see or hear or read about gasoline engines in order to improve the efficiency and speed of his ‘cruiser’" on the other (p. 18). His pronouncement revealed a dichotomy in educational circles: motivation vs. conformity, process vs. product, active and interactive vs. passive, application vs. assessment, and understanding vs. knowledge. Critics have charged that guided institutionalized learning experiences reinforce what society says an individual needs without considering their desires. Rogers describes the excitement that comes with learning, with a capital "L," as a personalized negotiation between these potentially conflicting needs. According to Rogers "the experience of the learner progresses along this line: ‘No, no, that’s not what I want;’ ‘Wait! This is closer to what I am interested in, what I need’; ‘Ah, here it is! Now I’m grasping and comprehending what I need and what I want to know!’" (p. 19, quoted in Smith (1999) [Online]).

Corbeil, Brougere and NESTA

Games provide a way of combining play, which has been seen as a child’s prerogative, and learning, which educators now see as a lifelong means of coping with our world. But with today’s technology, this is surely an arbitrary partition. Play is seen as a freely chosen, pleasurable, unproductive but challenging activity governed by rules and symbols and easily distinguishable from the "real world" (Corbeil, 1999, p. 165, citing Weisler and McCall, 1976). One of the intriguing yet frustrating elements of play is that it is internally motivated, making it difficult to manage or control. In childhood or adulthood, the world of play offers the advantages of learning within the contexts of "distance, pretending, involvement in an activity whose stakes are internal, the management of uncertainty" and lessons in winning and losing (Brougere, 1999, p. 138). Researchers have discovered that game players learn such useful skills as "strategic thinking, planning, communication, application of numbers, negotiating skills, group decision-making and data-handling" (Kirriemuir & McFarlane, 2004, p. 3).

Using Games as a Means for Learning

Squire and Jenkins

Using statistics to reinforce this argument, Kurt Squire and Henry Jenkins (2003) claim that:

Alan Rogers

Alan Rogers (2003) distinguishes between "acquisition learning," a process of task-specific experiences that we may not think of as learning, and "formalized learning," win which the individual is aware of learning taking place through the understanding of principles in a guided setting (see Smith [On-line]):

Holland, Jenkins & Squire

In their paper "Theory by Design," the three authors point out that games model both principles and processes, "particularly the dynamics of complex systems [where] students develop their own languages for illustrating those systems and grow incredibly adept at explaining them in their own terms" (Holland, Jenkins & Squire, 2003, p. 6). This satisfies some concerns about relevancy. Games also "enable teachers to observe their students’ problem-solving strategies in action and to assess their performance in the context of authentic and emotionally compelling problems" (7). An important issue for educators is that research supports the notion "that peer-to-peer teaching reinforces mastery" and therefore, this "information exchange in the context of gameplay" is entirely legitimate as pedagogical interaction (p. 7, citing Koschmann, Ed., 1996). Their question is simple: "The question for educators, then, is not whether games could someday work to teach students; they already do so. The question is how to help these two worlds, that of gaming and that of education, to work together" (7).

BACKGROUND

Bloom’s Taxonomy

Before Carl Rogers there was Benjamin Bloom. Bloom’s Taxonomy (1956) classified cognition into a hierarchy of skills ranging from knowledge of specifics to comprehension, application, analysis, synthesis and lastly, evaluation. Bloom said, "The major purpose in constructing a taxonomy of educational objectives is to facilitate communication" (p. 10). Although originally constructed for college-level instructors and researchers involved in assessment of educational programs, his system was eventually adapted by elementary and secondary teachers who found it useful in writing learning objectives that could be measured (see Anderson & Krathwohl, 2001). "We are of the opinion that although the objectives and test materials and techniques may be specified in an almost unlimited number of ways, the student behaviors involved in these objectives can be represented by a relatively small number of classes" (Bloom, p. 12).

Anderson & Krathwohl update Bloom

Bloom’s use of noun forms to classify levels of intellect reflect the somewhat static thinking of educators in the 1950s; however, he did suggest a shift away from cognition as content by establishing student behavior as a defining criteria. Anderson and Krathwohl (2001, 2002) devised a new version of Bloom’s classification system into a multi-dimensional framework that made distinctions in the cognitive domain between process and knowledge, Table 1.

Table 1: Knowledge Domain

In this new Taxonomy, Bloom’s original six levels remain, now stated in action verbs, with one modification and one addition. Anderson and Krathwohl ranked the process categories from the simplest to most complex activities, reversing Bloom’s "Synthesis" and "Evaluation" and reframing the ultimate intellectual experience of synthesis within a creative realm. "Simply stated, induction, which is involved in Creating, is a more complex process than deduction." (2001, p. 294). In the knowledge dimension, they also followed Bloom’s thinking but changed the levels to range from basic factual knowledge (what) to conceptual (why) and procedural (how) knowledge to the new category of meta-cognitive knowledge, which they defined as "Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition." (p. 29). Meta-cognitive awareness and creative intellectual activity, therefore, represent the highest functioning in the cognitive domain, according to the followers of Bloom and within the framework of a taxonomy of cognition.

ARGUMENT

Two learning theorists published hypotheses in the 1980s based on the shifting paradigms of intellectual content vs. cognitive process and on research in the emerging fields of adult learning and individualized instruction. Gagné and Keller looked at learning from the perspectives of an individual’s retention and motivation.

Gagné defined instruction as "a set of events external to the learner designed to support the internal processes of learning" (Gagné, 1977, 1985). These nine events are predicated on a pre-requisite knowledge and skill levels on the part of the learner and promoted a sequencing of instruction. He first caught the attention of the learner and established the lesson objective; then he presented new material within a context of previous learning; next he provided guidance, opportunity for practice and feedback on performance; and finally, he assessed the learning and encouraged its transference to future applications. "Gagné suggests that learning tasks for intellectual skills can be organized in a hierarchy according to complexity: stimulus recognition, response generation, procedure following, use of terminology, discriminations, concept formation, rule application, and problem solving" (TIP Theories [Online]). His theory is significant to game design and learning objectives in that the principles of sequencing and a hierarchy of intellectual skills provide a framework for individualized instruction tied to specific learning outcomes.

Keller defined the process of motivational design and, based on the theory of expectancy-value from the 1930s (see Tolman, 1932; Lewin, 1938), he developed the ARCS Model for student interest and relevance issues. "Expectancy-value theory assumes that people are motivated to engage in an activity if it is perceived to be linked to the satisfaction of personal needs (the value aspect), and if there is a positive expectancy for success (the expectancy aspect)" (Keller, 1987, pp. 2-3). For Keller, the ARCS acronym represented four natural aspects of learning that would ideally be intrinsic to courses of instruction: Attention, Relevance, Confidence and Satisfaction (1987). The significance of Keller’s work to game design and learning objectives lies in its focus on motivation and the design of instructional processes. "Learner motivation changes over time…and sometimes in unpredictable ways," according to Keller. "When students are motivated to learn, they want to work on highly task-relevant activities. They do not want to be distracted with unnecessary motivational activities" (1999, p. 42).

Motivation and retention, two conditions that form an axis for learning with a capital "L", extend the learner’s experience beyond simple but immediate lessons. Passing the test is no longer the goal, nor is the level of student interest ignored. In this respect, Keller’s ARCS is more appropriate than Gagné’s Nine Events to keep today’s learner motivated and leads to longer retention. The connection between Keller’s ARCS and the elements of play within the structure of games is easy to see (Figure 1), but their actual interconnection in educational settings with learning objectives has not yet been established.

Figure 1: Gagné Versus Keller Learning Design

The need for a new learning taxonomy

Following Keller’s ARCS model, games also have four key components: engagement, rules, goals and challenges. The ARCS model is ideally applicable to the use of games as a learning tool. The parallel between the ARCS model of learning and the basic game design model is easily seen in Figure 2.

Figure 2: ARCS and Game Design comparison

Kurt Squire’s research on the pedagogical relevance of games leads him to this comment about goals: "Exactly what students learn from the game-playing experience depends heavily on the goals they set for themselves. Imagine one player picking the ancient Egyptians [in a computer game Civilization III by Sid Meier] in order to write a paper on the influence of the Nile on ancient history, compared to another whose only goal is to conquer the world" (Squire & Jenkins, 2003, p. 13).

Squire on rules: "games…suspend the rules of reality in order to use the rules of a game" (Squire, 2003, p. 5).

In the 1990s, Malcolm Knowles and Jack Mezirow were credited with foundational theories in adult learning. Knowles (1990) proposed that "andragogy" (as opposed to "pedagogy") distinguishes adult learners from child learners in four ways: adults move from dependence to self-direction, they draw upon experience for learning, they approach learning as problem-solving, and they seek to apply learning immediately. Mezirow (1991) proposed that adults rely on interacting contexts of learning that include a frame of reference, conditions of communication, purpose and intentionality, self-image and the specific learning situation encountered. Mezirow further proposed the idea of adults engaging in "transformation" of previous learning: " Rather than merely adapting to changing circumstances by more diligently applying old ways of knowing, [adults] discover a need to acquire new perspectives in order to gain a more complete understanding of changing events and a higher degree of control in their lives. The formative learning of childhood becomes transformative learning in adulthood" (p.2).

Game designers are often game players, so the application of learning the intricacies of games to commercial product design is an excellent example of adult learning theory at work. Andrew Rollings and Ernest Adams have designed successful video games for top companies and are defining an emerging video game design theory.

Wolf and Perron (2003) have written that "game theory seems to be teetering on a threshold: Many academics want to see game theory establish itself as predominantly academic discipline, while others seek to broaden the conversation between game designers, consumers, journalists and scholars" (p 26).

According to Rollings and Adams (2003), "game design is the process of: Imagining a game. Defining the way it works. Describing the elements that make up the game (conceptual, functional, artistic, and others). Transmitting that information to the team that will build the game" (p 4).

Corbeil (1999) refers to the work of Caillois (1958) in distinguishing different aspects of play qualitatively. Games of competition, chance, imitation and "temporary madness" help to define the spectrum of game playing that ranges from pure fun to major challenges (Corbeil, p. 165, citing Caillois). Corbeil emphasizes the role of activity in games and learning: "Having fun requires making the effort of doing the activity—doing nothing is the epitome of being un-amused—and even a game of chance requires the decision to play by the rules and accept unfavorable results" (p. 166).

Game genres have been developed by different factions of the gaming industry. A popular categorization was devised by Herz (1997):

According to Kurt Squire’s research on what students learned about social studies from computer games, "Games are not simply problems or puzzles; they are microworlds, and in such environments students develop a much firmer sense of how specific social processes and practices are interwoven, and how different bodies of knowledge relate to each other….students can draw meaning from every element in their environment to solve problems that grow organically from their own goals and interests" (Squire & Jenkins, 2003, p. 15).

Conclusion

A theoretical Framework for the Pedagogical Evaluation of Video Game-Based Learning Environments needs to be developed. The framework should consist of a new Instructional Systems Design matrix to select video game capabilities based on specified learning objectives. The matrix should use an adaptation of Bloom’s Taxonomy of learning updated for the new media of video games. Also, the framework should provide a way to help rate, categorize, or better understand the context of game design as it applies to learning.

References