|
February 1, 2001
What Cognitive Modeling Has Taught Us About Math Learning-And How That Research Is Applied for Improved Achievement
By Steven Ritter, Ph.D.
Megan stared at the computer screen presenting the algebra problem.
"I need to calculate how much money I'd make if I worked 110 hours. I'll read the problem statement to find out how my salary is calculated."
As she started to read the problem statement, the computer chimed in, "You don't need to read the problem statement. You've already written an equation to calculate your income as a function of time. Use that function to calculate your income after working 110 hours."
"Right. I know how to do that," Megan said out loud, "but how did the computer know what I was about to do? Its like it's reading my mind."
"Not quite," said the computer, "but I'm working on it."
This scenario might seem to come straight out of a science fiction novel, but, in fact, it is not too far from the type of interaction that might happen in John Anderson's lab at Carnegie Mellon University. Anderson's system employs an eye-tracking device, which can tell where students are looking. These eye movements interact with a cognitive model, which is a computer program that knows how to solve algebra problems often encountered by high school students. The cognitive model knows that students can solve problems either by reasoning about the situation or by using algebraic symbolism. When the eye tracker notifies the cognitive model that the student is reading the problem statement while trying to complete this portion of the problem, the cognitive model infers that the student is reasoning from the situation, rather than from the algebraic expression. Since the system has watched the student solve several problems, it has a good sense of the student's abilities and pushes the student to use the algebraic strategy in this problem.
While it may be some time before eye tracking moves out of the lab and into the classroom, cognitive models have already made the transition. The history of the cognitive model provides a good example of how the latest research in cognitive psychology is guiding the development of educational software that gains an intimate knowledge of individual students' abilities and problem-solving strategies and uses that knowledge to customize instruction and help students learn.
The ACT-R Cognitive Architecture
For more than twenty years, John Anderson has been developing a theory of learning called Atomic Components of Thought-Rational (ACT-R) (Anderson and Lebiýre, 1998). Anderson's goal was to define a "cognitive architecture," which is really a way of laying out the basic rules about how the mind works. He decided that the best way to do this was to create a computer program that could reproduce the basic experimental results in learning, memory, and performance. Using a computer program forced Anderson to be explicit about each detail of the architecture.
If you find it hard to even imagine what kinds of choices you need to make in defining a cognitive architecture, consider the role of short-term memory in thinking. Everyone has had the experience of trying to remember something for a short period of time (for example, you might need to remember a phone number from the time you look it up in the phone book until the time that you dial the telephone). A number of research studies had looked at how many things people could hold in memory. George Miller (1956) summarized a number of these and proposed that people could keep about seven items in this short-term memory.
Now, think about developing a cognitive architecture that models these kinds of results. One proposal might be that there are seven storage areas in short-term memory. Things get placed there until they get "knocked out" by newer items. ACT-R makes a very different proposal. It says that things in memory (called chunks) get activated when you focus on them. Activation follows several well-defined rules, one of which is that there is a limit on total activation. In ACT-R, then, the limit on short-term memory is not a fixed limit (as in the "storage" model); it's a consequence of a limitation on the amount of activation that the mind can handle.
You can think of ACT-R as being the result of hundreds of such decisions about how the mind works, each based on and tested against empirical data. After years of refinement, the architecture is able to model a wide range of phenomena. In fact, the ACT-R framework is now one of the most widely used ways that researchers describe and interpret cognitive processes.
ACT-R Principles
Here are a few basic principles of the ACT-R theory:
There are two types of knowledge: Declarative and procedural
You can think of declarative knowledge as being "factual" knowledge and procedural knowledge as knowledge of how to do things. Declarative and procedural knowledge have different characteristics. Declarative knowledge is relatively slow and flexible and requires attention. Procedural knowledge is relatively fast and inflexible and can operate without much conscious attention. Almost any complex task involves elements of both declarative and procedural knowledge.
Declarative knowledge can be "proceduralized"
One good example is the process of learning to drive a car with a stick shift. Someone can tell you the steps involved (step on the clutch, move the gearshift, etc.), and you can use that declarative knowledge to guide your actions. But driving the car requires the steps to be executed quickly and smoothly, and to do that, you need procedural knowledge. As you practice the steps involved in shifting, more and more of the activity starts to depend on the faster procedural knowledge. After a while, the activity becomes fast and relatively unconscious. The procedural knowledge grows gradually; you learn as you complete the activity.
Knowledge of how to do things is composed of many different components, which are called productions. Each production can be expressed as a condition/action rule (e.g., "if my goal is to shift into first gear, then I should push down on the clutch").
Knowledge gets strengthened with use
This is the key to applying the ACT-R framework to education. Knowledge elements (both chunks and productions) get strengthened when they are used. If two productions are both applicable in a specific situation (i.e., their condition parts are both applicable), then the stronger one will tend to be selected. Now, consider a student trying to solve the following problems:
- simplify 2*3+4
- simplify 2+3*4
Suppose the student's knowledge contains the following productions:
- If my goal is to simplify an algebraic expression, and the expression includes two operators, then apply the operators from left to right
- If my goal is to simplify an algebraic expression, and the expression includes two operators, and one is addition and the other is multiplication, then apply the multiplication operator.
The first production represents a common misconception, which is to ignore order of operations and just proceed from left to right. Nevertheless, this production successfully solves problem A. The second production encodes (part of) the correct knowledge about order of operations and can solve both problems.
If these productions start out equally strong and a student is asked to solve problem A, production 1 might be picked to do the task. The task is successfully completed, so production 1 is strengthened. In this case, the student, by successfully completing a problem, has reinforced a misconception.
You can think of the mind as a collection of competing impulses. One of the tasks in learning is to refine those impulses so that helpful impulses are acted upon. In many cases, the refinements take the form of limiting the context in which the productions apply (e.g., production 2's condition is a specialization of production 1's condition), but overly specific productions will tend to get weaker, because they won't apply often.
It is important to remember that no one oversees this process; nothing eliminates a "bad" production (like the first one), but more appropriate productions can be strengthened to take its place. In this sense, the process is similar to evolution. The "fittest" productions survive, but fitness is sensitive to local conditions rather than a global sense of how to improve things.
From Psychological Model to Educational Software
A thorough understanding of how the mind works is one thing. Putting that theory into practice in the classroom is a different matter and became a related research project at Carnegie Mellon University, led by Anderson and his colleagues Albert Corbett and Ken Koedinger. The ACT-R theory had both a direct and an indirect effect on the development of an educational curriculum now called the Cognitive Tutor®.
ACT-R's direct role is to serve as the basis for an active cognitive model that is incorporated into software that students use as part of their regular classroom activities. The Cognitive Tutor software incorporates a large set of procedural components, which model many different strategies for solving algebra problems as well as common misconceptions that students harbor. When students take an action in the software, the cognitive model relates that action to what it knows about the strategies that students might employ in solving the problem. This process is called model tracing.
Since the cognitive model contains information about likely misconceptions, it is also able to recognize student actions that indicate misconceptions, and it presents students with feedback when they take such actions.
The software also incorporates a technique called knowledge tracing, which is a way to track the growth of individual students' knowledge over time. Through model tracing, the cognitive model identifies each student action with one or more productions. Since the model knows which productions the student has employed, it can estimate the strength of those productions. These estimates can also be used to make predictions about what kinds of problems students are able to solve and which types they still need to work on. The system then uses this information to choose problems for students that emphasize skills that they have not yet mastered. When a student demonstrates mastery of all the productions representing the skills introduced in a particular section of the curriculum, the student is allowed to proceed to the next section of the curriculum, which introduces new skills.
Knowledge tracing provides a built-in method for improving the cognitive model. Since the model's estimate of production strength can be converted into a prediction about student errors, the system can test its own predictions. When these predictions fail, then the cognitive model must not be reflecting the students' thought processes, and the researchers need to adjust the model. Data from tests of the model have shown that student learning proceeds at the production level, which serves as further evidence of ACT-R's representation of procedural knowledge as productions (Anderson and Corbett, 1993).
At an indirect level, ACT-R provides insights into the structure of the tasks that students are asked to complete, and forces curriculum developers to examine their assumptions about the skills that are required at each step in their learning. As a fundamental part of this process, psychology researchers work closely with active teachers to understand how to structure the curriculum in a way that works in real classrooms and that provides support for teachers in their implementation.
Validating the Approach
A strong foundation in cognitive theory gives curricula like the Cognitive Tutors a good chance to succeed, but continual testing and refinement with real students is the ultimate measure of success. Through the years, Cognitive Tutors have been subjected to several controlled evaluations (Koedinger, Anderson, Hadley, and Mark, 1997; Anderson, Corbett, Koedinger, and Pelletier, 1995). These evaluations have shown that students using the Cognitive Tutor Algebra I course perform significantly better on standardized tests than a comparable group of students using a traditional curriculum. On tests of problem solving and the ability to translate between different mathematical representations, students using the Cognitive Tutor course more than doubled the performance of control students.
Development, evaluation and improvement of the Cognitive Tutors continues. While the type of interaction described in the introduction to this article remains in the lab for now, the use of eye trackers has allowed researchers to gain further insights into students' problem solving processes (Gluck, 1999), which lead to further refinements in the systems' cognitive models. We believe that this kind of continual refinement will allow Cognitive Tutor courses to approach the kind of advantage that students might get from individual human tutoring.
Any School Can Implement the Cognitive Tutor
To experience the full Cognitive Tutor benefits of improved test scores and math comprehension, schools should have a computer lab in place or be in the process of equipping their lab. An allocation of technology funds will also be necessary for installation of the software. Usually curriculum administrators, principals, and teachers work with Carnegie Learning to have the program replace the current Algebra I program. The Cognitive Tutor is complete with software, printed workbooks, and lesson plans that include peer presentations and group discussions. Carnegie Learning staff are former math educators who understand the specific needs of teachers and curriculum administrators.
The Cognitive Tutor is used in more than 750 schools across the United States. In most schools that have used the program for several years, it has been possible to completely eliminate all general or "business" math courses, which greatly pleases most principals and superintendents. Recently the state of Alaska negotiated a statewide implementation of the program, so that any school in the state wishing to implement the Cognitive Tutor can do so. It has been recognized as an "Exemplary" program by the U.S. Department of Education and was also recognized recently at the National Education Summit in Palisades, N.Y., as one of the best results-based programs in the country.
Email: Steve Ritter
More information on the Cognitive Tutor can be found at carnegielearning.com.
References
Anderson, J. R. & Corbett, A. T. (1993). Tutoring of cognitive skill. In J. R. Anderson, Rules of the Mind (pp. 235-255). Hillsdale, NJ: Erlbaum.
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4 (2) 167-207.
Anderson, J. R. & Lebiýre, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum.
Gluck, K. (1999). Eye movements and algebra tutoring. Unpublished doctoral dissertation. Carnegie Mellon University.
Koedinger, K. R., Anderson, J. R.., Hadley, W. H.., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97.
|
