Early in my career as a principal, tech-savvy teachers in my school led a charge to replace blackboards with cutting-edge whiteboards. Over the course of a couple of years, we retired all our frumpy old chalkboards and replaced them with shiny new whiteboards. What changed as a result?
Some changes were immediately clear. For example,when we threw away our chalk, staff and students who were allergic to the dust breathed more freely. The teachers and parents on the school council felt justifiably proud; they had brought our school forward and up to date.
On the other hand, we also learned that the cost of each of the colored markers required to write on the whiteboards was considerably greater than the cost of a piece of chalk. The markers didn't last any longer than chalk, they gave off fumes (which some complained caused headaches), and once in a while someone would use a permanent marker on the whiteboard by accident, ruining the surface.
Results in terms of student outcomes were not so easily identified. Did standardized test scores improve because we taught with whiteboards instead of chalkboards? Did attendance improve, and were fewer students tardy to class? Was there a reduction in the number of new students referred to special education services? Did fewer skilled, experienced teachers whom we would have preferred stay at our school request transfers to other district schools?
Who knows? It never occurred to us to identify — let alone track — the specific changes we would expect as a result of spending literally thousands of dollars of scarce school discretionary funds to purchase and install a specific educational technology. Shame on us.
Calculating the Advantages
Conservative estimates indicate that in the past five years U.S. schools have spent more than $25 billion on various forms of technology, including the still-popular whiteboard (or course, it's interactive now and requires a computer and "pens"). According to Market Data Retrieval, spending will be up in 2006 in three-quarters of schools. Some of that money will be yours.
With these figures in mind, it's important to ask yourself how to increase the chances of getting value for your technology investment. There is no magic formula for calculating a specific dollar estimate of savings or returns on a new technology, nor is it wise to consult a complex, multipage set of criteria before making any major technology purchase. But before you spend another penny on ed technology, consider the following basic questions:
- How does the technology add value?
- How will you know the technology is working?
- Is the value added worth the cost — and the risk?
- A teacher has 5th grade students compose essays on paper, then exchange papers and correct each other's grammar and spelling. The teacher reads the papers and marks additional items to be corrected. Students then type their now-perfect essays into a word processing program, spending quite a bit of time experimenting with various fonts, and ultimately print copies to post on the bulletin board.
- Third grade students are taking a math test in the computer lab. It includes both word problems and computations. For both sections of the test, students do calculations on paper. Once they arrive at their answers, they click the matching choice on the screen.
- An 8th grade social studies teacher instructs students to read a textbook chapter, create an outline, and e-mail it to him. There are enough computers for only half the class, so some students read at their desks, waiting their turn to create an outline on the computer, while the others compose outlines on the computers, struggling to read books propped up between monitors or on their laps.
In these examples, technology is adding little if any value to the educational process. In the writing example, students did all the real work before they ever got to put fingers to the keyboard. The math test could have been given more efficiently back in the classroom using good old Scantron answer sheets. As for the social studies class, the technology might actually have interfered with learning.
The most frustrating thing about these examples is that they are real. They were perpetrated by intelligent, well-meaning educators who just didn't take the time to ask themselves, "How am I going to use this technology to do something better or something new that I couldn't do before?"
Beyond the Conventional
Now imagine how these scenarios might play out if the technology truly were adding value. Obviously, the 5th graders would be using the computer through every stage of the writing process. They might use concept-mapping software to brainstorm ideas and then convert them to outline form. They'd move to a word processing program to compose their first draft and take advantage of the program's tools for checking spelling and grammar. Using a collaborative workspace or by simply attaching their essays to e-mail messages, students could exchange, read, and comment on the actual substance of one another's work. With feedback from peers and the teacher, they would revise their essays until they were ready to be published electronically or on paper.
As for the 3rd graders, give them some calculators or handheld computers. Let them explore the world and guide them to formulate real-world math problems. Have them calculate how much the class field trip to the museum will cost, including gasoline and the bus driver's hourly rate. When they get their turn to go to the computer lab, let them use simulation or modeling software that helps them develop mathematical and logical concepts or try their hands at making simple tables and graphs using real data they collected back in the classroom or for homework. "What about memorizing the multiplication table?" you ask. Simple flash cards, weekly quizzes, and meaningful incentives can still accomplish that efficiently and far more cheaply than computerized drills.
Let the 8th graders read their textbooks at their desks, where there is room to lay them down, and perhaps write their outlines, too, on regular old notebook paper. But beyond the outline, what about having each student identify something puzzling or interesting in the chapter? Then, when they rotate to the computers, let them use the resources of the Internet and the power of search engines to delve into their individual topics.
Think Innovation
You've probably noticed a theme running through these examples: whether or not the technology is doing anything better or new depends a lot on what the teacher has the students do with it. In his best selling book, The World is Flat, Thomas Friedman writes, "Introducing new technology alone is never enough. The big spurts in productivity come when a new technology is combined with new ways of doing business." When we lament the fact that despite our huge investment in technology our educational results are not commensurately better, I suspect Friedman's observation points toward the reason why. We need to practice new ways of "doing education" that take advantage of what new technologies offer.
The Measurement Challenge
That leads to the second question: How will you know it's working? You'll be far more likely to realize the added value of technology if you give some thought ahead of time to what you expect to be different as a result. If students write on computers instead of on paper, how do you expect their writing will be different? Will they become better spellers and grammarians (perhaps just the opposite if they rely on the auto-correction tools of the word processor)? Will they become more fluent writers, and will they be more willing to write in the first place? Will there be a significant rise in the number of students who score at the highest level of the district's writing rubric? If your 3rd graders use calculators in math, will they score higher on state tests of computation skills? Not likely, but you might expect to see some dramatic improvement in problem-solving skills or attitudes toward math.
As for 8th-grade social scientists, would writing outlines on the computer yield a better grasp of the key points in the chapter than writing the same outline on paper? Probably not. On the other hand, we might predict that by using Internet resources and search engines to pursue unique, self-chosen topics, students would develop more sophisticated research skills and choose to apply them in their personal lives as well as within other classes or subjects. But real measures are needed. In some cases, existing measures will serve the purpose. Improvement in students' writing should certainly be reflected in scores on a well-written district writing rubric. If calculators are part of a well-thought-out program for increasing students' problem-solving skills, those gains should be reflected in their scores on the problem-solving section of a standardized test.
By the same token, some of the changes you may expect to see will require you to adopt or invent new measures. For example, how would you measure writing fluency and the willingness to write? How about attitudes toward math, Web-based research skills, or the degree to which students apply such skills in their personal lives? Yes, it can be a challenge to construct valid measures of such things, but it's a challenge that must be met if schools want to realize full value from technology investments.
To Purchase or Not to Purchase?
Finally, before going forward with a technology purchase, it must be determined whether the value the technology adds — and the risk — is worth the cost. At this stage of the process, you should have a clear idea of the value added by the technology, the changes you expect to see as a result of using the technology, and the measures that will confirm that those changes are really happening. Beyond that, consider these factors:
Consider total cost of ownership. What is your financial outlay going to be? With most technology purchases, buying the technology is only a fraction of the total cost. What else do you need to make it work, and what are the annual support costs? What will it cost to train staff to use the stuff, and how long will it last before you have to upgrade or replace it? There are some excellent tools available to help you do a thorough analysis of total cost of ownership.
Weigh opportunity cost. Opportunity cost is the value of the next best way you could spend the same money. (The concept assumes you have a finite pot of money, which sounds like most school districts I know.) Sure, we're always vaguely aware that when we buy one thing it precludes us from buying some other thing, but we tend not to force ourselves to identify the actual item or service we're passing on. Perhaps that means no new instruments for the band this year, charging parents for bus transportation, or reducing the number of teacher aides. Once you know what you're giving up, you will have a true understanding of the cost of the technology purchase.
Look at what might go wrong. In his book, Learned Optimism, psychologist Martin Seligman asserts that in most situations, optimism breeds success. However, he allows that there are times when people are better served by a bit of pessimism, such as decisions in which the downside risks are great. Before signing the purchase order, think about two things: What will happen if the new technology is a bust, and even if it works as planned, what might be some negative side effects? Only when you have convinced yourself you can deal with this worst-case scenario should you sign that paper.
So now you have a three-pronged plan to help ensure that you and your students get a decent return on your technology investment. Clearly identify how you expect the technology will add value by doing something better or enabling something new to happen. Make a concrete plan for measuring whether or not the technology is, in fact, producing results. Finally, do the math; be certain that you know what the technology is going to cost you and are convinced the anticipated results are worth it.
Michael Simkins is the creative director of TICAL in the Santa Cruz County Office of Education.
ROI Resources
Educational Technology Expert Panel: Exemplary and Promising Educational Technology Programs 2000 (PDF 7.82 MB)
This site provides information on seven programs judged by a U.S. Department of Education Expert Panel to be exemplary or promising. The publication includes an excellent set of criteria for assessing your own technology programs.
Basic Guide to Program Evaluation
This site provides sound advice on how you might measure whether or not new technology is doing what it's supposed to do.
Taking TCO to the Classroom and K-12 TCO Calculator
Determine just what new technology truly will cost with these tools.
Joel Barker's Implications Wheel
Joel Barker's Implications Wheel can help your planning group map both the positive and negative outcomes that might result from your new technology effort.
Making Sure Technology Pays Off
This narrated slideshow gives a concise overview of research findings related to technology use and the conditions that need to be in place for computer-based technology to have the most impact on student learning.
