Sign up for newsletter

June 15, 2002

The Administrator's Guide to Data-Driven Decision Making (cont'd)

STEP 1 Conduct an Information Inventory

Before considering complex and potentially expensive solutions such as data warehouses or online analytical processing tools, examine your current information management systems. Data-driven decision making is only as good as the data sources available. New systems cannot improve decision making if there are significant problems in the way data is being managed or in the existing student, financial, and human resources information systems. Start with these questions when evaluating your current information systems.

What data are you collecting now?

Most schools and districts house tremendous stores of data, parts of which may be inaccessible or unknown to decision makers. During the course of a single week, for example, guidance counselors keep student visit logs, assistant principals monitor disciplinary actions by student and time of day, and individual teachers record calls made to students' parents. All this data has potential value if it can be accessed and analyzed by the appropriate individuals. Your school is likely to have that data stored in databases, spreadsheets, text files, and on paper. Make a list of these sources as the first step in exploring ways to make the data accessible to both those who use it on a daily basis and the decision makers who need it for higher-level analyses.

Are your existing data sources compatible?

Once you've found your data sources, identify the obstacles to making connections between disparate databases. For example, can you ask whether there is a correlation between a teacher's professional development activities and their students' academic performance? Can you relate the money spent on a reading readiness program to its relative degrees of student success? Whether you can ask and answer these questions is determined by the technical design of your data sources. To insure seamless integration of different data, focus on three critical elements: keys, formats, and granularity.

Every record in every data source must include a standard key, such as a unique identification number, that lets you join different databases. Keys identify records about students, courses, staff members, and any other organizational unit you want to track. To merge data elements, identification numbers must be consistent across databases. Often various school departments code data in different ways. For example, the health office might code gender as M/F or Male/Female or even 0/1, while the principal's office uses only M/F. In order to join data in these fields, you must code gender the same way or you'll need to convert data to the same formats before taking the next step.

To be compatible, data must have similar granularity. Granularity is the level of detail inherent in a single piece of data. Test scores, for example, offer multiple levels of granularity from very fine, such as answers to individual questions, to very coarse, such as overall composite scores. Granularity must be similar if you're going to correlate data from different sources. For example, let's say you're trying to determine the effectiveness of a student attendance incentive program. Students with perfect attendance receive monthly rewards. Because attendance is recorded each day and the incentive program is measured in months, there is a difference in granularity. The daily attendance data would have to be aggregated into months in order to have the same granularity as the incentive data.

What additional data do you need to collect?

Think creatively and go beyond the confines of your traditional transactional databases, which most often include student demographics, grades, test scores, and attendance. Explore what useful new information you could gather about students, teachers, parents, and programs that could help you design targeted services and support. Following are examples of a broader vision of data collection.

Program participation. Track students' participation in curricular and extracurricular programs, including tutoring, sports, arts programs, and summer school. Record how much time students spend in each program, how much staff time is required to run them, and how much they cost. This data can be used to evaluate the success of various programs in terms of student performance and cost-effectiveness.

Discipline. Collect information about when and where disciplinary incidents occur. Capture data on the teachers and students involved in the incidents. Include records of the consequences for each behavioral infraction. This data can help measure the effectiveness of various discipline strategies, to identify hot spots that need more supervision, or to find patterns of incidents with teachers or students who may benefit from additional support.

User satisfaction. Distribute surveys that ask parents, students, and staff for their opinions on school and district programs. Include questions that solicit opinions on a broad array of issues including components of the curriculum, facilities, and extracurricular offerings. You can then compare feedback from parents, students, and staff to identify trends. Surveys frequently show where priorities differ between administrators and parents or students.

How frequently do you collect important data?

Ask whether you are collecting data often enough for it to be useful in decision making. Take the example of standardized tests: These high-stakes assessments are typically given in the spring, results arrive late in the school year, and new strategies based on the data cannot be implemented until the beginning of the next year, when students have moved on to the next grade. Because of the length of time between data sets, it's difficult to know which changes to your curriculum and assessment programs have been effective and which have not.

A solution is to collect data more frequently. With standardized tests, you often have the option of ordering different forms of the test that can be used at different times during a school year. Alternatively, you might also establish an in-house practice test routine that provides data to teachers on a monthly or quarterly basis.

Streamlining Your Data Collection

Here is one district's checklist.

• We started by combining the seven different databases that contained overlapping teacher information into a single networked database that was shared by seven staff members.
• Each staff member was then assigned responsibility for maintaining certain fields of the database.
• Next we identified and eliminated parallel systems for recording student information, library automation, and student nutrition.
• Finally, we held training sessions with each staff member on the importance of data in the organization and their role in maintaining information integrity by using examples of inaccurate data collection from the parallel systems we uncovered.

Go to Step 2 > > >

< < < Return to Intro

Read other articles from the May Issue

Send a letter to the Editor in response to this article.