|
May 1, 2003
Authentic Assessment of Pre-Service Students' Performance with a Palm or iPAC Handheld Tool
By Marlene Goss
Responding to the call for accountability, increased student learning, and teacher quality, the National Science Foundation (NSF) funded an exploratory feasibility study, West Chester University Partnership for Excellence. This project focused on developing a new model for supervision through collaboration and the use of an emerging innovative tool (like a Palmý or iPACý). This study was designed to catalyze the process of preparing quality teachers and to meet the goals of the NSF-funded CETP-PA or Collaborative for Excellence in Teacher Preparation in PA.
West Chester University of Pennsylvania graduates approximately 600 new teachers a year. All student teachers do their field experience in the Mid-Atlantic region. There is an urgency to prepare these students with the skills and competencies of quality teachers.
Since quality teachers use constructivist teaching methods in their classroom, the project needed a way to measure the ability to teach with strategies and activities inherent in constructivist teaching. WestED, the National Center for Improving Science Education, developed an instrument for measuring constructivist teaching for use on a handheld collecting device.
The assessment instrument relied heavily on the research of Frances Lawrence (1996), who studied the use of time-series design to measure students' achievement and Fred Newmann (1995) who used a database strategy to examine variability in authentic pedagogy and its effect on student performance. Their work directly informed the creation and design of a paper and pencil five-page assessment instrument approved for use by the CETP-PA Project members at the fourteen state system of higher education universities to observe pre-service teachers and faculty. The instrument, the Protocol for Classroom Observation (PCO) is an adaptation of the Classroom Observation Protocol (COP) originally developed by Frances Lawrenz et al, CAREI, University of Minnesota, adapted by the staff of the National Center for Improving Science Education, WestED, Senta Raizen, Director.
A West Chester University five-member team piloted the use of the assessment instrument, Protocol for Classroom Observation (PCO), unobtrusively using a Palm during supervision. The five-member team, led by Marlene Goss, Professor of Educational Technology, West Chester University, included Ellen Newcombe, Director of Technology, School of Education; Dave Bolton, Professor of Assessment and Measurement; Cheryl Krasowski, Science Professor and Supervisor of Secondary Education Science majors; and Chris Golberg, cooperating teacher at Interboro School District in Pennsylvania. The team conducted the research into practice study over a one-year period.
In this innovative technique the supervisor (observer) collects classroom observation data by point and click, reducing time and paperwork to increase productivity. The observer collects data by considering the behavior of the students in the classroom. After entering basic information of name, grade, subject of the lesson, date, and name of the observer and pre-service teacher, every five minutes the supervisor records the level of engagement of students in the classroom activities. In this way the supervisor tracks the level of student engagement during time-series observation. After observing and taking data for 20 minutes or more, the supervisor reflects using a ten-item rubric which measures the frequency of constructivist learning activities during the classroom visit. After the observation visit, the Supervisor uploads this data to a secure password protected segment on the AProý Website, The data is displayed as a record, and there is an opportunity for the supervisor to analyze all the information and make an evaluative decision about the entire quality of the student teacher's skill and competency. All the data is kept in a secure database. The database becomes valuable when used for IR (Institutional Research), informing administrators with accurate and reliable data.
After gear-up fall semester, when the pilot team worked to get the instrument on the Palm and work out the barriers for the process, the new supervision process was ready for use in the field. Field-testing began Spring 2002 term when C. Krasowski and C. Goldberg used the Palm to collect data during student teacher observations. They uploaded the data from their classroom observations to the APro site. This procedure took seconds. Once the data was in the database, they generated a report in seconds and sent it to the student teacher. The report could be personalized and edited before sending it through Email. Furthermore, the data was archived, as a record, at the Website, and could be retrieved anytime in the future. The Website is password protected and each member of the study has his own password. The method of using a handheld device to collect data and generate reports is easy and dependable. Within just one semester of use, each team member is ready to expand the application of the use of the APro system.
After using the new model for Supervision, the university supervisor and cooperating teacher started to analyze the results of the visit in new ways. The focus was not just turning out reports. The University supervisor started to question whether she should visit a pre-service student three days in a row, to see the progression of a lesson and give immediate feedback. This would help evaluate the adjustments a student could make with a supervisor taking a coaching role and working toward development with data-driven decisions. The cooperating teacher wanted to know if it was possible for the University faculty to use this same instrument so there could be conversations based on the cooperating teacher and University teacher using the same measuring tool for evaluating the pre-service teacher. Unfortunately, this study's co-operating teacher and University supervisor were not working at the same school.
All participants of the study were impressed with the speed with which the report was generated, edited, and sent to the pre-service teacher as well as the cooperating teacher and administrator. All participants wanted to continue using APro in the future.
We are continuing. School districts are placing their supervision assessment tool on the Palm devices, using the APro System and at least five universities are using the APro System, the method piloted here, to aggregate data from supervisors and in-service cooperating teachers working in the same school with the same pre-service teacher to determine their progress and development over time. Additionally, at one university, the university supervisor and content faculty member visits the student teacher. The APro technology has catalyzed the team approach preached in the past but all-too-often not practiced by many universities. This new model for supervision empowers them to collaborate to improve constructivist teaching in the classroom and increase student achievement. This Partnership for Excellence in Supervision directly contributes to attaining the goals of the CETP-PA Program by raising the level of the teacher quality.
Email: Marlene Goss
|
