Many teachers in the schools we support received Teacher Data Reports last week. This report is a DOE initiative to give teachers and school administrators information about the individual teacher’s impact on student test scores as compared to their peers.
Reactions on the ground.… Ouch!
To put it plainly … many teachers were really upset.
“There are so many things that a teacher does beyond preparing students to pass a math or reading test that can’t be measured…How can this possibly measure a teacher’s value add!? ” A reaction from one of our own professional developers at Teaching Matters.
But I find this report to be pretty complex. So before we reject it, we need to understand what it is and then discuss the merits and the potential improvements.
Teacher Data Report – What does it measure?
In two words. Teacher Contribution.
This is an attempt to measure the teacher’s impact on students test scores. It does this by trying to eliminate the effects on test scores caused by measurable factors that are out of the teacher’s control. For example:
- Student’s prior year reading and math scores, attendance, free lunch status, race, special ed status
- Unusual items like “student is new to school”
- And even something called “classroom effects” like the “percentage of class who attended summer school” or “percentage retained”
The report predicts what test scores a student is likely to receive given all these effects by using ten years worth of historical data. For example, Jenny with this history and within this classroom would be predicted to get the following score _______.
Then, it compares the score Jenny actually received with the predicted score. The difference is called the Teacher Contribution.
Predicted score + Teacher contribution = Actual Test score
The teacher’s contribution is then calculated for all students and compared to other similar teachers (same number of years of experience teaching the same subject). You are provided the following data. Compared to your peers are you in the top 20%, or in the middle at 60 % or in the bottom 30% of teacher contribution.
Why is the DOE doing this?
Research has now proven that teachers have a significant impact on how well students actually do on tests. Ironic, because many teachers feel the test doesn’t capture what students are learning in their classrooms, but actually the tests do capture the fact that teaching and teachers really matter most to student learning and performance on tests. So while teachers may feel the tests don’t adequately measure what you teach, they do capture that the teacher has a big impact on student performance. In fact, the difference between the lowest and highest performing teachers (measured by tests) is almost entire grade level of movement. Students move only half a year versus 1.5 years.
So teacher impact is important, yet there are few easy, reliable measures of teacher quality available.
New York is trying to be a thought leader in developing these metrics.
Couple of Questions …
So if I may, I would like to ask a couple of questions .
To move to teacher value ad assessments shouldn’t we move to testing system that occurs at the end of the cycle with the same teacher? Now, we know many teachers are drilling the first part of the year and “teaching” the second part. What if they could really plan for the test over the course of an entire year … and then be measured on their success.
Is it likely that teachers of students with mostly threes and fours on the test will find it difficult to make that top 20 %. Might these teachers find their “teacher contribution” is not showing up. If children can read adequately, teachers should be spending time more on higher order skills that are not currently measured by the test than worrying about moving high 3s to 4s on a test of reading. (That is my view anyway.)
Will teachers who spend time working to impact those factors “outside of their control” – get penalized by this model? Remember, the goal was to eliminate the impact of factors outside a teacher’s control from the calculation of teacher contribution. Of course higher attendance will increase performance, but it will also raise the bar for what the student needs to achieve if a teacher successfully raises attendence. Just a thought. Maybe that is not significant.
The data also breaks down performance by subgroups. How does this work if a teacher has only a few students in a particular group? Is the data reliable? I am hoping someone more statistically inclined can speak to that and help me better understand the issues here.
A final thought….
No one should look at data without first getting their own assumptions on the table first. If you as a principal find that you were not able to accurately predict your teachers ranking… all you have learned is that you need a lot more information.
Maybe you haven’t had time to visit the classroom or look at the student work being produced in it this year … maybe that is part of the problem.
What you do you think about this initiative? What are the potential benefits and/or pitfalls for our students?