New York’s Teacher Data Reports … What are they?

Many teachers in the schools we support received Teacher Data Reports last week. This report is a DOE initiative to give teachers and school administrators information about the individual teacher’s impact on student test scores as compared to their peers.

Reactions on the ground.… Ouch!

To put it plainly … many teachers were really upset.

“There are so many things that a teacher does beyond preparing students to pass a math or reading test that can’t be measured…How can this possibly measure a teacher’s value add!? ” A reaction from one of our own professional developers at Teaching Matters.

But I  find  this report to be pretty complex. So before we reject it, we need to understand what it is and then discuss the merits and the potential improvements.

Teacher Data Report – What does it measure?

In two words.  Teacher Contribution. 

This is an attempt to measure the teacher’s impact on students test scores. It does this by trying to eliminate the effects on test scores caused by  measurable factors that are out of the teacher’s control. For example:

• Student’s prior year reading and math scores, attendance, free lunch status, race, special ed status
• Unusual items like “student is new to school”
• And even something called “classroom effects” like the “percentage of class who attended summer school” or “percentage retained”

The report predicts what test scores a student is likely to receive given all these effects by  using ten years worth of historical data. For example, Jenny with this history and within this classroom would be predicted to get the following score _______.

Then, it compares the score Jenny actually received with the predicted score. The difference is called the Teacher Contribution.

Predicted score + Teacher contribution = Actual Test score

The teacher’s contribution is then calculated for all students and compared to other similar teachers (same number of years of experience teaching the same subject). You are provided the following data. Compared to your peers are you in the top 20%, or in the middle at 60 % or in the bottom 30% of teacher contribution.

Why is the DOE doing this?

Research has now proven that teachers have a significant impact on how well students actually do on tests. Ironic, because many teachers feel the test doesn’t capture what students are learning in their classrooms, but actually the tests do capture the fact that teaching and teachers really matter most to student learning and performance on tests. So while teachers may feel the tests don’t adequately measure what you teach, they do capture that the teacher has a big impact on student performance. In fact, the difference between the lowest and highest performing teachers (measured by tests) is almost entire grade level of movement. Students move only half a year versus 1.5 years.

So teacher impact is important, yet there are few easy, reliable measures of teacher quality available.
New York is trying to be a thought leader in developing these metrics.

Couple of Questions …

So if I may, I would like to ask a couple of questions .

To move to teacher value ad assessments shouldn’t we move to testing system that occurs at the end of the cycle with the same teacher? Now, we know many teachers are drilling  the first part of the year and “teaching” the second part. What if they could really plan for the test over the course of an entire year …   and then be measured on their success.

Is it likely that teachers of students with mostly threes and fours on the test will find it difficult to make that top 20 %. Might these teachers find their “teacher contribution” is not showing up. If children can read adequately, teachers should be spending time more on higher order skills that are not currently measured by the test than worrying about moving high 3s to 4s on a test of reading.  (That is my view anyway.)

Will teachers who spend time working to impact those factors “outside of their control” – get penalized by this model? Remember, the goal was to eliminate the impact of factors outside a teacher’s control from the calculation of teacher contribution. Of course higher attendance will increase performance, but it will also raise the bar for what the student needs to achieve if a teacher successfully raises attendence.  Just a thought. Maybe that is not significant.

The data also breaks down performance by subgroups. How does this work if a teacher has only a few students in a particular group?  Is the data reliable?  I am hoping someone more statistically inclined can speak to that and help me better understand the issues here.

A final thought….

No one should look at data without first getting their own assumptions on the table first. If you as a principal find that you were not able to accurately predict your teachers ranking… all you have learned is that you need a lot more information.

Maybe you haven’t had time to visit the classroom or look at the student work being produced in it this year …   maybe that is part of the problem. 

What you do you think about this initiative?  What are the potential benefits and/or pitfalls for our students?

Read Full Post »

Remixing Culture and 21st Century Skills

I’m always interested in helping teachers master that difficult balancing act between helping students learn today’s essential skills and teaching new skills required for future success. I found Lessig’s new book REMIX and Lessig’s TED talk pretty illuminating. It offered me a new way to think about the importance of ideas like the Read / Write WEB and why we should promote learning through student generated content – even for our most struggling students.

REMIX describes how participation in society and “culture creation” shifted from active (all of us doing it) to passive (big companies creating it and us watching and listening) many years ago. The original technologies of broadcast made us passive listeners rather than active creators. Now – it has shifted back to active, but our kids also have the tools of the broadcast media companies at their disposal.

Lessig offers John Phillip Souza’s prescient testimony against the dangers of the “talking machine” to explain the dangers of passive culture.

These talking machines are going to ruin the artistic development of music in this country. When I was a boy…in front of every house in the summer evenings, you would find young people together singing the songs of the day or old songs. Today you hear these infernal machines going night and day. We will not have a vocal cord left.

-John Philip Sousa

Sousa believed that children could develop a deep understanding of artistic work because they were actively engaged in making their own. And as talking machines (and big media) removed the need to sing and share music, people would become less able to interpret what they were hearing.

Today, our kids are creating again, but not always from scratch. They are in fact, turning off the TV in large numbers, and getting online to create and remix swaths of video, music, and other media (their culture.) As they remix, they reinterpret, and perhaps they learn. They offer us a glimpse of where culture creation is going. They are also breaking the law in the process… copyright law.

Why does this matter to us as educators?

We are required to ensure that kids master specific content and skills. But if the underpinnings of the culture and content creation emerging in the 21^st century is one that is largely made by us (a difficult idea to really wrap your head around) and not just purchased and passively consumed from large media companies – what skills do our students need? Think about it.

We know that this generation is already forcing a change on many businesses, and creating whole new 21^st century jobs etc. Journalists are having to become bloggers, marketing executives are finding that they can’t just tell us what to buy, but are having to become online community facilitators, etc. (“No sage on the stage” doesn’t just apply to teachers anymore). These are signs of the new communication, collaboration, creativity, analysis and information literacy skills .. espoused by ISTE and other groups.

The saving grace here is that many of our kids are leading the way. They are learning some of these skills, essentially unmediated… in spite of schools that ban the practice, and in spite of copyright laws that render them criminals…

For a few other choice examples of remixed content (created by kids but not approved for your students) see lessig’s video talk TED talk.

And for those who are accountability advocates – we can’t ignore this because it feels impossible to measure. If anything, this offers a direction for assessment innovation – assessing performance, innovation, and creative thinking. Until that we can measure these new cultural participatory basic skills, …we will narrow instruction if we only teach to the test. So in the meantime, allow your kids to remix a little after the test is over. (See www.fanfiction.org)

Read Full Post »