Seattle Education

For the news and views you might have missed

EPI Report on Test Scores and Teacher Evaluations

This just in from the Economic Policy Institute, EPI:

NEWS FROM THE EDUCATION PROGRAM

August 29, 2010 Contact: Educ_Prog@epi.org

In new EPI report, leading educational testing experts caution against heavy reliance on the use of test scores in teacher evaluation

Student test scores are not reliable indicators of teacher effectiveness, even with the addition of value-added modeling (VAM), a new Economic Policy Institute report by leading testing experts finds. Though VAM methods have allowed for more sophisticated comparisons of teachers than were possible in the past, they are still inaccurate, so test scores should not dominate the information used by school officials in making high-stakes decisions about the evaluation, discipline and compensation of teachers.

The Obama administration has encouraged states to adopt laws that use student test scores as a significant component in evaluating teachers, and a number of states have done so already. The Los Angeles Times recently used value-added methods to evaluate teachers in the Los Angeles Unified School District based on the test scores of their students, and Secretary of Education Arne Duncan supported the paper’s decision to publicly release this information, asserting that parents have a right to know how effective their teachers are.  But the conclusions of the expert co-authors of this report suggest that neither parents nor anyone else should believe that the Los Angeles Times analysis actually identifies which teachers are effective or ineffective in teaching children because the methods are incapable of doing so fairly and accurately.

The distinguished authors of EPI’s report, Problems with the Use of Student Test Scores to Evaluate Teachers, include four former presidents of the American Educational Research Association; two  former presidents of the National Council on Measurement in Education;  the current and two  former chairs of the Board of Testing and Assessment of the National Research Council of the National Academy of Sciences;  the president-elect of the Association for Public Policy Analysis and Management;  the former director of the Educational Testing Service’s Policy Information Center and a former associate director of the National Assessment of Educational Progress; a former assistant U.S. Secretary of Education; a former  and current member of the National Assessment Governing Board; and the current vice-president, a former president, and three other members of the National Academy of Education.

The co-authors make clear that the accuracy and reliability of analyses of student test scores, even in their most sophisticated form, is highly problematic for high stakes decisions regarding teachers . Consequently, policymakers and all stakeholders in education should rethink this new emphasis on the centrality of test scores for holding teachers accountable.

Analyses of VAM results show that they are often unstable across time, classes and tests; thus, test scores, even with the addition of VAM, are not accurate indicators of teacher effectiveness.    Student test scores, even with VAM, cannot fully account for the wide range of factors that influence student learning, particularly the backgrounds of students, school supports and the effects of summer learning loss.  As a result, teachers who teach students with the greatest educational needs appear to be less effective than they are.  Furthermore, VAM does not take into account nonrandom sorting of teachers to students across schools and students to teachers within schools.

There are further negative consequences of using test scores to evaluate teacher performance.  Teachers who are rewarded on the basis of their students’ test scores have an incentive to “teach to the test,” which narrows the curriculum not just between subject areas, but also within subject areas.  Furthermore, creating a system in which teachers are, in effect, competing with each other can reduce the incentive to collaborate within schools-and studies have shown that better schools are marked by teaching staffs that work together.  Finally, judging teachers based on test scores that do not genuinely assess students’ progress can demoralize teachers, encouraging them to leave the teaching field.

Evaluating teachers accurately is an extremely important piece of the effort to improve America’s schools, and VAM methods are appealing in that they seem to offer an objective and simplified way of comparing one teacher with another.  However, as EPI’s report makes clear, “There is simply no shortcut to the identification and removal of ineffective teachers.”  The authors conclude that that, “Although standardized test scores of students are one piece of information that school leaders may use to make judgments about teacher effectiveness, test scores should be only a small part of an overall comprehensive evaluation.”

The report’s co-authors are:

  • Eva L. Baker, Professor of education at UCLA and Co-Director of the National Center for Evaluation Standards and Student Testing (CRESST)
  • Paul E. Barton, former Director of the Policy Information Center of the Educational Testing Service
  • Linda Darling-Hammond, Professor of education at Stanford University, former President of the American Educational Research Association
  • Edward Haertel , Professor of education at Stanford University, former President of the National Council on Measurement in Education, Chair of the National Research Council’s Board on Testing and Assessment, former Chair of the committee on methodology of the National Assessment Governing Board
  • Helen F. Ladd, Professor of public policy and economics at Duke University, President-elect of the Association for Public Policy Analysis and Management
  • Robert L. Linn, professor emeritus at the University of Colorado, former President of the National Council on Measurement in Education and of the American Educational Research Association, former Chair of the National Research Council’s Board on Testing and Assessment
  • Diane Ravitch, Research Professor at New York University and historian of American education
  • Richard Rothstein, Research Associate of the Economic Policy Institute
  • Richard J. Shavelson, Professor of Education (Emeritus), former dean of the School of Education at Stanford University, and former president of the American Educational Research Association
  • Lorrie A. Shepard, Dean and professor at the School of Education at the University of Colorado at Boulder, former President of the American Educational Research Association, immediate past President of the National Academy of Education

The report is available at: http://www.epi.org/publications/entry/6276/

If you would like to subscribe to future mailings from EPI,
visit http://www.epi.org/signup.
Questions? Contact newsletter@epi.org.

Economic Policy Institute
1333 H Street, NW
Suite 300, East Tower
Washington, D.C. 20005

5 comments on “EPI Report on Test Scores and Teacher Evaluations

  1. Sahila
    August 30, 2010

    Some people are accusing the EPI report of being biased in favour of teachers/unions…

    Here is another report that says basically the same thing:
    http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf

  2. Sahila
    August 29, 2010

    @ anothermom:

    The education ‘deformers’ are not in denial…

    They have a very clear strategy and they are about 80% of the way towards successful, national implementation…

    Basing teacher pay on students’ standardised test results is a cornerstone of that deform agenda to privatise public education, standardise and narrow curricula, deprofessionalise the teaching corp and bust the unions…

    And here in Seattle, we have:
    * Broad Foundation plant Super MGJ,
    * a rubber stamping Board (4 of whom received Gates-related funding for their election campaigns, all of whom have been exposed to Broad propaganda at Board retreats and on junkets around the country),
    * more Broad fellows working as permanent staff and as interns in senior SPS management,
    * astro-turf groups funded by Broad and Gates lobbying locally and in Olympia,
    * a complicit media (the Seattle Times), and
    * general teacher and community ignorance of all this…

    We’re watching the last gasp of the last vestige of a democracy – a quality public education system….

  3. seattleducation2010
    August 29, 2010

    I don’t hear the fat lady singing.

    In Seattle it is definitely not too late. Teachers are in negotiations right now over these issues.

    The best thing that we can do now is to show support for our teachers during this siege. Call or e-mail your teachers, they are back and getting ready for fall classes this week, and let them know that we do not agree with high stakes testing and performance evaluations based on student test scores. That is what our Broad superintendent is trying to ram down their throats as we speak.

    Another action that you can take is to get in front of the school board on Wednesday and say your piece or stand behind others who will be able to speak.

    If you come to the meeting on Wednesday, wear blue in solidarity for our teachers.

    All of these actions can make a difference and change the tide.

    Information on the school board meeting can be found at http://www.seattleschools.org/area/board/index.dxml.

    As parents we have a voice, and a big one, in how we want to see our children educated.

    Dora

  4. Demian
    August 29, 2010

    We can never stop all bad things from happening, but it’s never too late to fight for what’s right.

  5. Anothermom
    August 29, 2010

    Dora,
    This report is a bombshell. I have only read to page ten and already it has confirmed what we all know. It is simply not a good practice to use student test scores in the manner that SPS desires when evaluating teacher effectiveness.The current ed reformers are in complete and total denial. I fear that it is too late because so many have bought and drank the koolaid.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: