Saturday, January 16, 2010

Teacher Evaluation and Performance Management

Commentary
by Professor Tony Wheeler
Professor Richard W. Scholl


What is at stake for your company when measuring performance? For the state, measuring teacher performance could be worth an additional $100M to a K-12 budget already approaching $700M, but it appears that the teacher unions have gotten in the way again (Projo, 1/15/09). In one corner, we have the state wanting to measure teacher effectiveness via student test scores. In the other corner, we have at least one teachers' union wanting a more comprehensive assessment of teaching effectiveness.

For many years, the topic of teacher evaluations has surfaced in discussions of educational improvement. On the surface, it stands to reason that the performance of teachers is an important determinate of student learning. Most recently, the use of student standardized tests scores as a measure of teacher performance has been a sticking point in obtaining union support for the state's grant proposal for federal Race to the Top funds. One aspect of the state's Race to the Top application, is that 51% of a teacher's evaluation will be based on test scores. While we are not fully aware of all the specific issues regarding this specific use of student test scores as teacher effectiveness evaluations, we believe that there a basic principles of performance management that bear directly on the issue of how teachers should be evaluated and for what purpose evaluations should be used.

Performance Management is More Than a Set of Evaluation Metrics
Typically, performance management constitutes a process whereby employers observe, document, and improve employee performance. Effective performance management involves the integration of appraisal, feedback, development, and behavioral change processes working in concert to improve employee performance. While in any performance management process there comes a point when it is appropriate to terminate ineffective employees, given the cost of replacing employees, effective companies attempt to first diagnose the causes of low performance and make efforts to improve performance. The use of student test scores as the foundation for a strategic performance management process appears to solely focus on "getting rid" of "teachers that can't cut the mustard." Like any type of employee performance, teaching effectiveness is a complex set of behaviors stemming from a combination of knowledge, skills, abilities, motivation, role expectations, and available resources. Before jumping into a plan of evaluation and action, we recommend a fuller understanding of the causes of ineffective teacher performance and student learning gaps. 

Outcome versus Process Measures of Performance
A fundamental issue in performance appraisal is whether to measure employee performance using process/behavioral measures or to use outcome/results measures. Each type of appraisal outcomes has issues to consider. For example, when using process measures, there is the danger of reducing creativity and forcing all employees to use a single method which may not fit all situations. In using outcome measures, we run the risk that employees lack control over the factors that lead to the desired outcomes.

Based on the current issue at hand, let's focus on the use of outcome measures. First, is the student achievement test a valid and reliable measure of student knowledge? Notice, knowledge not learning. You must document changes in student scores to determine student learning. Second, you then have to make the link that those changes in student learning directly result from teacher influence and not from things like parental involvement, socio-economic status, and the like over which teachers have no control. Third, what cut-off will the state establish for student performance that then suggests some "objective" evidence of teacher performance? Is it a certain amount of year-over-year performance from each student that is then aggregated (to school, grade, or teacher level)? Or is it some arbitrary number, like each student must pass a certain percent of the test? In either case, how can the state factor out teacher effectiveness from those numbers? Fourth, in their book Freakonomics, Steven Levitt and Stephen Dubner note how test scores can be manipulated and teachers feel pressure to "teach toward the test." In our opinion, the state's desire to use student test scores to account for 51% of a teacher's performance simply increases teachers' motivation to do whatever they can do to improve test scores. That's not student learning. Proceed at your own peril, State of Rhode Island.

Educational Improvement versus Teacher Evaluation
Whether or not test scores or other objective measures of learning have a place in teacher evaluation, we believe that it is imperative for schools to be able to assess student learning. In commenting on the use of test scores to evaluate teachers, Debra Gist, the state's education commissioner is quoted "We don't take this lightly. We will develop a very clear plan, based on very solid evidence. We need a consistent measure that is objective and reliable … that has some ability to reconcile what a fourth grader in Westerly is learning with what a fourth grader in Woonsocket is learning."

We could not agree more, but there is a big difference between using outcome-based assessments of learning as a part of an educational continuous improvement process and using these metrics to evaluate the effectiveness of an individual teacher. In many years of studying and working with performance management practices, we have learned that there is often a conflict over whether the same metrics are used for teacher development or for decisions about employee pay, promotion, or termination. When teachers' careers are at stake, they will naturally feel pressure to deemphasize student learning issues and instead concentrate on simply increasing student test scores by any means. In essence, the state wants student learning but implements a system designed to boost test scores. The focus on outcomes leads to teachers ignoring the very learning that the state wants students to experience.

4 comments:

  1. Andrea Cecconi comments.
    Well spoken. The fight in Minnesota is over this exact issue-- how can an assessment validated (or not) for STUDENT learning outcomes be the primary measure of a TEACHER's behavior? Sadly, this should not be a state-by-state fight. The criteria for RTTT by USDoE are based on the assumption that student test scores are a valid predictor of teacher performance on their own...and the states are buying into it.

    ReplyDelete
  2. It's true that the focus on outcomes distracts teachers from the learning process. Already overburdened by the student-to-teacher ratio, many teachers struggle to give attention where it's needed. Tying their performance to student test results makes this difficult task impossible while compromising the integrity of the entire system.

    Myra Shelley
    Management Software
    http://www.managepro.com

    ReplyDelete
  3. Andrea:

    You raise a very important issue that gets overlooked in any type of testing: validation. The validation process requires time, it requires establishing that what you say your test measures is actually measured. Typically, a validation process takes over a year, and it requires tracking tests scores from the same person over multiple points of time.

    That's not what we typically get with these mandated tests. We do not know how the tests are developed. I personally have never seen a school system test the same student on the same test over time to actually demonstrate that learning has occured (through changes in scores). What I typically see are schools tracking student scores across different grade level tests sporadically over K-12. They don't demonstrate changes over time, only ask to see a certain pass/fail percent achieved. That's not assessment.

    I've seen some of these state tests, and they look like basic knowledge tests to me. I think knowledge and facts are important but don't consititute learning that results from efforts of teachers.

    The states buy into the Fed program because the funding gun is being held to their heads. What choice did Rhode Island have but to accept the national standard? $100M is a huge amount of funding, especially in a cash-strapped state like Rhode Island.

    What is really ironic about this is that the ProJo ran an article yesterday about the dearth of qualified workers in the biotech/science field from Rhode Island. The state just doesn't produce enough skilled scientists. I wonder if these tests will demonstrate that Rhode Island schools can produce educated scientists.

    ReplyDelete
  4. Hi

    Tks very much for post:

    I like it and hope that you continue posting.

    Let me show other source that may be good for community.

    Source: Effective performance appraisal

    Best rgs
    David

    ReplyDelete