Rating the Rater: The annual game of Show and Tell

Written by Marcela Presecan on April 13, 2020. Posted in Articles, Individual Performance

While some companies spend decades on perfecting their individual performance review and evaluation systems, other companies such as Adobe, GAP or General Electric are eliminating them altogether.There is something about an annual performance review that takes both employee and manager out of their comfort zone. Moreover, there are also the inadvertent sturdy biases and the idiosyncratic rater effect that make individual performance reviews seem rather useless.

The interesting thing is that most of the inefficiency of an annual performance review comes with the aforementioned idiosyncratic rater effect: how much of what is being measured is the rating tendency of the rater. This means that without any context, the rater will provide a rating based on his perception of the rated element (e.g. performance, competency, behavior).

The shocking fact revealed through research is that 75% of the individual data collected through performance reviews is biased due to measurement errors, idiosyncratic effects and rater perspective. This leaves a mere 25% of data that really reflects on the ratee, which, in turn, means that individual performance reviews actually reveal more about the rater than the ratee.

So, how reliable is this individual annual performance review if it’s so biased and inconclusive? What is the acceptable amount of bias and subjectivity that rates an employee on a scale from 1 to 5, where 1 is “Unsatisfying” and 5 is “Outstanding”? Do we rate talent according to our own perception of talent? Do we rate high potential based on our own subjective definition of high potential? And how exactly do we configure a scale to measure talent or high potential, for that matter?

There are several instruments on which organizations generally base their annual performance evaluations on:

1. The Individual Scorecard: performance management tool that states individual objectives and uses KPIs to measure and monitor their achievement;
2. The Competencies List: list of attributes necessary to successfully fulfill a specific role and their desired performance level; competencies are rated on a scale of 1 to 5 for each competence;
3. Desired behavioral commitment: assesses the individual’s commitment to the company’s organizational values through behavior exhibited while at the workplace; behavior is rated on a scale of 1 to 5 for each type;

   ---

While the individual scorecard secures a measurable, objective evaluation of the individual, a subsequent assessment of an individual’s competencies and behavior will almost certainly be biased, as it mostly relies on the perception and tendency of the rater. Some of the multiple biases that research has encountered with performance ratings can be summarized as follows:

• The halo effect: the tendency to rate an individual using the same score, or a similar score, on all categories
• Similarity: the tendency to over-rate an individual due to similarities unrelated to performance (e.g. age, sex, gender, race)
• Central tendency: the tendency to assign average ratings, independent of the actual performance of the ratee (e.g. a general score of 3, on a scale from 1 to 5)
• Leniency: the tendency to give favorable ratings on an individual’s performance, regardless of the actual performance of the rated person
• Sternness: the tendency to assign unfavorable ratings, independent of the actual performance of the ratee
• Fear of retribution: tendency to give favorable ratings to individuals, out of fear of punishment

  ---

The accuracy of the rater’s feedback depends on other multiple factors such as:

• The rater’s familiarity and comfort with the performance review process
• The opportunity raters have to observe the ratee
• The time spent observing the ratee
• Liaison with the ratee on work-related tasks
• Degree of friendship between the rater and the ratee

  ---

We have rendered practices of monitoring past performance obsolete and praised the emergence of leading KPIs to push forward all that’s lagging in the performance of our companies. We have condemned organizations for searching performance in financials only, and we preached the use of performance management tools that look for multi-dimensionality in business success. We sponsored the achievement of greatness by encouraging people who embrace continuous learning and growth.

So why don’t we fully commit to our progressive ways and also look for the future in the people that we build our organizations upon? Why do we keep making them bow to a meager 1 to 5 scale, which is more biased than bias itself? Organizations who have dropped the ineffective ways of individual assessments based on annual performance appraisals stand to give us a lesson. And here are some of alternatives they suggest:

• Assessment of future intentions instead of past performance
• Regular check-in conversations to replace the annual “review”
• Focusing on people’s strengths instead of rating and perfecting their weaknesses
• Big data for real-time feedback on individual performance
• Cloud solutions for 360-degree reviews
• Opportunities to improve instead of issues to correct
• Total elimination of costly forms and useless questionnaires

  ---

So, to all the companies who say they have embraced the new ways in managing performance for success, be dared: Do you have the audacity to go all the way?

Image source:

• Pixabay
• Pixabay

Tags: Employee Evaluation, Individual Performance, Performance review