It seems that a major underlying challenge of using the Angoff method to set cutoff scores is how the minimally competent performer is defined. How do organizations typically define minimally competent performers? What resources do they have to utilize when developing this definition? Is there any way we can determine if the “obtained” definition of minimally competent performers is actually related to the “true” definition of minimally competent performers?
•The article mentioned how a company can enhance the performance of their workforce by raising the cutoff scores, but that the current employees must meet the new cutoff score(s). How does a company determine if this is necessary and then how do they assure that the current employees can pass the new cutoff? Also, how does a company avoid adverse impact by raising their cutoff score?
I found this article helpful as I prepare to interview student staff who have applied for positions in my office next year. It helped me reflect on the subjective nature of the approach we use. I realize that I need to more clearly identify job relevant KSAOs that might predict job success and do more to ‘train my judges’ (my full-time staff that is involved in interviewing and deciding. I realize we operate with some minimal proficiency requirements, but these are not well based in clear criteria.
I appreciated the description in the second study of getting a large number of judges as part of getting the buy-in of having women compete for the firefighter positions. This article gave me a sense of how young this science is. I was struck by the comment that there is not much evaluation research in organizational settings so recommendations for practice are hard to give (though they had lots of good ones.) Which did you find helpful?
Isn't there an issue with using the minimally competent person as a starting point for a cutoff? Isn't selection also about selecting talented individuals not just competent ones? It would seem that companies should not expected to select simply competent workers at the expense of talented ones? Why has the Angoff method received such legal acclaim?
So in this method, the judges are allowed to discuss their ratings before actually sticking with one? Seems shady. Could this be the reason for such high inter-rater agreement? I’m concerned. They mentioned that this had no influence in study 1 but had not data to back it up... Any thoughts?
How correct do you think SMEs evaluations of what is minimally needed for a job are? It seems to me that SMEs (people who probably hold the position now) would be biased toward setting these minimal thresholds high.
It seems that a major underlying challenge of using the Angoff method to set cutoff scores is how the minimally competent performer is defined. How do organizations typically define minimally competent performers? What resources do they have to utilize when developing this definition? Is there any way we can determine if the “obtained” definition of minimally competent performers is actually related to the “true” definition of minimally competent performers?
ReplyDelete•The article mentioned how a company can enhance the performance of their workforce by raising the cutoff scores, but that the current employees must meet the new cutoff score(s). How does a company determine if this is necessary and then how do they assure that the current employees can pass the new cutoff? Also, how does a company avoid adverse impact by raising their cutoff score?
ReplyDeleteI found this article helpful as I prepare to interview student staff who have applied for positions in my office next year. It helped me reflect on the subjective nature of the approach we use. I realize that I need to more clearly identify job relevant KSAOs that might predict job success and do more to ‘train my judges’ (my full-time staff that is involved in interviewing and deciding. I realize we operate with some minimal proficiency requirements, but these are not well based in clear criteria.
ReplyDeleteI appreciated the description in the second study of getting a large number of judges as part of getting the buy-in of having women compete for the firefighter positions. This article gave me a sense of how young this science is. I was struck by the comment that there is not much evaluation research in organizational settings so recommendations for practice are hard to give (though they had lots of good ones.) Which did you find helpful?
Isn't there an issue with using the minimally competent person as a starting point for a cutoff? Isn't selection also about selecting talented individuals not just competent ones? It would seem that companies should not expected to select simply competent workers at the expense of talented ones? Why has the Angoff method received such legal acclaim?
ReplyDeleteSo in this method, the judges are allowed to discuss their ratings before actually sticking with one? Seems shady. Could this be the reason for such high inter-rater agreement? I’m concerned. They mentioned that this had no influence in study 1 but had not data to back it up... Any thoughts?
ReplyDeleteHow correct do you think SMEs evaluations of what is minimally needed for a job are? It seems to me that SMEs (people who probably hold the position now) would be biased toward setting these minimal thresholds high.
ReplyDelete