I am really looking for an explanation as to why these tests would have sub-group differences? The articles that claim this really don't speculate as to why there is adverse impact in these tests. It is unexpected.
It appears that this tool would be good with social skills related constructs, thus possibly able to be combined with the cognitive ability test because there is lower adverse impact for that construct within work samples as this study found.
How important do you think it is for applicants to have at least a general understanding of a companies policies and procedures before doing a work sample test? Do you think this is something they should be informed about before the test?
While the conclusion was that large discrepancies may exist when using work sample tests for White and Black early stage applicants, I wonder if a better conclusion might be to use work sample tests as a part of a multi-stage selection system? That is, would these results still hold if the work sample tests were applied in a later or final selection stage? Perhaps then we would find fewer racial differences?
It seems this research is coming to the same conclusion that the Bobko, Roth, and Buster (2005) article - (part of an on-going research project it seems, eh? -here adding McFarland) - that work samples that test work dimensions that are more intellectually challenging, show more sub-group differences. To me we are back to the challenge of larger systemic change in our society to better address educational inequalities.
OK So this article did a good job of helping be understand how work sample tests are unique from situational essays and assessment centers. Do you think some of the limitations of work sample tests are worth their high face validity and positive applicant reactions?
The the authors addressed differences in reliability estimates on which corrections were based in prior studies. Was anyone else surprised that measures of reliability for subjective measures were lower than those of objective measures of performance? I was under the impression that objective measures were less reliable due to contamination.
I am really looking for an explanation as to why these tests would have sub-group differences? The articles that claim this really don't speculate as to why there is adverse impact in these tests. It is unexpected.
ReplyDeleteIt appears that this tool would be good with social skills related constructs, thus possibly able to be combined with the cognitive ability test because there is lower adverse impact for that construct within work samples as this study found.
ReplyDeleteHow important do you think it is for applicants to have at least a general understanding of a companies policies and procedures before doing a work sample test? Do you think this is something they should be informed about before the test?
ReplyDeleteWhile the conclusion was that large discrepancies may exist when using work sample tests for White and Black early stage applicants, I wonder if a better conclusion might be to use work sample tests as a part of a multi-stage selection system? That is, would these results still hold if the work sample tests were applied in a later or final selection stage? Perhaps then we would find fewer racial differences?
ReplyDeleteIt seems this research is coming to the same conclusion that the Bobko, Roth, and Buster (2005) article - (part of an on-going research project it seems, eh? -here adding McFarland) - that work samples that test work dimensions that are more intellectually challenging, show more sub-group differences. To me we are back to the challenge of larger systemic change in our society to better address educational inequalities.
ReplyDeleteOK So this article did a good job of helping be understand how work sample tests are unique from situational essays and assessment centers. Do you think some of the limitations of work sample tests are worth their high face validity and positive applicant reactions?
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThe the authors addressed differences in reliability estimates on which corrections were based in prior studies. Was anyone else surprised that measures of reliability for subjective measures were lower than those of objective measures of performance? I was under the impression that objective measures were less reliable due to contamination.
ReplyDelete