This is a practical study about an issue of measurement -regarding what is being measured with retesting. This leads me to wonder about how long policy and practices take to change. How much would this research need to replicated in order for guidelines and practice to be changed -e.g. not allow retesting?
•How does this apply to the use of standardized IQ tests that allow people to retake the test? We all know that we perform better partly because we are more familiar with the method of the test and maybe with less test anxiety the 2nd time. This article to me states that we should not let people retake any tests that are measuring general ability; otherwise the test is not accurate in predicting anymore.
The authors brought up the idea of fairness perceptions of individuals being able to retake a test. This might make an applicant feel that the organization is committed to procedural justice, whereas job incumbents might question the procedural justice of test-retaking, especially if these incumbents were hire without needed to retake an exam. How do organizations typically handle test-retaking? Is it commonly done? Based on the results of this particular study, what implications might test-retaking have for the criterion-related validity of a predictor?
Wouldn't this only apply if you are using the same test batteries? If one is using alternate forms or a computer adaptive test (e.g., GRE) this would seem less likely of an issue.
Also, the authors said "given the prevalence of test re-taking in selection practices..."; however, this seems to only be the case in academic circumstances. Do organizations really take this same view of retesting for selection purposes? Anecdotal evidence to the contrary: At TD Ameritrade, financial specialists are hired with sole understanding that they are to pass their series 66 and 6 within 6 months of hire. If not, they are immediately terminated. I'm not sure if this is the norm.
The authors using the example of the GRE reminds me that schools can see all of the student's scores for each time they have taken the test. Although schools are only supposed to pay attention to the highest score, it is hard to believe they aren't at least partly influenced by lower scores (especially if they are dramatically lower). Why do you think ETS shows all of the scores. What university level factor might effect how much a school is able to "ignore" those other scores?
It was mentioned that the people who had to retake the test to obtain a good enough score did not perform as well in medical school as those people who obtained a good enough score the first time. I wish the authors would have elaborated on this more. I would be curious to know if there are any mediating factors that play a role in their performance (those people who had to take the test for a second time). Perhaps these people might need extra time to pick up on a concept, but once this happens, are better at applying in once they gain experience. Basically, I’m curious about the predictability of performance for people who need to take a test multiple times in order to obtain a good enough score…
Lievens et al. found that a test of general reasoning ability demonstrated diminished predictive validity on subsequent administrations to the same individuals, suggesting that allowing individuals to retake GMA tests is not helpful in making selection decisions. Was anyone else surprised, however, to see that the correlation between the test and GPA for the lower scoring group was nonsignificant in the first administration and the strength of the correlation was half that of the high scoring group? Does this seem to undermine predictive validity on GMA tests across even relatively high cognitive ability levels since the participants were all students applying to medical and dental school?
This is a practical study about an issue of measurement -regarding what is being measured with retesting. This leads me to wonder about how long policy and practices take to change. How much would this research need to replicated in order for guidelines and practice to be changed -e.g. not allow retesting?
ReplyDelete•How does this apply to the use of standardized IQ tests that allow people to retake the test? We all know that we perform better partly because we are more familiar with the method of the test and maybe with less test anxiety the 2nd time. This article to me states that we should not let people retake any tests that are measuring general ability; otherwise the test is not accurate in predicting anymore.
ReplyDeleteThe authors brought up the idea of fairness perceptions of individuals being able to retake a test. This might make an applicant feel that the organization is committed to procedural justice, whereas job incumbents might question the procedural justice of test-retaking, especially if these incumbents were hire without needed to retake an exam. How do organizations typically handle test-retaking? Is it commonly done? Based on the results of this particular study, what implications might test-retaking have for the criterion-related validity of a predictor?
ReplyDeleteWouldn't this only apply if you are using the same test batteries? If one is using alternate forms or a computer adaptive test (e.g., GRE) this would seem less likely of an issue.
ReplyDeleteAlso, the authors said "given the prevalence of test re-taking in selection practices..."; however, this seems to only be the case in academic circumstances. Do organizations really take this same view of retesting for selection purposes? Anecdotal evidence to the contrary: At TD Ameritrade, financial specialists are hired with sole understanding that they are to pass their series 66 and 6 within 6 months of hire. If not, they are immediately terminated. I'm not sure if this is the norm.
The authors using the example of the GRE reminds me that schools can see all of the student's scores for each time they have taken the test. Although schools are only supposed to pay attention to the highest score, it is hard to believe they aren't at least partly influenced by lower scores (especially if they are dramatically lower). Why do you think ETS shows all of the scores. What university level factor might effect how much a school is able to "ignore" those other scores?
ReplyDeleteIt was mentioned that the people who had to retake the test to obtain a good enough score did not perform as well in medical school as those people who obtained a good enough score the first time. I wish the authors would have elaborated on this more. I would be curious to know if there are any mediating factors that play a role in their performance (those people who had to take the test for a second time). Perhaps these people might need extra time to pick up on a concept, but once this happens, are better at applying in once they gain experience. Basically, I’m curious about the predictability of performance for people who need to take a test multiple times in order to obtain a good enough score…
ReplyDeleteLievens et al. found that a test of general reasoning ability demonstrated diminished predictive validity on subsequent administrations to the same individuals, suggesting that allowing individuals to retake GMA tests is not helpful in making selection decisions. Was anyone else surprised, however, to see that the correlation between the test and GPA for the lower scoring group was nonsignificant in the first administration and the strength of the correlation was half that of the high scoring group? Does this seem to undermine predictive validity on GMA tests across even relatively high cognitive ability levels since the participants were all students applying to medical and dental school?
ReplyDelete