Could SJT's be used in another method besides a paper and pencil test or video based situations? For instance, actually put the participant in a "real" situation and see how they react, how they interact and deal with people. This might provide incremental validity above the paper and pencil test. How could we do this to make the situation more realistic while keeping costs in mind?
I think that SJTs are an interesting way to measure KSAOs for a job. However, I am a little concerned about the extent to which an individual would need to know how to respond to some of these questions (given that they are job/task related) before undergoing some sort of job-related training. In addition, would SJTs and integrity tests be highly correlated?
How do assessment centers compare to SJTs? I feel that they are similar except the assessment center measures more situational reactions vs. on a test (i.e., hypothetical).
Shane, I think you make a good point. I was wondering about the difference between using an SJT and an inbox task or an leaderless group discussion etc. I think the main difference is the SJT shows how the participant thinks they would respond vs. his/her actual response in the simulation. Also, SJTs are probably less expensive to design and administer. Does it seem that SJTs might be vulnerable to faking similarly to personality tests?
I thought about that too Shay. The Bergman article said that the theoretical keying was more vulnerable to faking because the questions developed were more transparent. I'm not sure how I feel about SJTs I'm looking forward to discussing this in class. They seem like a lot of work, though if they are well-constructed they show incremental validity and could be used for a long time (and show low sub-group differences.) The challenge is in the construction and scoring it seems.
Despite some attention to the construct aspect of SJTs, I find that it is more useful to regard SJTs as a measurement method than a measure of a particular KSAO. Do you think that there would be some benefits to measuring constructs such a personality via the SJT method compared to more traditional methods of measurement? Could you see SJTs being more resistant to faking? Somewhat unrelated – I would also like to see how rating open responses/short answers would make a difference in results.
Could SJT's be used in another method besides a paper and pencil test or video based situations? For instance, actually put the participant in a "real" situation and see how they react, how they interact and deal with people. This might provide incremental validity above the paper and pencil test. How could we do this to make the situation more realistic while keeping costs in mind?
ReplyDeleteI think that SJTs are an interesting way to measure KSAOs for a job. However, I am a little concerned about the extent to which an individual would need to know how to respond to some of these questions (given that they are job/task related) before undergoing some sort of job-related training. In addition, would SJTs and integrity tests be highly correlated?
ReplyDeleteHow do assessment centers compare to SJTs? I feel that they are similar except the assessment center measures more situational reactions vs. on a test (i.e., hypothetical).
ReplyDeleteShane, I think you make a good point. I was wondering about the difference between using an SJT and an inbox task or an leaderless group discussion etc. I think the main difference is the SJT shows how the participant thinks they would respond vs. his/her actual response in the simulation. Also, SJTs are probably less expensive to design and administer. Does it seem that SJTs might be vulnerable to faking similarly to personality tests?
ReplyDeleteI thought about that too Shay. The Bergman article said that the theoretical keying was more vulnerable to faking because the questions developed were more transparent. I'm not sure how I feel about SJTs I'm looking forward to discussing this in class. They seem like a lot of work, though if they are well-constructed they show incremental validity and could be used for a long time (and show low sub-group differences.)
ReplyDeleteThe challenge is in the construction and scoring it seems.
How big of a concern is faking when using SJTs for selection measures?
ReplyDeleteDespite some attention to the construct aspect of SJTs, I find that it is more useful to regard SJTs as a measurement method than a measure of a particular KSAO. Do you think that there would be some benefits to measuring constructs such a personality via the SJT method compared to more traditional methods of measurement? Could you see SJTs being more resistant to faking? Somewhat unrelated – I would also like to see how rating open responses/short answers would make a difference in results.
ReplyDelete