Would synthetic validation be a possible solution to the issue of rapidly evolving jobs without having to recreate an entire validation process based on a new job analysis?
It seems that the inferences necessary to use synthetic validation, such as determining job families and job components, stretch us even further from the relationship of predictor constructs and the measure performance domains for any given job. The authors indicate that there has yet to be a court decision made about synthetic validity. Do you think synthetic validity is a legally acceptable form of validation?
This one might be for Roni to answer - I gather from this reading that synthetic validity is an appropriate approach to validation when the sample size we have for one job of interest is too small to complete a traditional validation study. If sample size/power is not an issue, is better to use traditional validation techniques or a synthetic validation study? I would assume traditional validation techniques, since it seems to require less work than a synthetic validation study. However, if you have several jobs with large sample sizes, is it more feasible to do a synthetic validation study (such as the one we just read) or to use traditional validation methods for each job?
For what categories of jobs is it difficult or impossible to collect necessary data to conduct traditional criterion-related validity study? Do you think the synthetic validation technique will catch on?
I like the idea of synthetic validation because according to the authors, one could have a national database of appropriate measures for certain constructs; however, I do have some concerns. It could be possible that what is meant by problem solving in one context may not be the same in another. How does synthetic validation take into account the contextual (e.g., environmental or organizational) differences between jobs? Or do the authors simply chalk those differences up to error? Is that a correct way to handle these differences?
Also I agree with Maria Teresa in that synthetic validation would seem to work better for certain jobs. For example, I would imagine this to more useful in entry level jobs but perhaps less so as you move up the organizational hierarchy.
I have the same speculation as Shay does in that the validation process used in this article/study might be an appropriate approach to the continuous changes in the job, company, and hosted culture of the job site. Although, I have the same concerns as Shane because recent reading have strongly expressed the importance of contextual factors and performance.
Would synthetic validation be a possible solution to the issue of rapidly evolving jobs without having to recreate an entire validation process based on a new job analysis?
ReplyDeleteIt seems that the inferences necessary to use synthetic validation, such as determining job families and job components, stretch us even further from the relationship of predictor constructs and the measure performance domains for any given job. The authors indicate that there has yet to be a court decision made about synthetic validity. Do you think synthetic validity is a legally acceptable form of validation?
ReplyDeleteThis one might be for Roni to answer - I gather from this reading that synthetic validity is an appropriate approach to validation when the sample size we have for one job of interest is too small to complete a traditional validation study. If sample size/power is not an issue, is better to use traditional validation techniques or a synthetic validation study? I would assume traditional validation techniques, since it seems to require less work than a synthetic validation study. However, if you have several jobs with large sample sizes, is it more feasible to do a synthetic validation study (such as the one we just read) or to use traditional validation methods for each job?
ReplyDeleteWhat factors may contribute to how well synthetic validation works for a company?
ReplyDeleteFor what categories of jobs is it difficult or impossible to collect necessary data to conduct traditional criterion-related validity study?
ReplyDeleteDo you think the synthetic validation technique will catch on?
I like the idea of synthetic validation because according to the authors, one could have a national database of appropriate measures for certain constructs; however, I do have some concerns. It could be possible that what is meant by problem solving in one context may not be the same in another. How does synthetic validation take into account the contextual (e.g., environmental or organizational) differences between jobs? Or do the authors simply chalk those differences up to error? Is that a correct way to handle these differences?
ReplyDeleteAlso I agree with Maria Teresa in that synthetic validation would seem to work better for certain jobs. For example, I would imagine this to more useful in entry level jobs but perhaps less so as you move up the organizational hierarchy.
Maybe I missed the point, but could synthetic validation methods lead to a misuse of "off the shelf" selection systems?
ReplyDeleteI have the same speculation as Shay does in that the validation process used in this article/study might be an appropriate approach to the continuous changes in the job, company, and hosted culture of the job site. Although, I have the same concerns as Shane because recent reading have strongly expressed the importance of contextual factors and performance.
ReplyDelete