Runner up in the Indigogold Work Psychology Innovation Award 2015, Joel Ockwell, from the University of Sheffield, explains the inspiration behind his MSc project.
How can managers and HR professionals identify whether a candidate for recruitment or promotion is likely to perform well in their proposed role? How do they ensure that the assessment being carried out produces accurate results?
Situational judgement tests (SJTs) are psychometric tests that assess someone’s ability to identify the most effective course of action in a theoretical workplace situation, and these have a long track record of predicting workplace performance reasonably well. But any test is only as good as its design, and a critical aspect of SJTs is how responses are scored.
Worryingly, current research on the effectiveness of scoring methods is inconclusive, and current methods of scoring SJTs are unable to consider the effectiveness of different responses to a situation.
Working with Pearn Kandola, the business psychology consultancy, my study developed, tested and analysed an innovative method of scoring SJTs using data from legal and public sector organisations. The validity and diversity implications of the new scoring method were compared to other, more commonly used methods, and we also developed multiple versions of the new scoring method using insights provided by different subject matter experts, in order to identify what insights each group brought to a job.
The findings from the study showed that a new correlation scoring method is a more valid measure of overall ability. The results also provided an indication that incumbents are more likely to identify behaviours that lead to higher overall ability, whilst supervisors rate behaviours more effectively if they are good examples of competencies measured in assessment centres.
The findings have implications for practitioners and organisations alike. Scoring SJTs using a correlation method is likely to increase the validity of SJTs in selection processes, and when practitioners develop SJTs based on job analyses, role incumbents should be consulted to identify the indicators for high performance.
I was delighted to be selected as a finalist for the Indigogold Work Psychology Innovation Awards, and enjoyed being able to share my work with academics and practitioners at the follow-up Indigogold networking event. I am currently working as a business psychologist at PwC, and I hope to be able to use my research and knowledge on selection and assessment practices here in the future.