The Science Behind Effective Hiring: Insights from Test Partnership
An interview with Oliver Savill, Founder of Test Partnership
Q: Could you start by telling us about Test Partnership? What does the company do, and what inspired you to start it?
When I was applying for my first graduate job, I had to take a lot of psychometric tests. Numerical reasoning and verbal reasoning were the most common. I wrote some practice questions to help myself prepare, which I think helped because eventually, I received a few job offers, and I started working as a graduate engineer.
But in the evenings, I would work on a website that hosted these practice questions, and it became really popular. I added lots more questions, and eventually, the website started making money.
So I decided to quit my engineering job and go full-time selling practice tests. I hired an occupational psychologist and a few other people to help me, and we were getting hundreds of people taking these tests every day.
We soon realized we could use this data to trial and develop ‘real’ questions. It was quite exciting because we had so much data to work with. We’d post new questions online and very quickly have participant data on difficulty, validity, reliability, demographic data…everything we needed to develop a suite of robust assessments companies could use.
That’s how we’ve continued to develop so many different assessments and work on innovations such as AI cheat-proof tests (more on that later).
Q: Many people compare assessments to taking exams like SATs in the U.S. or college entrance exams in Europe. Is that a fair comparison, and is it what they're designed to do?
Both types of assessment measure forms of intelligence, but there are some important differences between assessments in occupational psychology and exams in education.
Assessments in an occupational setting are typically much shorter, as we can’t expect candidates to complete two hours worth of testing for every job. Additionally, there is a much bigger emphasis on candidate experience, as organizations don’t want to put off candidates from applying.
Occupational assessments tend to be more convenient and candidate-friendly than entrance exams. Candidates can usually complete them on their own time and from home rather than in a testing center, as is often the case with entrance exams.
Another important difference is what they’re trying to measure. The goal of occupational assessments is to help identify the highest potential candidates early on in the recruitment process so that the assessments will correlate with job performance.
SATs and entrance exams are more measures of what someone has remembered or their IQ rather than a measure that correlates with work performance.
Q: Sixteen years ago, I had my first experience with assessments when I applied for a job. When I got hired, I was told that the assessments were new, and everyone, even the existing employees, had to take them.
Surprisingly, the results showed that none of the current employees would have gotten hired if the assessments had been used in their hiring process. Do assessments truly predict whether a candidate will be a good hire?
There are a few interesting things going on here. We often find that when we test incumbent employees, they don’t score as highly as eager job applicants. This is because hopeful candidates, who literally have a job on the line, are dedicating 100% of their cognitive effort to performing, and it really shows in the data.
Incumbents, who complete these assessments usually just because they’re asked to, are more interested in assessing the test itself (they’re thinking, “What is this measuring?” and “How does this fit in with me getting my work done?”). They look at the layout, format, and structure and decide whether or not they like the assessment. Consequently, this divided attention always lowers their scores compared to candidates.
The other thing to probe here is: how good were those existing employees? My guess is that those hired without a robust assessment process were average performers. They were typical of the population, which is fine. However, many organizations want to hire above-average performers, and that’s where objective data from cognitive ability tests or personality questionnaires can give organizations an edge.
There’s a lot of randomness and luck involved in hiring, but if you can nudge the odds of a good hire in your favor, collectively the organization is better off. Maybe you were their best hire at that stage ;)
Stepping back and looking at the science behind cognitive ability tests, we have almost 100 years of evidence in occupational psychology showing that they are the best predictor of job performance.
An enormous body of research shows that cognitive ability -the ability to learn, solve problems, and make decisions- predicts workplace performance more effectively than any other known tool, particularly in complex technical, managerial, and professional work.
This may come as a surprise to people who are new to this body of research, but to occupational psychologists and psychometricians, it's just common knowledge.
That being said, they aren’t perfect predictors of performance, and there are many factors that also predict a candidate's likely job performance. This is why you can’t use ability tests and nothing else, as you would be leaving a great deal on the table.
However, for organizations that use only interviews to select candidates, you are leaving even more on the table by relying solely on interviews and ignoring ability tests, particularly in highly complex white-collar work. So they’re not perfect, but they’re the best we have.
Q: How is AI changing the world of assessments? Do you think we'll see more or fewer assessments in the future?
Right now, the world of assessment is in flux, with providers desperately trying to find their feet as the ground shifts beneath them.
On one hand, AI has presented a significant opportunity for assessment providers. As effective as AI cheating may be for some assessments (we’ll come to that in a bit), it has completely killed CV and application form sifting as a shortlisting tool.
Written content is more cheatable than anything. They are literally called “Large Language Models” (LLMs), so you can safely assume that every CV you read from now on will be AI-generated or curated.
Because LLMs allow you to massively scale up CV and application form submission, organizations are inundated with applications, as candidates can now apply for hundreds of jobs easily. So the need for robust assessments has never been higher, and in my view will only increase in the coming years because other screening tools have overnight become redundant.
However, AI is being used to cheat assessments too, but not equally. The most vulnerable type of assessment is knowledge tests (sometimes called hard skills tests), where the correct answer is public knowledge. For example something like “What feature of Adobe Photoshop allows you to highlight a free-form shape?” AI tools excel at scouring the internet and finding the answer, making these types of assessments redundant.
Traditional aptitude tests, such as verbal, numerical, and logical reasoning tests, are less vulnerable, but this will change as AI models improve. We are quickly reaching the state where AI can match the performance of the average candidate, after which point most organizations’ approaches to traditional online assessments will have to change.
The solution to AI cheating assessments lies in dynamic, interactive elements—components that an AI tool can’t interact with, like drag-and-drop or time-sensitive positive interaction. We have used these techniques to fight back against AI cheating, with great effect. Here's an example of what we’ve done: https://www.testpartnership.com/gamified-assessment.html
These assessments measure the same aptitudes as traditional ability tests but employ complex game mechanics. These human-centered interactive elements make the task unsuitable for LLMs. We see this as the future for candidate assessment, and it should be strongly considered for any high-volume recruitment campaign.
As I said in the intro, it’s exciting to be able to rapidly develop new assessments, and we’ve had fun developing new tests that guard against the threat of AI.
Overall, I envision the market dropping several outdated types of assessment, but concurrently seeing a proliferation of AI-resistant assessments, which will be used with greater frequency than ever before.
Q: How do you see the career landscape evolving? What advice would you give to early-career or mid-career professionals navigating this period of change?
AI is definitely here to stay, so professionals are well-advised to keep up to date with developments in AI and get comfortable using AI tools in their work. I suspect that those who are most adept with AI are the ones most likely to avoid job losses. So, take some time to get familiar with what's on the market.
Find innovative ways to boost your own productivity with AI and really lean into the change. The future is increasingly hard to predict, but I am hopeful that the net impact of AI will be positive.
I see careers evolving in a way that places less emphasis on knowledge and qualifications (for example, knowing a piece of software or having experience in an industry). Knowing facts will be replaced by AI telling us the facts.
Where AI can’t replace us—yet—is our human personality traits, such as employees’ level of curiosity, drive, empathy, learning agility, conscientiousness, etc. These human attributes will influence our use of assistive tools like AI and may make the difference between a company that thrives and a company that becomes a victim of evolution.
