Different Environments

The Illusion of General Validity: Why “One-Size-Fits-All” Tests Mislead in Recruitment

Can a single test capture the complexities of human potential?     It’s time to rethink how we measure talent for real-world success.

Bridging Science and Strategy in Talent Selection

Recruitment is often torn between two seemingly irreconcilable worlds: the scientific idealism of cognitive testing and the practical realities of human resource management. On one hand, scientists present compelling data that general intelligence, or “G,” stands as the most reliable predictor of job performance across roles and contexts [1], [2]. On the other, HR-practitioners, entrenched in the complex realities of hiring, often find such claims simplistic, unable to account for the multifaceted nature of job success [3], [4]. While general intelligence does indeed transfer across contexts [2], [5]–[7], helping bright minds succeed in various domains, the broader truth is far more nuanced. Intelligence, for all its predictive power, explains far less of the variation in job performance than one might hope [8]. 

Too some, it may seem comforting and convenient to rely on a single test to predict future success, but in practice, the one-test-fits-all approach mediates only a fraction of the risks involved in making hiring decisions. After all, what is the point of a measure that offers only vague insights into potential when what we really need is a precise understanding of how a candidate will thrive within the specific demands of our organizational environment?

One-Size-Doesn’t-Fit-All: Rethinking Standardized Testing

Test companies, aligned with the scientific consensus, offer tools that –allegedly– have undergone rigorous validation processes. These standardized assessments, somewhat supported by empirical research, are often marketed as comprehensive solutions for data-driven recruitment. Yet, there is an inherent overreach in these claims. Test producers too often push the idea that a single measure of general intelligence is sufficient for every hiring decision, perpetuating the illusion that this general validity applies universally. 

The crux of the problem lies here: while a test may be valid in a particular context, assuming its conclusions hold true across all environments is both misleading and dangerous. The “one-size-fits-all” approach to cognitive testing falls short because it fails to capture the complexities of real-world performance across varied roles and settings. This reliance on general intelligence not only oversimplifies the hiring process but also introduces unnecessary risk—risk that could easily be mitigated by a more refined approach.

Today’s advances in cognitive neuroscience make this overreliance even more puzzling. We now have the tools to probe deeper into human cognition and explore a wide array of specific abilities [9]—working memory, attention control, problem-solving—each offering insights far more relevant to the unique demands of individual roles [10]. It’s not that standardized tests are without merit, but relying on a singular, overarching measure of general intelligence seems increasingly crude when compared to the richness of what we now know about the brain. 

The solution lies in recalibrating our focus. We could shift from measuring broad intelligence to investigating narrower cognitive abilities that better predict success in specific contexts. The relevance of each cognitive ability can differ significantly depending on the task. 

From Generic to Genious: Targeted Insights for Tomorrow’s Talent

The challenge, then, is not simply one of measurement but of context. Different roles, industries, and organizational cultures demand different cognitive strengths. The question is no longer just whether someone is “intelligent” in a general sense, but which cognitive abilities are most crucial for success in a particular environment. By adopting a more refined, context-specific approach, businesses can reduce the risks of erroneous recruitments and ensure a tighter alignment between cognitive skills and the specific demands of the job. This requires moving beyond the notion that one test can predict all outcomes, toward a more sophisticated understanding of human potential, when and where she is – hopefully – performing.

In doing so, we not only dispel the illusion of general validity but also shift toward a smarter, more context-driven approach to recruitment. This approach acknowledges not only the complexity of cognitive abilities but, more importantly, the specific demands of each organization. Rather than fixating on the tests themselves or their supposed general applicability, the key lies in understanding the unique predictors of success within each company’s particular environment. By studying what truly drives performance in a given role businesses can adopt a more insightful, tailored method of hiring. This focus on what really matters in specific contexts allows companies to make more informed, data-driven decisions, reducing uncertainty and significantly increasing the likelihood that selected candidates will thrive and contribute meaningfully to the organization.

Alex bild

Alexander Klaréus

Head of Insight
References

 

[1]    J. E. Hunter, “Cognitive ability, cognitive aptitudes, job knowledge, and job performance,” J. Vocat. Behav., vol. 29, no. 3, pp. 340–362, Dec. 1986, doi: 10.1016/0001-8791(86)90013-8.

[2]    F. L. Schmidt, “The role of general cognitive ability and job performance: why there cannot be a debate,” HHUP, vol. 15, no. 1–2, pp. 187–210, Apr. 2002, doi: 10.1080/08959285.2002.9668091.

[3]    C. Gill, “Don’t know, don’t care: An exploration of evidence based knowledge and practice in human resource management,” Human Resource Management Review, Jun. 2017, doi: 10.1016/j.hrmr.2017.06.001.

[4]    S. D. Risavy, C. Robie, P. A. Fisher, J. Komar, and A. Perossa, “Selection tool use in Canadian tech companies: Assessing and explaining the research–practice gap.,” Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement, vol. 53, no. 4, pp. 445–455, Oct. 2021, doi: 10.1037/cbs0000263.

[5]    F. L. Schmidt and J. E. Hunter, “The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.,” Psychol. Bull., vol. 124, no. 2, pp. 262–274, 1998, doi: 10.1037/0033-2909.124.2.262.

[6]    J. E. Hunter, “Test validation for 12, 000 jobs: An application of synthetic validity and validity generalization to the GATB,” US Employment Service, US Department of Labor, 1983, 1983.

[7]    F. L. Schmidt and J. Hunter, “General mental ability in the world of work: occupational attainment and job performance.,” J. Pers. Soc. Psychol., vol. 86, no. 1, pp. 162–173, Jan. 2004, doi: 10.1037/0022-3514.86.1.162.

[8]    P. R. Sackett, S. Demeke, I. M. Bazian, A. M. Griebie, R. Priest, and N. R. Kuncel, “A contemporary look at the relationship between general cognitive ability and job performance.,” J. Appl. Psychol., vol. 109, no. 5, pp. 687–713, May 2024, doi: 10.1037/apl0001159.

[9]    A. Diamond, “Executive functions.,” Annu. Rev. Psychol., vol. 64, pp. 135–168, 2013, doi: 10.1146/annurev-psych-113011-143750.

[10]   C. D. Nye, J. Ma, and S. Wee, “Cognitive Ability and Job Performance: Meta-analytic Evidence for the Validity of Narrow Cognitive Abilities,” J. Bus. Psychol., vol. 37, no. 6, pp. 1119–1139, Dec. 2022, doi: 10.1007/s10869-022-09796-1. 

 
Screenshot 2024-02-27 at 3.43.36 PM

A Psychodynamic Perspective of Organizational Life

So, what do we really mean when we use the term psychodynamic? And how can we apply this perspective to the world of business?

Well, to put it simply, psychodynamic perspective has its roots in a set of theories that explain the unconscious mind, initially founded by the Austrian physician Sigmund Freud. In other words, the psychodynamic perspective helps us to understand unconscious psychological processes (for example wishes, fears and impulses) and explores how our current feelings and behavior are shaped by past experiences. In this piece of article, we will highlight a couple of important psychodynamic concepts in order to help you appreciate the relevance of this approach in relation to a business- and organizational context. We start with the maybe most well known one.

Unconscious slips

Freud, along with his followers, discovered that there are parts of our mental life that are hidden, which he referred to as the unconscious. However, he realized that it was difficult for people to accept the unconscious and noticed how there was a form of resistance to it, so he believed it could be accessed by attending to dreams, slips of tongue, and mistakes. These forms of “Freudian slips” were perceived as valuable means of communication that were beyond our awareness. Therefore, Freud recognized the importance to make an interpretation of these symbolic expressions.

But why is it even important to consider the unconscious mind through symbolic expressions?

Well, because it can impact our communication, reaction, and consequently business performance. The unconscious mind can help us understand ourselves and others. It can process beyond the surface and read what’s in between the lines. Attending to the unconscious can therefore give us invaluable insight into relational dynamics at work and in organizations.

From a psychodynamic perspective, communications can have two layers of meaning: While we may say something at the conscious level, we may communicate a hidden meaning unconsciously. As an example, when you hear a group of employees talking about their problems with the AC at work. On one level they may refer to the manager’s difficulty to follow through on his/her promises, at the same time, they may be making an unconscious reference to managers inability to ”fix and cool down” internal conflicts.

Projection as defense – a biased protection from your emotions

The unconscious also uses defenses as a way of protecting itself from painful emotions and external threats. In business context, these emotions could be a consequence of a conflict between different departments or managers and employees being in a competitive struggle with one another.

Having said this, defenses are not always bad or destructive, in fact, they can be quite constructive in that they allow the staff to grow and develop within the organization. But as you figure, there are times when defenses are harmful, resulting in employees leaving the company and/or preventing the organization from growth. As an example, we can see that institutions also develop defense mechanisms as a way of protecting themselves from painful and threatening emotions. One such defense is projection.

Projection is a form of defense that we use by locating unwanted feelings in others rather than in oneself because its simply too painful to acknowledge the ”badness” in oneself. According to Halton (2019), projection of feelings of badness outside the self helps to produce a state of falls goodness and self-idealization.

We end this article with a few scientific conclusions from Petriglieri and Stein (2012), and three things for you to bring along. Petriglieri and Stein discuss how individuals who work as top-level managers can engage in a process, whereby they unconsciously project unwanted aspects of themselves into others, in order to shape and sustain an identity that would fit that particular role. The recipients of the leader’s projections may then project these back at the leader, which can develop a harmful culture and intensify ongoing conflicts – which of course we want to avoid.

Here are three things to keep in mind dealing with projective behaviors:

1. Set aside time for reflection of your own and others’ expressed opinions.

From your recent conversation or digital meeting, try to pinpoint moments where you or someone acted defensive or reflected negative about others – was it a projection?

2. Support those in your surrounding who are under high visibility.

Take away pressure from exposed persons to reduce the risk of expressing negative projection.

3. Leader and follower – view it from both perspectives.

Understand and address both the transmitter’s and receiver’s perspective to see the gap between the leader’s capabilities and follower’s expectations.