The Psychology of Personnel Selection (II)

Here’s my first post about the book.

The second half of the book was about psychological constructs used for personnel selection. The constructs included in the coverage are: IQ/general mental ability, personality traits, creativity, leadership, and talent. Some of these chapters were better than others, and I actually talked a bit about some of the parts of the book which I had no intention of covering here elsewhere – go have a look at the comments I made in that thread if you want to know more about what kind of stuff is included in the book. They note in the introduction to chapter 10 that: “it remains unclear what talent actually is, whether it needs special nurturing to last and what it predicts” – and as far as I’m concerned, they could actually have stopped right then and there and this would have been perfectly fine. They didn’t, though.

I was far more interested in the stuff covered in the first couple of chapters in the second half than I was in the other stuff, and in this post I’ll restrict the coverage to the IQ/mental ability chapter and the personality trait chapter – if you’re curious to know more about what kind of stuff is covered in the last few chapters of the book, I again refer to the comments I made in the MR thread to which I link above. Despite the fact that I only spend time here on the first two chapters of the second half, the authors spend a combined 80 pages on these two topics, whereas the last three topics get a combined total of 60 pages – so I’m actually covering a big chunk of the remaining material even though I don’t talk about leadership, talent or creativity here.

I liked the book, but some parts of it were much better than others and the three star rating I gave it on goodreads is sort of a compromise rating; in this post I’ve mostly covered stuff I liked and/or found interesting. If you’re interested in this kind of stuff it’s not a bad book, but if you aren’t you probably don’t need to read it. The book never did get around to talking all that much about the interaction effects I talked about in the first post (i.e. ‘method interactions’), but as you might be able to tell from the coverage below there was significantly more focus on these aspects during the second half of the book than there was during the first half, which was nice.

Some observations from the book below and a few comments.

…

“To say that GMA [‘General Mental Ability’] predicts occupational outcomes, such as job or training performance, is as much a truism as an understatement, and is really beyond debate […] Indeed, there is so much evidence for the validity of GMA in the prediction of job and training performance that an entire book could be written simply describing these findings. There are several great and relatively compact sources of reference […] The predictive power of GMA at work is rivalled by no other psychological trait […] That said, GMA should not be used as single predictor of job performance as some traits, notably Conscientiousness and Integrity […], have incremental validity over and above GMA, explaining additional variance in occupational outcomes of interest […] The validity of GMA at work has been documented quite systematically since the end of World War I […] average GMA levels tend to increase with occupational level, that is, with the prestige of the job […] Furthermore, higher-level jobs tend to have substantially higher levels of inbound GMA, indicating that it is far more unlikely to find low-IQ scorers in high-level professions than it is to find high-IQ scorers in low-level professions. […] the standard deviations tend to decrease as people move up to higher-level professions, showing that these jobs tend to have not only people with higher but also more homogeneous GMA […] In a colossal quantitative review and meta-analysis of 425 studies on GMA and job performance across different levels of complexity (Hunter, 1980; Hunter & Hunter, 1984), typically referred to as ‘validity studies’, GMA was found to correlate significantly with performance at all levels of job complexity, though it is clear that the more complex the job, the more important GMA is; hence the common assertion that the relationship between GMA and job performance is moderated by job complexity. Indeed, it has been noted that job complexity is one of the few moderators of the effects of GMA on job performance […] Subsequent meta-analyses in the US were by and large congruent with Hunter’s findings […] the UK studies on GMA and job performance and training mirror the findings from the US. This is in line with the reported overlap in choices of test for measuring GMA. […] GMA is especially important for explaining individual differences in learning, which, in low-complexity professions, may not be that important after training, but, in high-complexity professions, may be needed especially in the job. However, it is clear (as much in the UK as in the US data) that GMA matters in every job and for both training and performance […] Studies in the European Community (EC) echo the pattern of results from US and UK studies.”

“Whilst powerful, the above reviewed studies provide no longitudinal evidence for the predictive power of GMA, making interpretation of causational paths largely speculative. However, there is equally impressive evidence for the longitudinal validity of GMA in the prediction of job performance. […] the longitudinal associations tend to hold even when socioeconomic status (SES) is taken into account. Thus, within the same family (where SES levels are the same for every member) family members with higher GMA tend to have significantly better jobs, and earn more, than their lower GMA relatives (Murray, 1998). In fact, at 1993 figures, and controlling for SES, people with average levels of GMA (IQ = 100) earned almost $20,000 less than siblings who were 20 IQ points brighter, and almost $10,000 more than siblings who scored 20 IQ points lower. […] Deary and colleagues found that childhood GMA accounted for 23.2 per cent and parental social class for 17.6 per cent of the total variance in social status attainment in mid life (Deary et al., 2005). The most compelling evidence for the longitudinal validity of GMA in the prediction of occupational level and income was provided by a study spanning back almost four decades (Judge et al., 1999). The authors reported correlations between GMA at age 12 and occupational level (r = .51) and income (r = .53) almost forty years later. Moreover, a reanalysis of these data (which also included the Big Five personality traits) estimated that the predictive power of GMA was almost 60 per cent higher than that of Conscientiousness (the trait that came second) (Schmidt & Hunter, 2004).”

“several robust studies, particularly in the US, report that whites tend to score higher than Hispanics, who tend to score higher than blacks on IQ tests. Estimates of white–black differences in IQ tend to give whites an average advantage of .85 to 1.00 standard deviation (that is, almost 15 IQ points), which is certainly ‘not trivial’ (Hough & Oswald, 2000, p. 636). Although group differences in job performance are somewhat less pronounced (Hattrup, Rock & Scalia, 1997; Waldman & Avolio, 1991), the mainstream view in intelligence research is that these differences are not caused by any test biases […] Despite this divisive picture of intellectual potential and job opportunities, there is little evidence for the benefits of ignoring GMA when it comes to selecting employees. In fact, most studies report just the opposite, namely detrimental effects of banning IQ-based personnel selection […] GMA-based selection is not necessarily a disadvantage for any group of society, as individuals would be rated on the basis of their own capability rather than their group membership […] people with an IQ
80 (about 10 per cent of whites and 30 per cent of blacks in the US) are currently considered unsuitable for the US army by federal law and there are few civilian employers who would hire under this GMA threshold”

I’ve seen people on the internet frame the army as a(/n) (good?) option for poor young black people in the US with limited options/who can’t afford to go to college; ‘a military career may be much better than a minimum-wage job, all things considered’. Considering that the 30 % number above is the proportion of all blacks who’d get rejected, the proportion of young poor blacks with limited options who may not be able to get in may be quite high. Although I’ve read about cutoffs like these before, I got a bit of a shock when I realized how many people don’t even have options like these available to them. Okay, back to the text – why does GMA matter?

“The main reason why GMA predicts job performance, and related outcomes, is that it causes faster, better, more effective and enduring knowledge acquisition. […] In simple terms, having a higher IQ means being able to learn faster. […] Another reason why IQ tests predict job performance is that higher GMA is linked to higher job role breadth, enabling brighter employees to perform a wider range of tasks and, in turn, be rated more highly by their supervisors […] Although GMA is a strong predictor of overall job performance (correlating at about r=.50, and thus explaining 25 per cent of the variance in job performance), it matters most in complex or intellectually demanding jobs (where it correlates at about r = .80 with job performance) and least in unintellectual or cognitively simple jobs (where it correlates with job performance at about r = .20). Objective measures of performance correlate more highly with GMA measures than subjective assessments of performance, such as supervisory ratings, do. […] Specific abilities, that is, variance in cognitive abilities unaccounted for by the general GMA factor, are insignificant predictors of job performance and related outcomes once GMA is taken into account. This is counterintuitive to most people because the layperson tends to overestimate the importance of situational and job-specific factors when interpreting the determinants of work performance.”

GMA is measured or tested via objective performance tests […], whereas personality traits are assessed via subjective inventories, notably self- or other-reports (but especially self-reports). In that sense, one can distinguish between cognitive abilities and personality traits on the basis of assessment methods, whereby the former reflect individual differences in the capacity to identify correct responses to a standardised test (verbal or non-verbal), whereas the latter reflect individual differences in general behavioural tendencies, assessed only subjectively, that is, through people’s accounts (one’s own or others’). This led to a now well-established distinction in psychology to refer to cognitive abilities in terms of maximal performance and personality traits in terms of typical performance […] With regard to job and training performance, which have been the criteria par excellence in personnel selection for over a century, it is interesting that although GMA is a good predictor of job and training performance, we use maximal performance measures (ability tests) to predict typical performance (aggregate levels of achievement at work for instance income or occupational level). […] It should be noted that personality traits are not only assessed via self-report inventories (though that is indeed the most common way of assessing them). Observation, situational tests, projective techniques and even objective measures can also serve as measures of personality […] The most commonly used forms of observation in personnel selection are interviews […] and biodata […] Employers may not explicitly state that what they are assessing in an interview or looking for in biodata is indeed traces of personality, but there is longstanding evidence for the fact that candidates’/interviewees’ personality traits affect employers’ decisions (Wagner, 1949), despite the fact that most interviewees fake […] Although it has long been argued that personality traits – or indeed any psychological construct – should not be assessed only with one method, e.g., self-report or interview […], but with a multi-method approach […], most researchers and practitioners continue to rely on single methods of assessment and, in the case of personality, that method is self-report inventories. However, it is important to disentangle the variance that is caused by the method of assessment and the actual trait or construct that is being assessed. This is a complex theoretical and methodological issue: for example, self-reports of cognitive ability tend to correlate with performance tests of cognitive ability only at r = .40 (Chamorro-Premuzic, Moutafi & Furnham, 2005), meaning they share less than 20 per cent of the variance.”

When I read that last sentence I thought to myself that r = .40 is very low. Yeah, I know about the Dunning–Kruger effect and all that stuff – I’ve written about stuff like that before here on several occasions (see e.g. this) – but even so. This is strange to me, considering how good people usually are (‘seem to me to be?’) at figuring out where they belong in the social hierarchies they are members of. Note that a correlation like that is not explained by an ‘everybody overestimate themselves and this means that there’s not a very good correspondance between actual scores and self-evaluations’-argument – if everybody did overestimate themselves to the same extent, the ordering would be completely unaffected, and GMA metrics only care about the orderings. You need smart people to overestimate themselves less than the not-so-smart people, or to be more likely to underestimate themselves, to see correlations like these. Okay, anyway, back to the book. Quite a bit of the personality trait chapter covered things which I’ve previously read about in much more detail in e.g. Funder or Leary & Hoyle, but not all of it was review and I do want to talk a bit about some of the stuff in the book which I either have not read about before, or have at least not talked about much here on the blog:

“The question of whether personality inventories should be used or not in the context of personnel selection has divided practitioners and researchers for decades. Practitioners tend to assign much more weight to personality than to abilities, but are reluctant to accept the validity of self-reports because common sense indicates that people can and will fake. On the other hand, researchers are still debating whether faking is really a problem and whether the validities of personality inventories are acceptable, meaningless or high. […] Thus the answer to the question of whether personality tests should be used in personnel selection will depend mostly on who you ask, even if answers are based on exactly the same data. [I consider this to be a big red flag, but perhaps that’s just me…] […] What is beyond debate is that personality inventories are weaker predictors of job and training performance than are cognitive ability tests”

“Regardless of where one stands in relation to the use of personality inventories […], it is clear that Conscientiousness is the most important personality predictor of job performance […], and thus the most important non-ability factor in personnel selection, at least among the Big Five personality traits. […] Agreeableness seems to be advantageous in jobs requiring interpersonal interactions or where getting along is paramount […] A typical case is customer service jobs, and indeed Agreeableness has been found to predict performance on these jobs quite well […], especially if based on teamwork rather than individualistic tasks”

“Even since personality inventories were developed there have been objections to the use of such tests in personnel selection […] In the context of work psychology the two main criticisms are that it is easy to fake responses to a personality inventory and that personality traits are only weak predictors of occupational outcomes. […] In an attempt to provide a comprehensive review of the literature, Michael Campion (in Morgeson et al., 2007) examined the salient studies on faking […], concluding that:
‘Four overall conclusions can be drawn from this review of the research literature on faking in personality tests. First, the total number of studies on the topic is large, suggesting that faking has been viewed as an important problem. Second, people can and apparently do fake their responses on personality tests. Third, almost half the studies where criterion-related validity was studied found some effect of faking on criterion-related validity. Fourth, there has been substantial research devoted to techniques for detecting and mitigating faking, but no techniques appear to solve the problem adequately.'” […]

“Even if faking can be overcome, or in cases where it does not seriously threaten the validity of personality traits as predictors of work-related outcomes, critics of the use of personality inventories in personnel selection have another, often more fundamental, objection, namely the fact that the magnitude of the association between personality traits and the predicted criteria is modest at best, and often non-significant […] The irony is that opposite conclusions are often drawn from exactly the same data. […] Regardless of the magnitude of the correlation between personality scores and work-related outcomes, it is clear that the validity of personality inventories is largely dependent on the type of criterion we chose to predict. Thus, unlike with GMA, many factors moderate the effects of personality on job and training performance […] It is plausible to predict that the validities of personality traits will increase substantially if the correct criteria (or predictors) are chosen.”