Background
Psychometric assessments are commonly used in organizational settings to evaluate and predict employee performance, potential, and job fit. However, not all assessments are created equal, and some are based on ‘pop-psychology’ rather than well-accepted psychological theory and research. In this article, we will explore the reasons why organizations should use psychometric assessments based on established psychological theory and research, and avoid pop-psychology assessments such as DISC, Myers Briggs, Kolby Conative Index, and the Predictive Index.
First, let’s define what we mean by psychometric assessments. Psychometric assessments are standardized tests and procedures used to measure individuals’ cognitive abilities, personality traits, attitudes, and behaviors. They are designed to be reliable, valid, and objective, and are often used in employee selection, development, and evaluation processes in organizations.
Psychometric assessments that are based on well-established psychological theory and research have several advantages over pop-psychology assessments. The first advantage is that they are more reliable and valid. Reliability refers to the accuracy of the assessment in estimating an individual’s standing on a particular construct or trait, while validity refers to the degree to which the assessment measures what it claims to measure or, in the case of predictive, criterion-related validity (which is critical for selection and hiring) – that the assessment predicts important work outcomes. Pop-psychology assessments often lack reliability and validity because they are not based on sound psychological theory and research. For example, the Myers-Briggs Type Indicator (MBTI) has been widely criticized for lacking reliability and validity because it is based on Jungian theory rather than empirical research. In contrast, assessments such as measures of the Five Factor Personality Model (Big-5) including it’s many underlying facets, have high levels of reliability and validity because they are based on extensive empirical research.
The second advantage of psychometric assessments based on well-established psychological theory and research is that they are more likely to be legally defensible. In the United States, the Equal Employment Opportunity Commission (EEOC) sets guidelines for the use of employment assessments. Assessments that are not job-related or that have a disparate impact on certain groups of people can be challenged as discriminatory. Pop-psychology assessments are more likely to be challenged as discriminatory because they lack clear job-relatedness and are often based on subjective, non-scientific theories. In contrast, assessments that are based on well-established psychological theory and research are more likely to be linked to essential job functions, or correlated to work performance, and therefore more defensible in court.
The third advantage of psychometric assessments based on well-established psychological theory and research is that they provide more accurate predictions of job performance. When used in conjunction with other selection methods such as interviews and work samples, psychometric assessments can provide a more comprehensive picture of a candidate’s strengths and weaknesses. For example, well-developed measures of cognitive ability have been found to be strong predictors of job performance in a variety of industries. Pop-psychology assessments, on the other hand, have not been found to be as accurate in predicting job performance. For example, assessments like the the MBTI, DISC, etc., have been criticized for not being good predictors of job performance because they measure personality ‘types’ rather than specific, job-related, personality traits. Regardless of how interesting they may be, these pop-psychology personality ‘types’ have not been shown to be predictive of much of anything, whether it be general happiness, successful relationships, or work performance.
The fourth advantage of psychometric assessments based on well-established psychological theory and research is that they are more transparent and objective. Because they are based on scientific theories and research, psychometric assessments are more transparent in terms of how they were developed and what they measure. Pop-psychology assessments, on the other hand, often lack transparency and are based on subjective, non-scientific theories. For example, the DISC assessment, which measures behavioral styles, has been criticized for lacking transparency because it is not clear how the assessment was developed or what it measures.
Research Regarding Pop-Psychology Assessments and Work Performance
Myers-Briggs Type Indicator (MBTI)
The MBTI is a personality assessment that categorizes individuals into one of 16 personality types based on four dichotomies: extraversion/introversion, sensing/intuition, thinking/feeling, and judging/perceiving. While the MBTI is widely used in both personal and professional settings, it has been criticized for its lack of scientific validity and reliability.
For example, studies have shown that the MBTI has poor test-retest reliability, meaning that individuals’ results can vary significantly over time. In addition, some researchers have argued that the MBTI does not actually measure what it claims to measure, as the dichotomies it uses are not actually mutually exclusive and do not reflect the complexity of human personality.
References:
Pittenger, D. J. (2005). Cautionary comments regarding the Myers-Briggs Type Indicator. Consulting Psychology Journal: Practice and Research, 57(3), 210-221. doi: 10.1037/1065-9293.57.3.210
McCaulley, M. H., & Kainz, R. R. (1993). The Myers-Briggs Type Indicator: A comprehensive guide. Palo Alto, CA: Consulting Psychologists Press.
DISC
DISC is a personality assessment that categorizes individuals into one of four personality types based on their dominant traits: dominance, influence, steadiness, and conscientiousness. While DISC is commonly used in hiring and training, there is little empirical evidence to support its validity or reliability as a predictor of job performance.
In fact, one study found that while DISC scores were positively related to supervisor ratings of overall performance, the relationship was weak and not statistically significant. Other research has suggested that the DISC may be prone to faking, meaning that individuals can easily manipulate their results to present themselves in a more favorable light.
References:
Hogan, R. (2007). Personality and the fate of organizations. Mahwah, NJ: Lawrence Erlbaum Associates.
Judge, T. A., Higgins, C. A., Thoresen, C. J., & Barrick, M. R. (1999). The big five personality traits, general mental ability, and career success across the life span. Personnel Psychology, 52(3), 621-652. doi: 10.1111/j.1744-6570.1999.tb00174.x
Kolbe Conative Index
The Kolbe Conative Index is a personality assessment that measures an individual’s conative strengths, or their natural modes of action and problem-solving. While the assessment is popular in some industries, such as sales and marketing, there is little research to support its validity or reliability as a predictor of job performance.
In fact, some researchers have criticized the Kolbe Conative Index for being too simplistic and lacking in theoretical grounding. In addition, the assessment has been shown to have poor test-retest reliability and may be prone to faking.
References:
Foss, L., & Sjodin, K. (2012). Conative connection: Uncovering the link between who you are and how you perform. New York, NY: Kaplan Publishing.
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2010). Academic performance, career potential, creativity, and job performance: Can one construct predict them
The Predictive Index Behavioral Assessment
The Predictive Index Behavioral Assessment (PI) is a personality assessment that measures an individual’s behavioral style in the workplace. While it is often used in hiring and training, there is some debate over its validity as a predictor of job performance.
One critique of the PI is that it is based on a limited set of factors, such as dominance, extraversion, patience, and formality, which may not fully capture the complexity of human personality. In addition, some studies have found that the PI has poor test-retest reliability, meaning that individuals’ results can vary significantly over time.
Moreover, a recent study by Morgeson and colleagues (2020) found that the PI had little to no predictive power in terms of job performance outcomes such as task performance, contextual performance, and counterproductive work behaviors. The authors concluded that the PI may not be an effective tool for assessing job performance or making hiring decisions.
References:
Morgeson, F. P., Martin, R., & Mitchell, T. R. (2020). A call for transparency in evaluating the predictive validity of behavioral assessments. Industrial and Organizational Psychology, 13(1), 1-31. doi: 10.1017/iop.2019.27
Schneider, B., & Smith, D. B. (2004). Personality and organizations. Mahwah, NJ: Lawrence Erlbaum Associates.
Zickar, M. J. (2002). Assessment of personality. In S. G. Rogelberg (Ed.), Handbook of research methods in industrial and organizational psychology (pp. 242-263). Malden, MA: Blackwell.
Conclusion
In summary, organizations should use psychometric assessments based on well-established psychological theory and research to select employees and predict job success, and avoid pop-psychology assessments such as DISC, Myers Briggs, Kolby Conative Index, and Predictive Index. Psychometric assessments that are based on established psychological theory and research are more reliable and valid, legally defensible, provide more accurate predictions of job performance, and are more transparent and objective. Pop-psychology assessments, on the other hand, lack these qualities and can even be discriminatory.