Short assessment of the Big Five: robust across survey methods except telephone interviewing

Frieder R Lang, Dennis John, Oliver Lüdtke, Jürgen Schupp, Gert G Wagner, Frieder R Lang, Dennis John, Oliver Lüdtke, Jürgen Schupp, Gert G Wagner

Abstract

We examined measurement invariance and age-related robustness of a short 15-item Big Five Inventory (BFI-S) of personality dimensions, which is well suited for applications in large-scale multidisciplinary surveys. The BFI-S was assessed in three different interviewing conditions: computer-assisted or paper-assisted face-to-face interviewing, computer-assisted telephone interviewing, and a self-administered questionnaire. Randomized probability samples from a large-scale German panel survey and a related probability telephone study were used in order to test method effects on self-report measures of personality characteristics across early, middle, and late adulthood. Exploratory structural equation modeling was used in order to test for measurement invariance of the five-factor model of personality trait domains across different assessment methods. For the short inventory, findings suggest strong robustness of self-report measures of personality dimensions among young and middle-aged adults. In old age, telephone interviewing was associated with greater distortions in reliable personality assessment. It is concluded that the greater mental workload of telephone interviewing limits the reliability of self-report personality assessment. Face-to-face surveys and self-administrated questionnaire completion are clearly better suited than phone surveys when personality traits in age-heterogeneous samples are assessed.

Figures

Fig. 1
Fig. 1
Construct variability increases with age in computer-assisted telephone interviewing. CATI, computer-assisted telephone interviewing; FACE, face-to-face interviewing; SELF, self-administered questionnaire completion

References

    1. Allemand M, Zimprich D, Hendriks AAJ. Age differences in five personality domains across the life span. Developmental Psychology. 2008;44:758–770. doi: 10.1037/0012-1649.44.3.758.
    1. Allemand M, Zimprich D, Hertzog C. Cross-sectional age differences and longitudinal age changes of personality in middle adulthood and old age. Journal of Personality. 2007;75:323–358. doi: 10.1111/j.1467-6494.2006.00441.x.
    1. Allport GW, Odbert HS. Trait names: A psycholexical study. Psychological Monographs. 1936;47(1):Whole No. 211.
    1. Aluja A, García O, García LF, Seisdedos N. Invariance of the “NEO-PI-R” factor structure across exploratory and confirmatory factor analyses. Personality and Individual Differences. 2005;38:1879–1889. doi: 10.1016/j.paid.2004.11.014.
    1. Asparouhov T, Muthén B. Exploratory structural equation modeling. Structural Equation Modeling. 2009;16:397–438. doi: 10.1080/10705510903008204.
    1. Baltes PB, Reese HW, Nesselroade JR. Life-span developmental psychology: Introduction to research methods. Monterey: Brooks Cole; 1977.
    1. Bauer TN, Truxillo DM, Paronto ME, Weekley JA, Campion MA. Applicant reactions to different selection technology: Face-to-face, interactive voice response, and computer-assisted telephone screening interviews. International Journal of Selection and Assessment. 2004;12:135–148. doi: 10.1111/j.0965-075X.2004.00269.x.
    1. Benet-Martinez V, John OP. Los Cinco Grandes across cultures and ethnic groups: Multitrait–multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology. 1998;75:729–750. doi: 10.1037/0022-3514.75.3.729.
    1. Bentler PM. Comparative fit indices in structural models. Psychological Bulletin. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238.
    1. Blickle G, Kramer J, Mierke J. Telephone-administered intelligence testing for research in work and organizational psychology. European Journal of Psychological Assessment. 2010;26:154–161. doi: 10.1027/1015-5759/a000022.
    1. Borghans L, Duckworth AL, Heckman JJ, ter Weel B. The economics and psychology of personality traits. Journal of Human Resources. 2008;43:972–1059.
    1. Borkenau P, Ostendorf F. Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences. 1990;11:515–524. doi: 10.1016/0191-8869(90)90065-Y.
    1. Brannick MT. Critical comments on applying covariance structure modeling. Journal of Organizational Behavior. 1995;16:201–213. doi: 10.1002/job.4030160303.
    1. Cattell RB. Description and measurement of personality. Yonkers-on-Hudson: World Book; 1946.
    1. Cercone JJ, Danielson CK, Ruggiero KJ, Kilpatrick DG. Telephone surveys of traumatic experiences and other sensitive topics. In: Buchanan D, Fisher CB, Gable L, editors. Research with high-risk populations: Balancing science, ethics, and law. Washington: American Psychological Association; 2009. pp. 77–92.
    1. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling. 2007;14:464–504.
    1. Cheung GW, Rensvold RB. The effects of model parsimony and sampling error on the fit of structural equation models. Organizational Research Methods. 2001;4:236–264. doi: 10.1177/109442810143004.
    1. Church AT, Burke PJ. Exploratory and confirmatory tests of the Big Five and Tellegen’s three- and four-dimensional models. Journal of Personality and Social Psychology. 1994;66:93–114. doi: 10.1037/0022-3514.66.1.93.
    1. Costa PT, Jr, McCrae RR. Domains and facets: Hierarchical personality assessment using the Revised NEO Personality Inventory. Journal of Personality Assessment. 1995;64:21–50. doi: 10.1207/s15327752jpa6401_2.
    1. Davis JA, Smith TW. The NORC General Social Survey: A user’s guide. Newbury Park: Sage; 1992.
    1. Donnellan MB, Lucas RE. Age differences in the Big Five across the life span: Evidence from two national samples. Psychology and Aging. 2008;23:558–566. doi: 10.1037/a0012897.
    1. Donnellan MB, Oswald FL, Baird BM, Lucas RE. The Mini-IPIP Scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment. 2006;18:192–203. doi: 10.1037/1040-3590.18.2.192.
    1. Eysenck H. Dimensions of personality. London: Routledge & Kegan Paul; 1947.
    1. Gagné P, Hancock GR. Measurement model quality, sample size, and solution propriety in confirmatory factor models. Multivariate Behavioral Research. 2006;41:65–83. doi: 10.1207/s15327906mbr4101_5.
    1. Gerlitz Y, Schupp J. Zur Erhebung der Big-Five-basierten Persönlichkeitsmerkmale im SOEP [Assessment of Big Five personality characteristics in the SOEP]. German Institute of Economic Research (Research Notes 4) Berlin: DIW; 2005.
    1. Goldberg LR. An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology. 1990;59:1216–1229. doi: 10.1037/0022-3514.59.6.1216.
    1. Gosling SD, Rentfrow PJ, Swann WB., Jr A very brief measure of the Big Five personality domains. Journal of Research in Personality. 2003;37:504–528. doi: 10.1016/S0092-6566(03)00046-1.
    1. Groves RM, Fowler FJ, Jr, Coupler MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology. 2. Hoboken: Wiley; 2009.
    1. Groves RM, Mathiowetz NA. Computer assisted telephone interviewing: Effects on interviewers and respondents. Public Opinion Quarterly. 1984;48:356–369. doi: 10.1086/268831.
    1. Hansell S, Sparacino J, Ronchi D, Strodtbeck FL. Ego-development responses in written questionnaires and telephone interviews. Journal of Personality and Social Psychology. 1984;47:1118–1128. doi: 10.1037/0022-3514.47.5.1118.
    1. Headey B, Muffels R, Wagner GG. Long-running German panel survey shows that personal and economic choices, not just genes, matter for happiness. Proceedings of the National Academy of Sciences. 2010;107:17922–17926. doi: 10.1073/pnas.1008612107.
    1. Heineck G, Anger S. The returns to cognitive abilities and personality traits in Germany. Labour Economics. 2010;17:535–546. doi: 10.1016/j.labeco.2009.06.001.
    1. Herman JB. Mixed-mode data collection: Telephone and personal interviewing. Journal of Applied Psychology. 1977;62:399–404. doi: 10.1037/0021-9010.62.4.399.
    1. Holbrook AL, Green MC, Krosnick JA. Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly. 2003;67:79–125. doi: 10.1086/346010.
    1. Holden RR, Troister T. Developments in the self-report assessment of personality and psychopathology in adults. Canadian Psychology. 2009;50:120–130.
    1. Hopwood CJ, Donnellan MB. How should the internal structure of personality inventories be evaluated? Personality and Social Psychology Review. 2010;14:332–346. doi: 10.1177/1088868310361240.
    1. John OP, Srivastava S. The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In: Pervin LA, John OP, editors. Handbook of personality theory and research. New York: Guilford; 1999. pp. 102–138.
    1. Klages L. Die Grundlagen der Charakterkunde [Foundations of characterology] Leipzig: Bart; 1926.
    1. Lang, F. R. (2005). Erfassung des kognitiven Leistungspotenzials und der "Big Five" mit Computer-Assisted-Personal-Interviewing (CAPI): Zur Reliabilität und Validität zweier ultrakurzer Tests und des BFI-S [Assessment of cognitive capabilities and the Big Five with Computer-Assisted Personal Interviewing (CAPI): Reliability and validity]. German Institute of Economic Research (Research Notes 9/2005), Berlin: DIW.
    1. Lang FR, Baltes PB, Wagner GG. Desired lifetime and end-of-life desires across adulthood from 20 to 90: A dual-source information model. Journal of Gerontology: Psychological Science. 2007;62:268–276.
    1. Lang FR, Lüdtke O, Asendorpf J. Testgüte und psychometrische Äquivalenz der deutschen Version des Big Five Inventory (BFI) bei jungen, mittelalten und alten Erwachsenen [Validity and psychometric equivalence of the German version of the Big Five Inventory in young, middle-aged and old adults] Diagnostica. 2001;47:111–121. doi: 10.1026//0012-1924.47.3.111.
    1. Lucas RE, Donnellan MB. Age differences in personality: Evidence from a nationally representative Australian sample. Developmental Psychology. 2009;45:1353–1363. doi: 10.1037/a0013914.
    1. Marsh HW, Hau K-T, Balla JR, Grayson D. Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research. 1998;33:181–220. doi: 10.1207/s15327906mbr3302_1.
    1. Marsh HW, Hau K-T, Wen Z. In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralising Hu & Bentler’s (1999) findings. Structural Equation Modeling. 2004;11:320–341. doi: 10.1207/s15328007sem1103_2.
    1. Marsh HW, Lüdtke O, Muthén B, Asparouhov T, Morin AJS, Trautwein U, Nagengast B. A new look at the big-five factor structure through exploratory structural equation modeling. Psychological Assessment. 2010;22:471–491. doi: 10.1037/a0019227.
    1. Marsh HW, Muthén B, Asparouhov T, Lüdtke O, Robitzsch A, Morin AJS, Trautwein U. Exploratory structural equation modeling, integrating CFA and EFA: Applications to students’ evaluations of university teaching. Structural Equation Modeling. 2009;16:439–476. doi: 10.1080/10705510903008220.
    1. Marsh, H. W., Nagengast, B., & Morin, A. J. S. (2011). Measurement invariance of big-five factors over the lifespan: ESEM tests of gender, age, plasticity, maturity, and La Dolce Vita effects. Manuscript under review.
    1. Marsh HW, Trautwein U, Lüdtke O, Köller O, Baumert J. Integration of multidimensional self-concept and core personality constructs: Construct validation and relations to well-being and achievement. Journal of Personality. 2006;74:403–455. doi: 10.1111/j.1467-6494.2005.00380.x.
    1. Marshall MB, De Fruyt F, Rolland J-P, Bagby RM. Socially desirable responding and the factorial stability of the NEO PI-R. Psychological Assessment. 2005;17:379–384. doi: 10.1037/1040-3590.17.3.379.
    1. McCrae RR, Costa PT., Jr Personality trait structure as a human universal. American Psychologist. 1997;52:509–516. doi: 10.1037/0003-066X.52.5.509.
    1. McCrae RR, Zonderman AB, Costa PT, Jr, Bond MH, Paunonen SV. Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology. 1996;70:552–566. doi: 10.1037/0022-3514.70.3.552.
    1. McHorney CA, Kosinski M, Ware JE., Jr Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: Results from a national survey. Medical Care. 1994;32:551–567. doi: 10.1097/00005650-199406000-00002.
    1. Meredith W. Measurement invariance, factor analysis and factorial invariance. Psychometrika. 1993;58:525–543. doi: 10.1007/BF02294825.
    1. Mischel W, Shoda Y. A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions dynamics, and invariance in personality structure. Psychological Review. 1995;102:246–268. doi: 10.1037/0033-295X.102.2.246.
    1. Muthén LK, Muthén BO. Mplus user’s guide. 5. Los Angeles: Muthén & Muthén; 2008.
    1. Rammstedt B, Goldberg LR, Borg I. The measurement equivalence of the Big-Five factor markers for persons with different levels of education. Journal of Research in Personality. 2010;44:53–61. doi: 10.1016/j.jrp.2009.10.005.
    1. Rammstedt B, Holzinger B, Rammsayer TH. Zur Äquivalenz der Papier-Bleistift- und einer computergestützten Version des NEO-Fünf- Faktoren-Inventars (NEO-FFI) [Equivalence of a paper-pencil- and a computer-based version of the NEO-FFI] Diagnostica. 2004;50:88–97. doi: 10.1026/0012-1924.50.2.88.
    1. Rammstedt B, John OP. Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. Journal of Research in Personality. 2007;41:203–212. doi: 10.1016/j.jrp.2006.02.001.
    1. Rentfrow PJ. Statewide differences in personality. Toward a psychological geography of the United States. American Psychologist. 2010;65:548–558.
    1. Richman WL, Kiesler S, Weisband S, Drasgow F. A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology. 1999;84:754–775. doi: 10.1037/0021-9010.84.5.754.
    1. Roberts BW, Walton KE, Viechtbauer W. Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin. 2006;132:1–25. doi: 10.1037/0033-2909.132.1.1.
    1. Schräpler JP, Schupp J, Wagner GG. Changing from PAPI to CAPI: Introducing CAPI in a longitudinal study. Journal of Official Statistics. 2010;26:233–269.
    1. Schwarz N. Self-reports: How the questions shape the answers. American Psychologist. 1999;54:93–105. doi: 10.1037/0003-066X.54.2.93.
    1. Siedler T, Schupp J, Wagner GG. Innovative methods within in the context of archival data: Examples from household panel surveys. In: Trzesniewski KH, Donnellan MB, Lucas RE, editors. Secondary data analysis. Washington: American Psychological Association; 2011. pp. 103–118.
    1. Smith TW, Williams PG. Personality and health: Advantages and limitations of the five-factor model. Journal of Personality. 1992;60:395–423. doi: 10.1111/j.1467-6494.1992.tb00978.x.
    1. Stoop I, Billiet J, Koch A, Fitzgerald R. Improving survey response: Lessons learned from the European Social Survey. Chichester: Wiley; 2010.
    1. Sutin AR, Costa PT, Miech R, Eaton WW. Personality and career success: Concurrent and longitudinal relations. European Journal of Personality. 2009;23:71–84. doi: 10.1002/per.704.
    1. Swanberg AB, Martinsen OL. Personality, approaches to learning and achievement. Educational Psychology. 2010;30:75–88. doi: 10.1080/01443410903410474.
    1. Taylor MF, Brice J, Buck N, Prentice-Lane E. British household panel survey user manual: Vol. A. Introduction, technical report and appendices. Colchester: University of Essex; 2009.
    1. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38:1–10. doi: 10.1007/BF02291170.
    1. Tun PA, Lachman ME. Age differences in reaction time and attention in a National Telephone sample of adults: Education, sex, and task complexity matter. Developmental Psychology. 2008;44:1421–1429. doi: 10.1037/a0012845.
    1. Vassend O, Skrondal A. Validation of the NEO Personality Inventory and the five-factor model: Can findings from exploratory and confirmatory factor analysis be reconciled? European Journal of Personality. 1997;11:147–166. doi: 10.1002/(SICI)1099-0984(199706)11:2<147::AID-PER278>;2-E.
    1. Wagner GG, Frick JR, Schupp J. The German Socio-Economic Panel Study (SOEP): Scope, evolution and enhancements. Schmollers Jahrbuch. 2007;1:139–170.
    1. Winkelmann L, Winkelmann R. Personality, work, and satisfaction: Evidence from the German Socio-Economic Panel. Journal of Positive Psychology. 2008;3:266–275. doi: 10.1080/17439760802399232.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe