A Reliability Generalization Meta-analysis of the Dimensional Obsessive-Compulsive Scale

  1. Rubén López-Nicolás 1
  2. María Rubio-Aparicio 2
  3. Carmen López-Ibáñez 1
  4. Julio Sánchez-Meca 1
  1. 1 Universidad de Murcia
    info

    Universidad de Murcia

    Murcia, España

    ROR https://ror.org/03p3aeb86

  2. 2 Universitat d'Alacant
    info

    Universitat d'Alacant

    Alicante, España

    ROR https://ror.org/05t8bcz72

Journal:
Psicothema

ISSN: 0214-9915,  1886-144X

Year of publication: 2021

Issue Title: Monográfico Mariano Yela

Volume: 33

Issue: 3

Pages: 481-489

Type: Article

DOI: 10.7334/PSICOTHEMA2020.455 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Psicothema

Sustainable development goals

Abstract

Background: The Dimensional Obsessive-Compulsive Scale (DOCS) is a well-established tool for assessing obsessive-compulsive symptomatology. A reliability generalization meta-analysis was conducted to estimate the average reliability of DOCS scores and how reliability estimates vary according to the composition and variability of samples, to identify study characteristics that can explain its variability, and to estimate the reliability induction rate. Method: A literature search produced 86 studies that met the inclusion criteria. Results: For the DOCS total scores, an average alpha coefficient of .925 was found (95% CI [.920,.931]), as well as mean alphas of .881, .905, .913, and .914 for Contamination, Responsibility, Unacceptable Thoughts, and Symmetry subscales, respectively. Moderator analysis showed that internal consistency fell significantly the more clinical and subclinical participants there were in the sample, and the larger the mean score in the sample for the total scores. The most important moderator variables for the subscales were the standard deviation and the mean of the scores. Conclusions: The DOCS scores exhibited excellent internal consistency reliability for both total score and subscale scores and DOCS is suitable both for research and clinical purposes.

€ View funding

Funding information

This research was funded with a grant from the Spanish Government, Ministerio de Economía, Industria y Competitividad, and FEDER funds (project n. PSI2016-77676-P).

Funders

Bibliographic References

  • Abramowitz, J.S., Deacon, B.J., Olatunji, B.O., Wheaton, M.G., Berman, N.C., Losardo, D., Timpano, K.R., McGrath, P.B., Riemann, B.C., Adams, T., Björgvinsson, T., Storch, E.A., & Hale, L.R. (2010). Assessment of obsessive-compulsive symptom dimensions: Development and evaluation of the Dimensional Obsessive-Compulsive Scale. Psychological Assessment, 22(1), 180-198. https://doi.org/10.1037/a0018260
  • Appelbaum, M., Cooper, H., Kline, R.B., Mayo-Wilson, E., Nezu, A.M., & Rao, S.M. (2018). Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. American Psychologist, 73, 3-25. https://doi. org/10.1037/amp0000191
  • Bonett, D.G. (2002). Sample size requirements for estimating intraclass correlations with desired precision. Statistics in Medicine, 21(9), 1331- 1335.
  • Botella, J., Suero, M., & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 15(4), 386-397. https://doi.org/10.1037/ a0019626
  • Charter, R.A. (2003). A breakdown of reliability coefficients by test type and reliability methods, and the clinical implications of low reliability. The Journal of General Psychology, 130, 290-304. https://doi. org/10.1080/00221300309601160
  • Cooper, H., Hedges, L.V., & Valentine, J.C. (2019). The handbook of research synthesis and meta-analysis (3rd ed.). Russell Sage Foundation.
  • Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart, & Winston.
  • Green, C.E., Chen, C.E., Helms, J.E., & Henze, K.T. (2011). Recent reliability reporting practices in Psychological Assessment: Recognizing the people behind the data. Psychological Assessment, 23, 656-669. https://doi.apa.org/doi/10.1037/a0023089
  • Gu, F., Little, T.D., & Kingston, N.M. (2013). Misestimation of reliability using coefficient alpha and structural equation modeling when assumptions of tau-equivalence and uncorrelated errors are violated. Methodology, 9(1), 30-40. https://doi.org/10.1027/1614-2241/ a000052
  • Hartung, J., & Knapp, G. (2001). On tests of the overall treatment effect in the meta-analysis with normally distributed responses. Statistics in Medicine, 20, 1771-1782. https://doi.org/10.1002/sim.791
  • Heldref Foundation (1997). Guidelines for contributors. Journal of Experimental Education, 65, 95-96.
  • Henson, R.K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: Some recommendations for conducting “reliability generalization” studies. Measurement and Evaluation in Counseling and Development, 35, 113-127. https://doi.org/10.1080/07 481756.2002.12069054
  • Higgins, J.P.T., & Thompson, S.G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558. https://doi. org/10.1002/sim.1186
  • Huedo-Medina, T.B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I² index? Psychological Methods, 11(2), 193-206. https://doi.org/10.1037/1082- 989x.11.2.193
  • Knapp, G., & Hartung, J. (2003). Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22, 2693- 2710. https://doi.org/10.1002/sim.1482
  • Lenz, A.S., Ho, C.-M., Rocha, L., & Aras, Y. (2020). Reliability generalization of scores on the Post-Traumatic Growth Inventory. Measurement and Evaluation in Counseling and Development. Advance online publication. https://doi.org/10.1080/07481756.2020.1747940
  • López-López, J.A., Botella, J., Sánchez-Meca, J., & Marín-Martínez, F. (2013). Alternatives for mixed-effects meta-regression models in the reliability generalization approach: A simulation study. Journal of Educational and Behavioral Statistics, 38(5), 443-469. https://doi. org/10.3102/1076998612466142
  • López-Pina, J.A., Sánchez-Meca, J., López-López, J.A., Marín-Martínez, F., Núñez-Núñez, R.M., Rosa-Alcázar, A.I., Gómez-Conesa, A., & Ferrer-Requena, J. (2015). The Yale-Brown Obsessive Compulsive Scale: A reliability generalization meta-analysis. Assessment, 22(5), 619-628. https://doi.org/10.1177/1073191114551954
  • Mataix-Cols, D., Rosario-Campos, M.C., & Leckman, J.F. (2005). A multidimensional model of obsessive-compulsive disorder. The American Journal of Psychiatry, 162(2), 228-238. https://doi. org/10.1176/appi.ajp.162.2.228
  • McNeish, D. (2018). Thanks coeffi cient alpha, we’ll take it from here. Psychological Methods, 23, 412-412. https://doi.org/10.1037/met0000144
  • Nunnally J.C., & Bernstein I.H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
  • Ñúñez-Ñúñez, R.M., Rubio-Aparicio, M., Marín-Martínez, F., Sánchez-Meca, J., López-Pina, J.A., & López-López, J.A. (2020). A reliability generalization meta-analysis of the Padua Inventory-Revised (PIR-R) [Manuscript submitted for publication]. Meta-Analysis Unit, Department of Basic Psychology and Methodology, University of Murcia.
  • Rodríguez, M.C., & Maeda, Y. (2006). Meta-analysis of coefficient alpha. Psychological Methods, 11, 306-322. https://doi.org/10.1037/1082- 989X.11.3.306
  • Rubio-Aparicio, M., Núñez-Núñez, R.M., Sánchez-Meca, J., López-Pina, J.A., Marín-Martínez, F., & López-López, J.A. (2020). The Padua Inventory–Washington State University Revision of obsessions and compulsions: A reliability generalization meta-analysis. Journal of Personality Assessment, 102(1), 113-123. https://doi.org/10.1080/002 23891.2018.1483378
  • Ruscio, A.M., Stein, D.J., Chiu, W.T., & Kessler, R.C. (2010). The epidemiology of obsessive-compulsive disorder in the National Comorbidity Survey Replication. Molecular Psychiatry, 15, 53-63. https://doi.org/10.1038/mp.2008.94
  • Sánchez-Meca, J., & Marín-Martínez, F. (2008). Confidence intervals for the overall effect size in random-effects meta-analysis. Psychological Methods, 13(1), 31-48. https://doi.org/10.1037/1082-989X.13.1.31
  • Sánchez-Meca, J., López-Pina, J.A., López-López, J.A., Marín-Martínez, F., Rosa-Alcázar, A.I., & Gómez-Conesa, A. (2011). The Maudsley Obsessive-Compulsive Inventory: A reliability generalization metaanalysis. International Journal of Clinical and Health Psychology, 11(3), 473-493.
  • Sánchez-Meca, J., López-López, J.A., & López-Pina, J.A. (2013). Some recommended statistical analytic practices when reliability generalization studies are conducted. The British Journal of Mathematical and Statistical Psychology, 66(3), 402-425. https://doi. org/10.1111/j.2044-8317.2012.02057.x
  • Sánchez-Meca, J., López-Pina, J.A., Rubio-Aparicio, M., Marín-Martínez, F., Núñez-Núñez, R.M., López-García, J.J., & López-López, J.A. (2019, May 27-31). REGEMA: Guidelines for conducting and reporting reliability generalization meta-analyses [Paper presentation]. Research Synthesis, Dubrovnik, Croatia. http://dx.doi.org/10.23668/ psycharchives.2449
  • Sánchez-Meca, J., Marín-Martínez, F., Núñez-Núñez, R.M., Rubio-Aparicio, M., López-López, J.A., Blázquez-Rincón, D.M., López-Ibáñez, C., López-Nicolás, R., López-Pina, J.A., & López-García, J.J. (2019, July 8-10). Reporting practices of reliability generalization meta-analysis: An assessment with the REGEMA checklist [Paper presentation]. XVI Congress of Methodology of the Social and Health Sciences, Madrid, Spain.
  • Sánchez-Meca, J., Rubio-Aparicio, M., López-Pina, J.A., Núñez-Núñez, R.M., & Marín-Martínez, F. (2015, July). The phenomenon of reliability induction in the social and health sciences [Paper presentation]. XIV Congress of Methodology of Social and Health Sciences, Palma de Mallorca, Spain.
  • Sánchez-Meca, J., Rubio-Aparicio, M., Núñez-Núñez, R.M., López-Pina, J.A., Marín-Martínez, F., & López-López, J.A. (2017). A reliability generalization meta-analysis of the Padua Inventory of Obsessions and Compulsions. The Spanish Journal of Psychology, 20, 1-15. https://doi. org/10.1017/sjp.2017.65
  • Scherer, R., & Teo, T. (2020). A tutorial on the meta-analytic structural equation modeling of reliability coefficients. Psychological Methods, 25(6), 747-775. http://dx.doi.org/10.1037/met0000261
  • Shields, A.L., & Caruso, J.C. (2004). A reliability induction and reliability generalization study of the cage questionnaire. Educational and Psychological Measurement, 64(2), 254-270. https://doi. org/10.1177/0013164403261814
  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107-120. https://doi. org/10.1007/s11336-008-9101-0
  • Streiner, D.L., & Norman, G.R. (2008). Health measurement scales: A practical guide to their development and use (4th ed.). Oxford University Press.
  • Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
  • Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20. https://doi. org/10.1177/0013164498058001002
  • Vacha-Haase, T., Kogan, L.R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those of test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509-522. https://doi.org/10.1177/ 00131640021970682
  • Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36, 1-48. https://doi. org/10.18637/jss.v036.i03
  • Viechtbauer, W., López-López, J.A., Sánchez-Meca, J., & Marín-Martínez, F. (2015). A comparison of procedures to test for moderators in mixedeffects meta-regression models. Psychological Methods, 20(3), 360- 374. https://doi.org/10.1037/met0000023
  • Watkins, M.W. (2017). The reliability of multidimensional neuropsychological measures: From alpha to omega. The Clinical Neuropsychologist, 31(6-7), 1113-1126. http://dx.doi.org/10.1080/138 54046.2017.1317364
  • Wilkinson, L., & the APA Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
  • Yang, Y., & Green, S.B. (2011). Coefficient alpha: A reliability coefficient for the 21st Century? Journal of Psychoeducational Assessment, 29, 377-392. https://doi.org/10.1177%2F0734282911406668
Data source: Dialnet