Abstract
The Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer in Journal of Psychopathology and Behavioral Assessment, 26, 41–54, 2004) is a widely used measure of emotion regulation, however, few studies have examined the scale’s psychometric properties in clinical samples. The current study examined the latent factor structure of the scale, in addition to other psychometric properties, in a naturalistic sample of adult outpatients (n = 344) receiving dialectical behavior therapy (DBT). A number of models were fitted, including a bifactor model, which has not previously been examined with the DERS, but which may be more appropriate for addressing the study’s two primary aims. These aims were to: (1) investigate some of the prior concerns raised about the Awareness subscale items; and (2) evaluate the appropriateness of using DERS total vs. subscale scores. Results indicated that a modified bifactor model which excluded the Awareness items from the general factor fit the data best. Additional findings suggested that the DERS total score (excluding Awareness items) was reliable and had good criterion-related validity in this sample. However, the findings related to the DERS subscales were mixed. This study extends prior research on the DERS by demonstrating the utility of the scale with individuals receiving DBT for problems with emotion regulation, as well as by clarifying some of the prior questions about the scale’s latent structure.
Similar content being viewed by others
Notes
Computation of omega values requires knowing the error variance (Brunner et al. 2012; Rodriguez et al. 2016b). The estimation procedure used in our CFA analyses (WLSMV), with items treated as ordered categorical, does not produce estimates for error variance. Thus, for computation of omega values, parameter estimates were obtained using a robust ML-based estimator (MLR in Mplus; Li 2016; Raykov and Marcoulides 2011), treating the DERS items as continuous. This was justified given that the DERS has 5-response categories (Beauducel and Herzberg 2006; Dolan 1994) and a robust form of ML estimation was employed (Li 2016). Model fit statistics were acceptable using MLR estimation to fit the same bifactor model presented in Figure 1: χ2(563) = 1089.15, p < .001; RMSEA = .05, SRMR = .08; CFI = .92; TLI = .91. Consistent with Li’s (2016) simulation results, factor loadings were overall lower using MLR estimation than WLSMV (Figure 1), but reasonably comparable. A table of the bifactor results using MLR estimation is available upon request from the authors.
An alternative approach to fitting models 5–7 would be to remove Awareness items altogether, although such models provide less information regarding how Awareness relates to the others factors. The results are very similar and lend themselves to comparable conclusions: 5 correlated factors: χ2(585) = 1443.68, p < .001; RMSEA = .09, CFI = .93, TLI = .92; Higher-order, 5 primary factors: χ2(400) = 1315.09, p < .001; RMSEA = .08, CFI = .93, TLI = .93; Bifactor, 5 grouping factors: χ2(375) = 958.50, p < .001; RMSEA = .07, CFI = .96, TLI = .95).
References
Addis, M. & Linehan, M. M. (1989). Predicting suicidal behavior: Psychometric properties of the Suicidal Behaviors Questionnaire. Poster presented at the Annual Meeting of the Association for the Advancement Behavior Therapy, Washington, D.C.
Afshari, A., Neshat-Doost, H. T., Maracy, M. R., Ahmady, M. K., & Amiri, S. (2014). The effective comparison between emotion-focused cognitive behavioral group therapy and cognitive behavioral group therapy in children with separation anxiety disorder. Journal of Research in Medical Sciences, 19, 221–227.
Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2012). An examination of the latent structure of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 34, 382–392.
Bardeen, J. R., Fergus, T. A., Hannan, S. M., & Orcutt, H. K. (2016). Addressing psychometric limitations of the difficulties in emotion regulation scale through item modification. Journal of Personality Assessment, 93, 298–309.
Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186–203.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.
Bonn-Miller, M. O., Vujanovic, A. A., & Zvolensky, M. J. (2008). Emotional dysregulation: association with coping-oriented marijuana use motives among current marijuana users. Substance Use & Misuse, 43(11), 1653–1665.
Brown, T. A. (2006). Confirmatory factor analysis for applied sciences. New York: The Guilford Press.
Brown, M. Z., Comtois, K. A., & Linehan, M. M. (2002). Reasons for suicide attempts and nonsuicidal self-injury in women with borderline personality disorder. Journal of Abnormal Psychology, 111, 198–202.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills: Sage.
Brunner, M., Nagy, G., & Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80(4), 796–846.
Buckholdt, K. E., Parra, G. R., & Jobe-Shields, L. (2009). Emotion regulation as a mediator of the relation between emotional socialization and deliberate self-harm. American Journal of Orthopsychiatry, 79(4), 482–490.
Byrne, B. M. (2012). Structural equation modeling with Mplus: basic concepts, applications, and programming. New York: Routledge/Taylor & Francis Group.
Chapman, A. L., & Dixon-Gordon, K. L. (2007). Emotional antecedents and consequences of deliberate self-harm and suicide attempts. Suicide and Life Threatening Behavior, 37, 543–552.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.
Cho, Y., & Hong, S. (2013). The new factor structure of the Korean version of the difficulties in emotion regulation scale (K-DERS) incorporating method factor. Measurement and Evaluation in Counseling and Development, 46, 192–201.
Cohn, A. M., Jakupcak, M., Seibert, A. L., Hildebrandt, T. B., & Zeichner, A. (2010). The role of emotion dysregulation in the association between men’s restrictive emotionality and use of physical aggression. Psychology of Men & Masculinity, 11(1), 53–64.
Cole, D. A. (1988). Hopelessness, social desirability, depression and parasuicide in two college student samples. Journal of Consulting and Clinical Psychology, 56, 131–136.
Cooper, J. L., O’Shea, A. E., Atkinson, M. J., & Wade, T. D. (2014). Examination of the difficulties in emotion regulation scale and its relation to disordered eating in a young female sample. International Journal of Eating Disorders, 47, 630–639.
Cotton, C. R., Peters, D. K., & Range, L. M. (1995). Psychometric properties of the suicidal behaviors questionnaire. Death Studies, 19, 391–397.
Coutinho, J., Ribeiro, E., Ferreirinha, R., & Dias, P. (2010). The Portuguese version of the difficulties in emotion regulation scale and its relationship with psychopathological symptoms. Revista de Psiquiatria Clinica, 37, 145–151.
Dan-Glauser, E. S., & Scherer, K. R. (2013). The difficulties in emotion regulation scale (DERS): factor structure and consistent of a French translation. Swiss Journal of Psychology, 72, 5–11.
Dolan, C. V. (1994). Factor analysis of variables with 2, 3, 5 and 7 response categories: a comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology, 47(2), 309–326.
Ehring, T., Fischer, S., Schnülle, J., Bösterling, A., & Tuschen-Caffier, B. (2008). Characteristics of emotion regulation in recovered depressed versus never depressed individuals. Personality and Individual Differences, 44, 1574–1584.
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466–491.
Fowler, J. C., Charak, R., Elhai, J. D., Allen, J. G., Frueh, B. C., & Oldham, J. M. (2014). Construct validity and factor structure of difficulties in emotion regulation scale among adults with severe mental illness. Journal of Psychiatric Research, 58, 175–180.
Fox, H. C., Axelrod, S. R., Paliwal, P., Sleeper, J., & Sinha, R. (2007). Difficulties in emotion regulation and impulse control during cocaine abstinence. Drug and Alcohol Dependence, 89(2–3), 298–301.
Fox, H. C., Hong, K. A., & Sinha, R. (2008). Difficulties in emotion regulation and impulse control in recently abstinent alcoholics compared with social drinkers. Addictive Behaviors, 33(2), 388–394.
Giromini, L., Velotti, P., de Campora, G., Bonalume, L., & Zavattini, G. C. (2012). Cultural adaptation of the difficulties in emotion regulation scale: reliability and validity in an Italian sample. Journal of Clinical Psychology, 68, 989–1007.
Glenn, C. R., & Klonsky, D. E. (2009). Emotion regulation as a core feature of borderline personality disorder. Journal of Personality Disorders, 23, 20–28.
Gomez-Simon, I., Penelo, E., & de la Osa, N. (2014). Factor structure and measurement invariance of the difficulties emotion regulation scale (DERS) in Spanish adolescents. Psicothema, 26, 401–408.
Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emotion regulation and dysregulation: development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 26, 41–54.
Gratz, K. L., Bardeen, J. R., Levy, R., Dixon-Gordon, K. L., & Tull, M. T. (2015). Mechanisms of change in an emotion regulation group therapy for deliberate self-harm among women with borderline personality disorder. Behavior Research and Therapy, 65, 29–35.
Gross, J. J. (2007). Handbook of emotion regulation. New York: Guilford.
Hannesdottir, D. K., & Ollendick, T. H. (2007). The role of emotion regulation in the treatment of child anxiety disorders. Clinical Child and Family Psychology Review, 10, 273–293.
Harrison, A., Sullivan, S., Tchanturia, K., & Treasure, J. (2009). Emotion recognition and regulation in anorexia nervosa. Clinical Psychology & Psychotherapy, 16, 348–356.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
Iverson, K. M., Follette, V. M., Pistorello, J., & Fruzzetti, A. E. (2011). An investigation of experiential avoidance, emotion dysregulation, and distress tolerance in young adult outpatients with borderline personality disorder symptoms. Personality Disorders: Theory, Research, and Treatment.
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.
Kokonyei, G., Urban, R., Reinhardt, M., Jozan, A., & Demetrovics, Z. (2014). The difficulties in emotion regulation scale: factor structure in chronic pain patients. Journal of Clinical Psychology, 70, 589–600.
Koons, C. R., Robins, C. J., Tweed, J. L., Lynch, T. R., Gonzalez, A. M., Morse, J. Q., et al. (2001). Efficacy of dialectical behavior therapy in women veterans with borderline personality disorder. Behavior Therapy, 32, 371–390.
Lambert, M. J., Morton, J. J., Hatfield, D., Harmon, C., Hamilton, S., Reid, R. C., … Burlingame, G. M. (2004). Administration and Scoring Manual for the OQ-45.2 (Outcome Measures). American Professional Credentialing Services.
Lavender, J. M., Wonderlich, S. A., Peterson, C. B., Crosby, R. D., Engel, S. G., Mitchell, J. E., et al. (2014). Dimensions of emotion dysregulation in bulimia nervosa. European Eating Disorders Review, 22(3), 212–216.
Li, C. (2016). Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949.
Linehan, M. M. (1993). Cognitive behavior therapy for borderline personality disorder. New York: Guilford.
Linehan, M. M., Armstrong, H. E., Suarez, A., Allmon, D., & Heard, H. L. (1991). Cognitive-behavioral treatment of chronically parasuicidal borderline clients. Archives of General Psychiatry, 48, 1060–1064.
Linehan, M. M., Schmidt, H., Dimeff, L. A., Craft, J. C., Kanter, J., & Comtois, K. A. (1999). Dialectical behavior therapy for clients with borderline personality disorder and drug-dependence. American Journal on Addiction, 8, 279–292.
Linehan, M. M., Dimeff, L. A., Reynolds, S. K., Comtois, K. A., Welch, S. S., Heagerty, P., & Kivlahan, D. R. (2002). Dialectical behavior therapy versus comprehensive validation plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorder. Drug and Alcohol Dependence, 67, 13–26.
McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah: Erlbaum.
McKenzie, C. K., & Gross, J. J. (2014). Nonsuicidal self-injury: an emotion regulation perspective. Psychopathology, 47, 207–219.
Mennin, D. S. (2004). Emotion regulation therapy for generalized anxiety disorder. Clinical Psychology and Psychotherapy, 11, 17–29.
Mennin, D. S. (2006). Emotion regulation therapy: an integrative approach to treatment resistant anxiety disorders. Journal of Contemporary Psychotherapy, 36, 95–105.
Mennin, D. S., Heimberg, R. G., Turk, C. L., & Fresco, D. M. (2005). Preliminary evidence for an emotion dysregulation model of generalized anxiety disorder. Behavior Research and Therapy, 43, 1281–1310.
Muthen, L.K., & Muthen, B.O. (1998–2010). Mplus user’s guide (6th ed.). Los Angeles: Muthen & Muthen
Neacsiu, A. D., Eberle, E. W., Kramer, R., Weismann, T., & Linehan, M. M. (2015). Dialectical behavior therapy skills for transdiagnostic emotion dysregulation: a pilot randomized controlled trial. Behavior Research and Therapy, 59, 40–51.
Neumann, A., van Lier, P. A. C., Gratz, K. L., & Koot, H. M. (2010). Multidimensional assessment of emotion regulation difficulties in adolescents using the difficulties in emotion regulation scale. Assessment, 17, 138–149.
Nock, M. K. (2009). Why do people hurt themselves?: new insights into the nature and functions of self-injury. Current Directions in Psychological Science, 18, 78–83.
Osman, A., Bagge, C. C., Guitierrez, P. M., Konick, L. C., Kooper, B. A., & Barrios, F. X. (2001). The suicidal behaviors questionnaire–revised (SBQ-R): validation with clinical and nonclinical samples. Assessment, 5, 443–454.
Perez, J., Venta, A., Garnaat, S., & Sharp, C. (2012). The difficulty in emotion regulation scale: factor structure and association with nonsuicidal self-injury in adolescent inpatients. Journal of Psychopathology and Behavioral Assessment, 34, 393–404.
Pfohl, B., Blum, N., St John, D., McCormick, B., Allen, J., & Black, D. W. (2009). Reliability and validity of the borderline evaluation of severity over time (BEST): a self-rated scale to measure severity and change in persons with borderline personality disorder. Journal or Personality Disorders, 23, 281–293.
Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. New York: Routledge/Taylor & Francis Group.
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696.
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544–559.
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140.
Rizvi, S. L., & Steffel, L. M. (2014). A pilot study of 2 brief forms of dialectical behavior therapy skills training for emotion dysregulation in college students. Journal of American College Health, 62, 434–439.
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237.
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluating bifactor models: calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150.
Ruganci, R. N., & Gencoz, T. (2010). Psychometric properties of a Turkish version of the difficulties in emotion regulation scale. Journal of Clinical Psychology, 66, 442–455.
Safer, D. L., Telch, C. F., & Agras, W. S. (2001). Dialectical behavior therapy for bulimia nervosa. American Journal of Psychiatry, 158, 632–634.
Salsman, N. L., & Linehan, M. M. (2012). An investigation of the relationships among negative affect, difficulties in emotion regulation, and features of borderline personality disorder. Journal of Psychopathology and Behavioral Assessment, 34(2), 260–267.
Salters-Pedneault, K., Roemer, L., Tull, M. T., Rucker, L., & Mennin, D. S. (2006). Evidence of broad deficits in emotion regulation associated with chronic worry and generalized anxiety disorder. Cognitive Therapy and Research, 30(4), 469–480.
Seligowski, A. V., Lee, D. J., Bardeen, J. R., & Orcutt, H. K. (2015). Emotion regulation and posttraumatic stress symptoms: a meta-analysis. Cognitive Behaviour Therapy, 44, 87–102.
Sheppes, G., Suri, G., & Gross, J. J. (2015). Emotion regulation and psychopathology. Annual Review of Clinical Psychology, 11, 379–405.
Slee, N., Spinhoven, P., Garnefski, N., & Arensman, E. (2008). Emotion regulation as mediator of treatment outcome in therapy for deliberate self-harm. Clinical Psychology & Psychotherapy, 15(4), 205–216.
Snorrason, I., Smári, J., & Ólafsson, R. P. (2010). Emotion regulation in pathological skin picking: findings from a non-treatment seeking sample. Journal of Behavior Therapy and Experimental Psychiatry, 41(3), 238–245.
Staples, A. M., & Mohlman, J. (2012). Psychometric properties of the GAD-Q-IV and DERS in older, community-dwelling GAD patients and controls. Journal of Anxiety Disorders, 26(3), 385–392.
Steiger, J. H. and Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City.
Steinberg, E. A. & Drabick, D. A. (2015). A developmental psychopathology perspective on ADHD and comorbid conditions: The role of emotion regulation. Child Psychiatry and Human Development.
Svaldi, J., Griepenstroh, J., Tuschen-Caffier, B., & Ehring, T. (2012). Emotion regulation deficits in eating disorders: a marker of eating pathology or general psychopathology? Psychiatry Research, 197, 103–111.
Telch, C. F., Agras, W. S., & Linehan, M. M. (2001). Dialectical behavior therapy for binge eating disorder. Journal of Consulting and Clinical Psychology, 69, 1061–1065.
Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.
Tull, M. T., Barrett, H. M., McMillan, E. S., & Roemer, L. (2007). A preliminary investigation of the relationship between emotion regulation difficulties and posttraumatic stress symptoms. Behavior Therapy, 38(3), 303–313.
Verheul, R., van den Bosch, L. M. C., Koeter, M. W. J., de Ridder, M. A. J., Stijnen, T., & van den Brink, W. (2003). Dialectical behavior therapy for women with borderline personality disorder: 12-month, randomized clinical trial in the Netherlands. British Journal of Psychiatry, 182, 135–140.
Weinberg, A., & Klonsky, D. (2009). Measurement of emotion dysregulation in adolescents. Psychological Assessment, 21, 616–621.
Westermann, S., & Lincoln, T. M. (2011). Emotion regulation difficulties are relevant to persecutory ideation. Psychology and Psychotherapy: Theory, Research and Practice, 84(3), 273–287.
Whiteside, U., Chen, E., Neighbors, C., Hunter, D., & Lo, T. (2007). Difficulties regulating emotions: do binge eaters have fewer strategies to modulate and tolerate negative affect? Eating Behaviors, 8, 162–169.
Williams, A. D., Grisham, J. R., Erskine, A., & Cassedy, E. (2012). Deficits in emotion regulation associated with pathological gambling. British Journal of Clinical Psychology, 51(2), 223–238.
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach's α, Revelle's β, and Mcdonald's ωH: their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 1–11.
Acknowledgements
We would like to acknowledge the contributions of all administrative staff and clinicians at EBTCS to this research. Moreover, we would like to specifically acknowledge Jamie Bedics, Ph.D., Junny Kim, B.A., and Natalie Meade, B.A. for their contributions to a poster presentation about these data. Some of the data for this manuscript were previously presented at the Association for Behavior and Cognitive Therapies (ABCT) Annual Convention in San Francisco, CA (November 2010).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
There was no funding source for this study.
Experiment Participants
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed Consent
Informed consent was obtained whenever possible from individual participants included in the study, with the exceptions noted in the Method section that were IRB approved.
Conflict of Interest
Travis Osborne, Jared Michonski, Jennifer Sayrs, Stacy Shaw Welch and Leslie Karwoski Anderson declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Osborne, T.L., Michonski, J., Sayrs, J. et al. Factor Structure of the Difficulties in Emotion Regulation Scale (DERS) in Adult Outpatients Receiving Dialectical Behavior Therapy (DBT). J Psychopathol Behav Assess 39, 355–371 (2017). https://doi.org/10.1007/s10862-017-9586-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-017-9586-x