Psychometric Properties of the Emotional Outburst Inventory (EMO-I): Rating What Children Do When They Are Irritable

J Clin Psychiatry 2022;83(2):21m14015

To cite: Carlson GA, Silver J, Klein DN. Psychometric properties of the Emotional Outburst Inventory (EMO-I): rating what children do when they are irritable. J Clin Psychiatry. 2022;83(2):21m14015.
To share: https://doi.org/10.4088/JCP.21m14015

© Copyright 2022 Physicians Postgraduate Press, Inc.

^aRenaissance School of Medicine at Stony Brook University, Stony Brook, New York
^bStony Brook University, Stony Brook, New York
*Corresponding author: Gabrielle A. Carlson, MD, Stony Brook University, Putnam Hall, South Campus, Stony Brook, NY 11794-8790 (Gabrielle.Carlson@Stonybrook.edu).

A significant number of children and adolescents present to outpatient departments,¹ emergency departments,² and inpatient units³ and receive school disciplinary referrals⁴ because they respond to ordinary frustrations and disappointments with volcanic anger or distress, which we will call serious emotional outbursts. Ironically, it is difficult to estimate the number of children with outbursts for 3 reasons.

First, serious emotional outbursts have never had a clear home in DSM classification systems, and without a diagnosis it is difficult to study the phenomenon. At present, only two conditions have outbursts as part of their definition: disruptive mood dysregulation disorder (DMDD) and intermittent explosive disorder (IED). Outbursts in DMDD are defined in DSM-5⁵ as “severe recurrent temper outbursts manifested verbally (eg, verbal rages) and/or behaviorally (eg, physical aggression toward people or property) that are grossly out of proportion in intensity or duration to the situation or provocation”^(p156) occurring 3 or more times a week. The diagnosis also requires irritability between outbursts and exclusion of other disorders in which outbursts can occur. Until DSM-5,⁵ IED had been operationalized as a condition of “discrete episodes of failure to resist aggressive impulses resulting in assaultive acts, occurring with little or no provocation,” applicable “only after other mental disorders that might account for episodes of aggressive behavior have been ruled out.”^(p466) Many childhood disorders have been associated with aggressive outbursts. DSM-5 added criteria for how often such outbursts should occur based on studies of adults without data to support those numbers in children.⁶

Second, though many DSM-defined conditions include irritability as a symptom, irritability is neither defined within disorders nor included in the glossary of terms. Some research interviews define irritability differently for different disorders, whereas others use the same ratings and definitions interchangeably across disorders; eg, depression irritability is sometimes used to fulfill mania criteria.⁷

Third, studies of irritability have often conflated “tonic irritability,” the grouchy and persistently angry mood component, and temper outbursts, a manifestation of “phasic irritability.”^8,9 To study irritability, investigators have appropriated items from existing rating scales¹⁰ and repurposed variables from existing interview segments¹¹ as well as creating new measures like the Affective Reactivity Index (ARI).¹² While these options have furthered research on irritability in general, they have done little to distinguish tonic and phasic irritability or define aspects of phasic irritability. In fact, these two components may predict different outcomes,¹³ differ in heritability,¹⁴ and have different relationships to attention-deficit/hyperactivity disorder severity.¹¹ Vidal-Ribas¹⁵ recently critiqued the field, calling for the development of measures “specifically designed to assess phasic and tonic irritability [which] should capture, not only the potential presence of each, but also the frequency, intensity, duration, and contextual factors. These measures should inquire about specific behaviors.”^(p1465)

Though there are many measures of mood dysregulation and aggression,^16–18 there are surprisingly few studies of actual outbursts or phasic irritability. Data are also scarce in clinical samples of children and adolescents. The Outburst Monitoring Scale (OMS),¹⁹ a parent-report questionnaire that measures past-week aggressive behaviors (shouting, cursing, fighting), was validated in only 23 “aggressive” adolescents and 20 controls. The Rage Attacks Questionnaire,²⁰ developed for use in children with Tourette’s disorder with severe outbursts, has not been used for other conditions. Finally, the Rage Outbursts and Anger Rating Scale,²¹ which consists of 3 clinician-rated items measuring frequency, intensity, and duration of past-week rages, was used to study outbursts in samples of children with obsessive-compulsive disorder²¹ and anxiety²² and has not been used outside those conditions.

In 2002, Carlson and colleagues developed the Children’s Agitation Inventory to record actual outburst behaviors in hospitalized children requiring “as needed” medication for their severe outbursts.^23,24 Constructed from a list of the behaviors observed during children’s agitated emotional outbursts, the 17 codable items consisted of verbal acts (eg, yelling/screaming), discrete physical acts (eg, hitting or kicking others or objects), and emotional behaviors (eg, looking tearful/sad). These behaviors, and their duration and frequency, were the foundation of the Irritability Inventory (IRRI), which was developed as a screening instrument for characterizing outbursts in outpatients and inpatients.^25–28 The IRRI’s utility was subsequently demonstrated in 242 psychiatrically hospitalized children.²⁸ It was significantly (P < .001) correlated with the ARI (r = 0.67),¹² irritable oppositional defiant symptoms (r = 0.55), the Child Mania Rating Scale (CMRS; r = 0.52),²⁹ and number of medications needed for agitation per day (r = 0.22).³ Children with parent-rated outbursts on the IRRI were significantly more likely to have outbursts while hospitalized (odds ratio [OR] = 3.87; 95% CI, 1.76–8.50).²⁸

The IRRI has since been renamed the Emotional Outburst Inventory (EMO-I) to prevent confusion with the ARI.¹² The paucity of measures for emotionally impairing outbursts prompted us to examine the psychometric characteristics of the EMO-I in greater detail. We hypothesized the following: (1) EMO-I outburst severity, frequency, and duration will correlate significantly with other measures of irritability; (2) EMO-I severity, frequency, and duration will be associated with measures of impairment and treatment intensity; and (3) the EMO-I will contribute variance over and above other measures of irritability in associations with impairment and treatment intensity.

METHODS

Participants

A total of 2,838 children and adolescents between the ages of 5 and 18 years were seen in the Stony Brook Child and Adolescent Psychiatry Outpatient Clinic (Stony Brook, New York) between February 2005 and June 2014. A suburban medical school-based clinic, it serves Suffolk County, Long Island, with a population of roughly 2.5 million people. Information was taken from a clinic database using safeguards to protect participant identities for a project approved by the university’s Institutional Review Board.

Procedure

A general information form (with demographic information and treatment history) and parent rating scales (see the following section), including the EMO-I, were completed before the evaluation. Caregivers were instructed to check or circle the rating that best described the child’s current behavior and problem. A complete set of parent rating scales was available for 2,552 families (89.9%; EMO-I Total), of whom 1,772 (69.4%) evidenced any anger problem on the EMO-I (EMO-I Positive); 780 (30.6%) had no anger endorsements (EMO-I Negative). (See Supplementary Tables 1 and 2 for sample comparisons.)

Measures

The Emotional Outburst Inventory. The EMO-I (Supplementary Table 3) includes questions about how easy it is for the child or adolescent to get angry, followed by questions about 6 common anger triggers, 9 anger behaviors, outburst frequency and duration, irritability between outbursts, and feelings after the outburst. For this study, we focus only on outburst severity (number of child behaviors during an outburst, not counting “expresses anger appropriately”), outburst frequency (ie, daily, weekly, several times a month, less than monthly), and outburst duration (< 15 minutes, up to 30 minutes, up to 60 minutes, up to 2 hours, half a day, longer) (Table 2). A score of zero was defined as no anger problems (EMO-I Negative).

Irritability measures. Parents completed the Child Behavior Checklist (CBCL),³⁰ which assesses a broad range of psychopathology. Items are rated on a 3-point scale (0 = absent, 1 = occurs sometimes, 2 = occurs often). The CBCL was used to derive the dysregulation profile (DP), which was originally referred to as the “CBCL pediatric bipolar profile”^31,32 but was subsequently understood to represent poor emotional and behavioral self-regulation.^33,34 The DP was considered positive if the child had T scores ≥ 67 on the CBCL anxiety/depression, inattention/hyperactivity, and aggression subscales, and negative otherwise. In addition, an irritability subscale, used in prior studies,^35,36 was extracted from the CBCL. It is the sum of 3 items: “temper tantrums or hot temper,” “stubborn, sullen or irritable,” and “sudden changes in mood or feelings.” Cronbach α was 0.82.

Parents also completed the Child and Adolescent Symptom Inventory-4R^37,38 (CASI-4R), a widely used DSM-IV–based symptom scale. We used the sum of 3 oppositional defiant disorder irritability items (“loses temper,” “angry and resentful,” “easily annoyed”; Cronbach α = 0.81), the temper loss item alone, and the irritability item from the depression section (“is irritable most of the day”). CASI-4R items are rated as “never,” “sometimes,” “often,” or “very often”.

Finally, 3 irritability and outburst items from the Child Mania Rating Scale–Parent report (CMRS-P)²⁹ were used: “spends hours or days at a time being extremely irritable”; “has rage attacks, intense and prolonged temper tantrums”; and “experiences rapid mood swings.” Each item is rated on a 4-point scale: 0 (never/rarely), 1 (sometimes), 2 (often), and 3 (very often). Cronbach α was 0.84.

Treatment and impairment. We examined 3 variables from the parent form reflecting treatments the child had received (hospitalization, emergency department referral, atypical antipsychotics). Help-seeking for outbursts was recorded after the evaluation, with data entered on 1,833 consecutive cases. Finally, impairment due to psychopathology was derived from the sum of 14 CASI-4R impairment questions found at the end of symptom sections where parents are asked, “How often do the behaviors in the relevant symptom category interfere with the child’s ability to do schoolwork or get along with other people?” (Cronbach α = 0.86; correlation with CBCL total problems = 0.70, P < .001). The CASI-4R impairment measure is further described elsewhere.^38–40

Analyses. For the EMO-I outburst severity domain, Cronbach α was computed using R.⁴¹ Polychoric correlation dispersion matrices were used because of the ordinal nature of the data. All other statistical analyses were performed using SPSS 25 (IBM Corp, 2017). To evaluate convergent validity, point-biserial and Pearson correlations were used to examine the relationships of the EMO-I domains with concurrent measures of irritability. To assess construct validity, we examined associations of the EMO-I with treatment and impairment. We conducted simultaneous linear and logistic regression analyses, with the EMO-I domains as the independent variables and the indicators of treatment and impairment as the dependent variables. We also examined incremental validity by determining whether the EMO-I explained a statistically significant amount of variance in treatment and impairment after accounting for existing and better-validated measures of irritability (the CBCL-DP and the CBCL irritability subscale). Two-step hierarchical multiple linear regressions were conducted with the specified treatment/impairment indicators as the dependent variable. In step 1, the CBCL-DP or CBCL irritability subscale scores and child age, race/ethnicity, and sex were entered. In step 2, the EMO-I domain scores were entered. We then re-ran the models reversing the order of entry for scores on the EMO-I and the CBCL-DP or CBCL irritability subscale.

RESULTS

Of the EMO-I Total sample (n = 2,552), 69.4% were male, 73.0% were White, 8.6% were Black, 11.8% were Latinx, and 3.6% were Asian or indigenous. Mean (SD) age was 12.1 (3.5) years. About half (54.5%) lived with both biological parents. Less than half of their parents (41.9%) had no college education. The majority of children (72.5%) were school referred. There were small but significant correlations between demographic variables and EMO-I variables (Table 1 and Supplementary Table 1).

Evaluation/treatment for outbursts was sought in 21.5% of the sample. Significant differences between the EMO-I Positive and Negative subsamples (children with and without anger problems) were that the former were less often school referred; their parents were more likely single parents and were more likely to seek help for outbursts, and their irritability ratings were higher (Supplementary Tables 1 and 2).

The EMO-I Positive children, who comprised 69.4% of the sample, engaged in a mean (SD) of 3.67 (1.98) outburst behaviors. The majority argued, whined or sulked, slammed doors, punched walls, made a mess, destroyed property, or swore or shouted at or insulted others. Moreover, 80.8% of EMO-I Positive children had aggressive outbursts (destroyed property, hit/kicked/bit/spit, and/or needed restraint). Also, 40.1% of the sample had outbursts at least weekly, and 41.1% had outbursts lasting at least 30 minutes (Table 2 and Supplementary Table 1). We used number of outburst behaviors as indicators of severity. Cronbach α for the severity domain for the EMO-I Total sample was 0.83. Severity was correlated with frequency (r = 0.46, P < .01) and duration (r = 0.36, P < .01). Frequency and duration correlated (r = 0.25, P < .01) (Table 3).

Convergent Validity

Pearson correlations of the EMO-I with the irritability measures for the total sample appear in Table 3. The EMO-I domain scores were significantly associated with all of the irritability measures. However, the correlations with variables with a phasic irritability component like temper loss or rage were larger than the correlation with depression irritability, providing preliminary evidence for discriminant validity. Associations with the EMO-I severity domain (median r = 0.66) were stronger than associations with the frequency (median r = 0.47) and duration (median r = 0.39) domains. The median correlation among the 6 established irritability scales was r = 0.66.

Construct Validity

In multiple regressions controlling for child age, race/ethnicity, and sex, the EMO-I was significantly associated with all indicators of treatment and impairment, although relationships varied by EMO-I domain (Table 4). Outburst severity was related to hospitalization, antipsychotic use, emergency department referral, help-seeking for outbursts, and CASI-4R impairment. Duration was also related to hospitalizations and help-seeking for outbursts. Notably, each of these associations is independent in the sense that each was over and above each of the other EMO-I domains.

Incremental Validity

In hierarchical multiple regressions, the EMO-I severity domain explained unique variance over and above the CBCL-DP on all 5 treatment/impairment validators (accounting for an additional 3%–12% of variance). The frequency domain added unique variance to the impairment domain; the duration domain added unique variance to the impairment and hospitalizations domains (Table 5). Notably, the CBCL-DP was associated with all 5 validators before the EMO-I was entered into the model. However, with EMO-I in the model, the CBCL-DP was no longer significantly associated with hospitalization, emergency department referral, or parents seeking help for outbursts.

When the incremental utility of the EMO-I was considered over and above that of the CBCL irritability subscale, the severity domain again explained unique variance on all 5 treatment/impairment validators (accounting for an additional 2%–5% of variance), and the duration and frequency domains were uniquely associated with impairment (Table 5). Similarly, the CBCL irritability subscale was associated with all 5 validators before the EMO-I was entered into the model. However, with EMO-I in the model, the CBCL irritability subscale was no longer significantly associated with emergency department referral.

DISCUSSION

To summarize, the specific focus of the EMO-I is on the phasic component of irritability. We found that anger problems were common (69.4%) in the total clinic sample, and help-seeking for outbursts was the presenting problem for 21.5% of children, compared with rates of comparably defined outbursts in 2%–5% in non-psychiatric samples.^42,43 The EMO-I exhibited good internal consistency and good convergent and construct validity. We found moderate-high associations between existing measures of irritability and the EMO-I subscales, especially EMO-I severity (ie, the number of behaviors shown during an outburst). Correlations with EMO-I frequency and duration were also uniformly significant, but at a lower magnitude. Outburst severity uniquely predicted all service use and impairment variables, whereas outburst duration uniquely predicted impairment and hospitalizations but not other treatment variables or help-seeking; frequency uniquely predicted impairment but not treatment or help-seeking. This finding is important since most rating scales address frequency of outbursts without distinguishing the behaviors manifested. The more moderate correlation between the EMO-I frequency and duration subscales and existing measures of irritability suggests they are measuring overlapping but different aspects of irritability that need separate measurement focus. Taken together, these findings highlight the importance of measuring phasic irritability for understanding clinical impairment and treatment intensity.

The extent to which the EMO-I adds to existing measures of irritability is an important question. We compared the EMO-I with the CBCL-DP and CBCL irritability subscale in predicting our 5 treatment and impairment variables. In hierarchical regressions, the addition of the EMO-I domains consistently accounted for additional variance compared to the CBCL-DP and CBCL irritability subscale. In contrast, the CBCL-DP and irritability subscale made unique contributions to fewer impairment variables after accounting for the EMO-I. Thus, the EMO-I domains, especially severity, provided incremental validity to more established measures of dysregulation and irritability.

Strengths and Limitations

The study’s strengths include a large, clinical sample with a rating scale completion rate of 90%. In addition, we compared the EMO-I to multiple existing irritability scales and used clinically important validators to test both construct and incremental validity. Finally, the use of a clinical sample with a wide age range yields findings that may be more generalizable to other clinical settings.

However, several limitations should be taken into account. First, the sample may be subject to referral bias, a limitation that is ubiquitous in clinic-based studies.⁴⁴ The EMO-I Total sample may generalize best to tertiary care child psychiatry clinics and the EMO-I Positive subsample to children with a problem with irritability/temper. Future research should validate the EMO-I in other clinical settings. Second, we relied solely on parent report for all measures, raising the possibility that associations are inflated by shared method variance. Additional research is needed to determine whether the EMO-I can be used by other informants. Importantly, however, our previous preliminary work suggested that our findings might generalize to other clinical settings and types of validators, as we found that there was a significant association between parent reports on the EMO-I and the child’s outburst behavior in an inpatient unit.^27,29 Third, we were unable to compare the EMO-I to the widely used ARI,¹² as at the time of data collection the ARI¹² was not available, although our more recent inpatient data are promising.³⁰ Future research on the reliability and validity of the EMO-I should incorporate irritability measures such as the ARI¹² as well as other measures of impairment. Fourth, our sample reflects the racial/ethnic distribution of Suffolk County, Long Island, which does not represent most of the country. Finally, the cross-sectional nature of our results precludes examination of the predictive validity and utility of the EMO-I as a measure of change, something the field sorely needs.

CONCLUSIONS

The EMO-I is a parent-report measure of emotional outbursts specifically designed to characterize outburst behaviors, duration, and frequency. We found preliminary evidence that the EMO-I is associated with multiple indicators of irritability, treatment, and impairment in psychiatrically referred children and adds information to existing irritability measures. These findings are a first step toward the validation of the EMO-I, which appears to be a useful addition to the growing apparatus of methods and measures designed to assess and advance understanding of irritability and emotional outbursts by providing an in-depth assessment of phasic irritability.

Submitted: March 30, 2021; accepted September 8, 2021.
Published online: January 25, 2022.
Potential conflicts of interest: Dr Carlson’s spouse is on the data safety monitoring boards for Lundbeck and Pfizer. Ms Silver and Dr Klein have no conflicts to disclose.
Funding/support: Support for this research was provided by funding though the American Academy of Child and Adolescent Psychiatry and NSF 16-588 (Silver).
Role of the sponsor: The American Academy of Child and Adolescent Psychiatry provided funding for statistical analysis and free access publication of this article.
Supplementary material: Available at Psychiatrist.com.

Clinical Points

• Serious emotional outbursts, one of the components of phasic irritability, need to be studied separately from tonic irritability, as they contribute separately to treatment seeking and various levels of treatment intensity and impairment
• The Emotional Outburst Inventory (EMO-I) is a brief clinical rating tool that can be used to identify and quantify the important dimensions of outbursts, including their severity, frequency, and duration.
• These data suggest that outburst behavior severity rather than frequency and duration explains more of the variance in treatment type and impairment.

Editor’s Note: We encourage authors to submit papers for consideration as a part of our Focus on Childhood and Adolescent Mental Health section. Please contact Karen D. Wagner, MD, PhD, at kwagner@psychiatrist.com.