Readability of English-Language Self-Report Measures in Bipolar Disorder

The utility of self-report questionnaires is limited by the comprehension abilities of their target audience. When written at an appropriate reading level, they can provide valuable insight to physicians in diagnosing patients with mental disorders such as bipolar disorder. When written at an elevated reading level, they may perpetuate health disparities within socioeconomic populations. The American Medical Association recommends that the reading level for such materials remains below the fifth- to sixth-grade level.¹ Research has shown that people with mental health challenges have reduced literacy compared to the general population.² Evidence suggests that individuals with mental illness may have limited available opportunities for reading and writing due to potential attentive and functional impairment, as well as reduced access to general education and vocational training.²

Research indicates that materials related to psychiatric conditions often exceed the recommended reading levels for the public. A study on patient health pamphlets in mental health reported an average readability at the eighth-grade reading level.³ Previous readability studies have found elevated reading levels of questionnaires in depression, anxiety, and binge-eating disorders.^4,5 It was reported that publicly available web-based information on bipolar disorder is currently written at an elevated reading level and often requires higher-level education to be utilized appropriately.⁶

Emotion and mood can strongly influence attention, perception, memory, and reasoning, which are crucial for reading comprehension.⁷ Studies show that bipolar disorder mood swings or episodes may affect working memory processes.⁸ Researchers also noted that symptoms of cognitive impairment remained during the resolution of the acute episode and euthymic periods of bipolar disorder.⁹ Sleep deprivation may further exacerbate attention problems and impair memory, making it more challenging for individuals with bipolar disorder to process information.¹⁰ Evidence suggests that cognition is more impaired following partial compared to total sleep deprivation, further suggesting that these specific types of sleep restriction be given special attention within the context of bipolar disorder.¹¹ In addition, the first episode of bipolar disorder was associated with cognitive deficits that were widespread on neuropsychological testing.¹²

Pharmacologic treatments can have effects on cognition in bipolar disorder. In a study on second-generation antipsychotics in patients with bipolar disorder, risperidone and olanzapine-treated patients performed poorly on working memory, attention, and verbal fluency tasks compared with healthy individuals.¹³ In contrast, patients with bipolar disorder taking anticonvulsants or lithium had an improvement in cognitive performance.¹⁴ A cross-sectional study comparing psychomotor speed, memory, attention, and executive functioning in bipolar disorder patients taking valproic acid, lamotrigine, carbamazepine, or lithium showed that lamotrigine-treated patients had the highest cognitive performance, followed by patients treated with lithium, valproic acid, and carbamazepine.¹⁴

Many have sought to develop a variety of screening tools that would aid in identifying people with bipolar disorder. While there are numerous available tools, their quality is of interest. This is the first study, to our knowledge, to analyze the readability of self-report measures in bipolar disorder. The objective of this study was to calculate the overall readability of measures in bipolar disorder and the individual readability of the items and instructions sections of measures in bipolar disorder. We hypothesize that the reading levels of these questionnaires will be elevated compared to the recommended standard.

METHODS

Systematic reviews and measures were selected from a publicly available literature search in the PsycINFO, PubMed, PubMed Central, and Google Scholar databases. We performed the literature search using the Medical Subject Heading term “bipolar disorders” combined with “surveys and questionnaires” to identify measures. For all measures reviewed, the inclusions criteria were (1) self-reported, (2) written in the English language, (3) assessed symptoms of bipolar disorder, (4) publicly available, and (5) used for the assessment of adults. Exclusion criteria included clinician-observed measures, clinician standardized interviews, and measures not developed in English in the United States. The last date a database search was conducted was December 28, 2023.

Twenty-eight measures in bipolar disorder were identified from a systematic review by Cerimele and colleagues¹⁵ in 2019 and a literature search, 19 of which were later excluded because they did not meet the above inclusion criteria or were not obtainable. Four of these 9 measures did not include standard instructions and thus were excluded from analyses involving instructions. The following measures were included: Altman Self-Rating Mania Scale, Carroll Depression Scale, Inventory of Depressive Symptoms Self-Report, Quick Inventory of Depressive Symptoms Self-Report, Affective Self-Rating Scale, Daily Mood Monitoring, Internal State Scale (ISS), Multidimensional Assessment of Thymic States (MAThyS), and NIMH Prospective Life-Chart Methodology–Self.^16–24

Readability was assessed via readable software (readable.com). Each measure was analyzed via 4 readability formulas from the readable software: Gunning Fog, Simple Measure of Gobbledygook (SMOG), FORCAST, and Flesch Reading Ease. The Gunning Fog score is computed with the following equation^25–27:

These validated indices were included as they are commonly used throughout the United States to analyze text and have been previously used to assess the readability of measures and patient education materials in prior studies in psychiatry. These readability formulas utilize various forms of information to calculate each score, including the number of syllables per word, the length of sentences, and the degree of deviation from basic word lists. The Gunning Fog, SMOG, and FORCAST indices provided a corresponding grade level (ie, sixth grade and seventh grade) based on the measure analyzed. However, the Flesch Reading Ease Score was calculated by a score and was manually converted to a grade level by the authors for analysis. The readability for each measure was calculated by averaging scores across all 4 indices. The readability of the item and instruction sections was analyzed separately. Measures with an average readability greater than 6.00 were considered above the recommended reading level. Mean and SD readability scores for each metric across measures were computed. Analysis of the data was performed using Microsoft Excel (version 16.49).

Table 1 illustrates the conversion metrics between the score and grade level for the Flesch Reading Ease Score index and highlights the differences between the readability indices.²⁹

RESULTS

All measures had at least 1 component (instructions or items) written above the recommended reading level. All measures with instructions had a mean readability score above the recommended sixth-grade level. The items sections of 8 out of 9 measures included in our study had a mean readability score above the recommended sixth grade level. The mean reading level of the instructions and items sections was 9.35 (SD=1.45; range, 7.03–11.51) and 8.78 (SD=1.73; range, 5.59–11.31), respectively.A reading comprehension level of high school or above was required in 6 out of 9 measures (67%).

Figure 1 depicts the average readability of the instructions and items sections for each bipolar measure.³⁰ The readability analysis of various bipolar disorder assessment measures revealed a wide range of scores. The MAThyS measure exhibited the highest readability score, with instructions averaging 11.50 and items averaging 10.01, indicating a higher level of complexity. In contrast, the ISS measure had lower readability scores, with instructions averaging 7.03 and items averaging 5.59, suggesting it is easier to read and comprehend. This suggests that some measures may need to be modified to ensure they are accessible to readers across a broader spectrum of literacy.

Figure 2 highlights the readability scores of each measure using Gunning Fog, SMOG, FORCAST, and Flesch Reading Ease Score indices.³⁰ The readability of measures for bipolar disorder was evaluated using 4 indices: Flesch Reading Ease, Gunning Fog, SMOG, and FORCAST. The mean of the Gunning Fog index was 6.79, the Flesch Reading Ease index was 9.00, the SMOG index was 9.10, and the FORCAST was 10.22. The readability scores computed by the different indices correspond to reading grade levels in the United States.

DISCUSSION

The difficulties with the diagnosis of bipolar disorder have been discussed for decades from the physician’s point of view due to its purely clinical presentation and challenges with the identification of symptoms. Overdiagnosis of bipolar disorder was initially noted in the literature in 2007–2008.^31–33 Contributing factors for misdiagnosis were identified as personality disorders as possible confounders and patient history of substance abuse.³² The DSM-IV was notable for introducing hypomania as a less severe subset of symptoms than mania; however, the criteria for hypomania were less clearly defined, and diagnostic overlap between other psychiatric illnesses was not robustly incorporated in the diagnostic criteria, increasing the difficulty of making a correct diagnosis.

In 2009, the International Society for Bipolar Disorders recommended using symptom measurement surveys in addition to history taking to diagnose bipolar disorder. After this, a study further supported their use in bipolar disorder with positive and negative predictive values greater than 85%.³⁴ When written at an appropriate reading level, these measures provide crucial information to physicians in the diagnosis of patients with mental disorders such as bipolar disorder. With the development of the DSM-5 and self-report symptom screening tools, this dual approach to diagnosing patients with bipolar disorder has become a common standard. Even outside the health system, many young adults can find self-screening tools to assess their symptoms on the readily accessible internet.³⁵ Screening surveys tend to have high sensitivity but low specificity, potentially adding to the trend of overdiagnosis if the screening tool is utilized by itself.

Additionally, the accessibility and standardization of screening surveys to all populations have been called into question for many psychiatric disorders in the literature.³⁶ Doyen’s³⁷ review critically analyzes strategies for specifically diagnosing bipolar disorder and discusses concerns regarding disproportionate misdiagnosis in communities with comorbid posttraumatic stress disorder and lower socioeconomic status. Thus, given the possibility of overdiagnosis or misdiagnosis, self-report questionnaires must be carefully scrutinized to limit potential downstream consequences amid the established difficulties with screening. With misdiagnosis comes unnecessary treatment with mood stabilizers and antipsychotics, and both classes of medications have significant and potentially dangerous side effects.

The empirical results reported should be considered in light of some limitations. Although readability formulas are valuable tools, they cannot capture the full complexity of medical jargon. Readability formulas do not consider qualitative factors such as layout, spacing, font, and text size, which may also impact readability. In this study, only the English versions were analyzed; therefore, these findings cannot be applied to any translations available in other languages. These readability formulas are based on grade levels in the United States and may not directly relate to reading levels of other English-speaking countries. The AMA and NIH guidelines also recommend patient-facing materials in the United States. They should not be generalized to other countries with different readability standards. For example, the UK National Health Services recommends that materials be written at or below a 9-year-old reading level.³⁸

Our study focuses on evaluating the readability of screening surveys; with 9 out of 10 screening tools scoring above the average reading level in our study, the accuracy and utility of these measures to screen for mania and depression symptoms may not be standardized among each patient. A significant challenge in ensuring the effectiveness of the self-report questionnaire is ensuring that a patient is interpreting the questions appropriately. If the self-report symptom tool is utilized with a higher readability than the average patient, it may not be guaranteed that this tool will improve the diagnosis process. Amid the ongoing discussion of how best to diagnose bipolar disorder, our study results show that caution may be necessary before administering and utilizing this screening.

Article Information
Published Online: June 26, 2025. https://doi.org/10.4088/PCC.24m03892
© 2025 Physicians Postgraduate Press, Inc.
Submitted: November 16, 2024; accepted March 7, 2025.
To Cite: Nickel J, Shanmugam S, Jona N, et al. Readability of English-language self report measures in bipolar disorder. Prim Care Companion CNS Disord 2025;27(3): 24m03892.
Affiliations: Wake Forest University School of Medicine, Winston-Salem, North Carolina (Nickel, Shanmugam, Jona); Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, North Carolina (Shah, Gligorovic).
Corresponding Author: Sujeeth Shanmugam, BS, Wake Forest University School of Medicine, 4618 Country Club Rd, Winston-Salem, NC 27104 ([email protected]).
Relevant Financial Relationships: None.
Funding/Support: None.

Clinical Points

• Self-assessment measures are vital tools to help in the diagnosis of mental disorders including bipolar disorder.
• Readability of self-assessment measures should be at or below a sixth-grade reading level to be suitable for the general population.
• Currently, all assessed self-assessment measures had at least 1 component written above the sixth-grade reading level.