Qualitative Methods in Early-Phase Drug Trials: Broadening the Scope of Data and Methods From an RCT of N-Acetylcysteine in Schizophrenia
Objective: The pharmacokinetic profile of a drug often gives little indication of its potential therapeutic application, with many therapeutic uses of drugs being discovered serendipitously while being studied for different indications. As hypothesis-driven, quantitative research methodology is exclusively used in early-phase trials, unexpected but important phenomena may escape detection. In this context, this study aimed to examine the potential for integrating qualitative research methods with quantitative methods in early-phase drug trials. To our knowledge, this mixed methodology has not previously been applied to blinded psychopharmacologic trials.
Method: We undertook qualitative data analysis of clinical observations on the dataset of a randomized, double-blind, placebo-controlled trial of N-acetylcysteine (NAC) in patients with DSM-IV-TR-diagnosed schizophrenia (N = 140). Textual data on all participants, deliberately collected for this purpose, were coded using NVivo 2, and emergent themes were analyzed in a blinded manner in the NAC and placebo groups. The trial was conducted from November 2002 to July 2005.
Results: The principal findings of the published trial could be replicated using a qualitative methodology. In addition, significant differences between NAC- and placebo-treated participants emerged for positive and affective symptoms, which had not been captured by the rating scales utilized in the quantitative trial. Qualitative data in this study subsequently led to a positive trial of NAC in bipolar disorder.
Conclusions: The use of qualitative methods may yield broader data and has the potential to complement traditional quantitative methods and detect unexpected efficacy and safety signals, thereby maximizing the findings of early-phase clinical trial research.
Trial Registration: www.anzctr.org.au Identifier:
J Clin Psychiatry 2011;72(7):909-913
© Copyright 2010 Physicians Postgraduate Press, Inc.
Submitted: September 21, 2009; accepted December 14, 2009.
Online ahead of print: September 21, 2010 (doi:10.4088/JCP.09m05741yel).
Corresponding author: Michael Berk, MD, PhD, Department of Clinical and Biomedical Sciences: Barwon Health, University of Melbourne, PO Box 281, Geelong 3220, Victoria, Australia (firstname.lastname@example.org).
One of the great challenges in the development of any novel potential psychotropic agent is the absence of a Rosetta stone capable of discerning potential clinical utility within a known neurochemical profile. Knowing that an agent has properties in a given set of receptors often gives little indication of its therapeutic potential. The history of pharmacology is marked by the serendipitous discovery of drug effects that were far removed from their initial indications, produced by careful clinical observations. This is particularly the case in psychiatry, in which complex and incompletely understood pathogenic mechanisms underlie most disorders, therefore limiting the success of drugs designed on the basis of their neurochemical profiles. Yet, clinical drug trials are rooted in quantitative research principles, which, although suitable for testing hypotheses, are suboptimal for the systematic detection of unexpected phenomena. Typically, a series of standardized rating instruments are utilized and selected according to the primary and ancillary features of the disorder. Excluding adverse effects, no other nonstandardized outcome data are collected. However, much observational and experiential data are available from encounters with participants in drug trials and are a rich but underutilized source of information. The inclusion of qualitative research methods in clinical drug trials may be one way to address this deficiency, as they provide the framework within which to examine the subjective and contextualized data obtained from clinical trial participants and may also generate hypotheses for further systematic examination through inductive analysis. Qualitative methods may therefore complement and strengthen quantitative methods traditionally used in drug trials by uncovering new information and enhancing the findings of clinical trials.
Indeed, the integration of quantitative and qualitative research methods, or mixed-methods research (MMR), is well developed and is commonly used in psychological and clinical research.1 To the best of our knowledge, however, the use of MMR in drug trials has not yet been explored. It has the potential to offer a wide-angle lens, exploratory view of clinical data that complements the focused, telephoto view provided by specific rating instruments. Given this context, this study aimed to develop methodology to explore the utility of adding qualitative research methods to early-phase drug trials and, in particular, to learn whether additional findings can be gleaned by integrating qualitative with quantitative methods. This methodology was implemented in a recently completed, randomized, placebo-controlled trial of N-acetylcysteine (NAC) in schizophrenia.2 N-acetylcysteine used in this context is analogous to the introduction of a novel agent for this disorder. It acts as a precursor to glutathione and hence as a free radical scavenger, as well as a glutamatergic agent via cysteine glutamate exchange. Akin to many novel agents, NAC has a pharmacologic profile that does not clearly indicate its potential clinical profile of efficacy.3
The detailed methodology of the NAC in schizophrenia trial (www.anzctr.org.au identifier: 2 In brief, the study was conducted from November 2002 to July 2005 across 4 sites in Victoria, Australia, and 1 site in Lausanne, Switzerland. Recruitment occurred predominantly through mental health services. Participants were aged between 18 and 65 years and, in order to be included in the trial, had to meet the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR)4 criteria for schizophrenia; have a Positive and Negative Symptom Scale (PANSS) total score of ≥ 55 or have at least 2 of the positive and/or negative items score > 3; or have a Clinical Global Impressions-Severity (CGI-S) score of ≥ 3. Exclusion criteria included active medical disorders, pregnancy, concurrent use of mood stabilizers, and also current use of 500 mg of NAC per day, 200 Î¼g of selenium per day, or 500 IU of Vitamin E per day. The participants were randomly assigned to treatment with NAC (at a dose of 2 g per day) or placebo in a double-blind manner, while maintained on treatment with their usual antipsychotic medication for the duration of the trial. The participants were assessed at baseline, every 2 weeks for the first 8 weeks, and every 4 weeks thereafter to a total of 24 weeks, whereupon the NAC or placebo treatment was stopped. A postdiscontinuation visit was conducted 4 weeks (± 2 weeks) after the trial concluded.) has been published elsewhere.
A total of 665 people were screened for eligibility in the primary study. Of these, 140 who met the criteria for inclusion were enrolled in the study: 71 were randomly assigned to the placebo group and 69 to the treatment (NAC) group. Of the 140 enrolled participants, 10 participants from the Lausanne site were not eligible, as case notes were not available, and data were missing on 6 further participants, leaving 124 eligible participants in the Australian sites (58 in the NAC group and 66 in the placebo group) who formed the cohort for this study (Figure 1). Demographic data regarding the cohort are published in the primary article.2 The trial was approved by each participating research and ethics committee and was conducted according to Good Clinical Practice guidelines. After receiving a complete description of the study, all participants signed a plain-English Participant Information and Consent Form.
In addition to the standard rating instruments used, which included the PANSS, CGI-S, Global Assessment of Functioning Scale, and the Social and Occupational Functioning Assessment Scale, clinicians at the Australian sites kept detailed case notes on participants; these comprised notes from participants’ verbal descriptions and the clinician researchers’ observations of changes or persistence of symptoms at each visit. The notes were collected and recorded on the case record form in a section designated for qualitative data. All researchers were experienced mental health clinicians (general practitioners or clinical psychologists) and were trained in use of the study instruments. Data were collected following each clinical encounter and entered into the case notes, at which point the clinician researchers summarized their impressions of participant status and progress. These data comprised both subjective participant reports as well as objective clinical observations. Qualitative data were obtained consisting of 2 a priori components: (1) participants’ subjective, verbal descriptions of their symptoms, including persistence and changes over the course of the trial; and (2) clinicians’ descriptions of their observations and assessment of each participant’s mental state. These data were not recorded verbatim, but were succinct summaries of the researchers’ clinical observations, transcribed during the assessment interview or immediately postinterview. The case notes were derived in the course of assessment visits or interviews from participants’ responses to open questions, participant-initiated themes, and elaboration of their responses following prompting by the clinician researcher. The data included notes on participants’ personal circumstances, psychosocial events, symptoms, and adverse events, particularly perceived improvement in or persistence of the aforementioned.
The data were deidentified prior to analysis. All postbaseline data were pooled, generating a compilation of textual items, including subjective reports and researcher observations of changes in participants’ clinical status. These transitions were dichotomized as either "improvements" in specified clinical features or "persistence," reflecting persistence of, aggravation of, or lack of change in aspects of the person’s mental state.
Using an interpretive approach, our analysis aimed to generate theory regarding the use of qualitative methods in the context of clinical drug trials (grounded theory approach).5 The case notes were typed and the textual data were entered into NVivo 2 (2002; QSR International, Cambridge, Massachusetts) for initial analysis. NVivo 2 is a qualitative research software tool that assists in managing, shaping, and making sense of unstructured information. It can be used to classify, sort, code, and arrange textual data for analysis.
The focus of the analysis was on identification and extrapolation of thematic constructs identified during the coding process. The process of coding and analysis described by Miles and Huberman6-8 was utilized. The textual data were imported into NVivo 2. The process then followed a method encompassing overall reading of textual data, searching for similarities and differences, cutting and sorting the data into conceptual categories, creating themes, and then serially examining these themes for further analysis, guided by the study’s aim. Initially, data were screened (combed) in order to isolate individual themes. A preanalysis screening of the narrative files was conducted, as a precoding combing process, which provided a template for potentially emerging themes. The data were subsequently combed between 3 and 7 times for emergent themes, in order to ensure that further themes did not emerge with additional combing (saturation). Coding and recoding of text was done until the presaturation point was attained. The presaturation point was defined by the absence of further emerging themes after approximately 5 combing trials. The remaining themes were then analyzed to establish a pattern of discussion or narrative salience, as extracted from the textual information. Salience was the number of times a transition occurred in a document and was reported. A qualitative salience matrix, a semistatistical matrix containing raw frequency collation of data, as emerging from the textual analysis of the data, was generated within NVivo 2, portraying a listing of all cases (document) coded according to the identified themes, showing frequency salience.
Emergent themes were identified and, where they occurred, were examined for their presence across all cases. Themes did not have to be present in all cases to be included, and all reported changes were analyzed. The final themes agreed upon by the researchers were named and based upon the earlier emergent themes. The next step entailed sentence-by-sentence coding of the textual data according to the overarching twin emergent themes of improvements or persistence in mental-state phenomena, which were recorded. Within these overarching themes, 22 emergent subthemes were dichotomized. The overarching themes are indicative of clinically documented changes in mental state—either improvement or persistence or aggravation of observed or reported clinical features among the participants, not merely the absence or presence of diagnostic psychiatric symptoms. The themes are essentially self-explanatory in their titles (being dichotomized into "improved" or "persistent"), and their interpretation is based on psychiatric symptomatology and operational definitions stipulated by DSM-IV-TR criteria for mental disorders, with specific relevance to individuals with schizophrenia.
Once the themes were identified for each individual, coded, and pooled, the sample was split into active (NAC) and placebo groups by an independent researcher, who was blind to and had no role in the coding process, on the basis of the randomization matrix. In a departure from conventional qualitative reporting, quotations and emergent themes were not highlighted in isolation, with the focus entirely on aggregate data that differentiated the treatment and placebo groups. Hence, statistical analysis to determine the P values of individual themes, based on recorded frequency salience, was performed. Data were analyzed using SPSS Version 15 (SPSS Inc, Chicago, Illinois), and the nonparametric Wilcoxon signed rank test was used. The statistical approach applied allows for the triangulation of the qualitative data to reinforce the credibility or reliability of the results obtained.6
The qualitative analysis of the textual data revealed a number of differences between the NAC-treated and placebo-treated groups. These differences were not consistently reflected in the quantitative analysis. With regard to improvements in mental state, there were a significantly greater number of reports of improved insight, adequate self-care, improved social interactivity, motivation and volition, improved mood stability, and psychomotor stability among participants receiving NAC, compared to those receiving placebo. Persistent parameters reported more frequently as unchanged or worsened among participants taking placebo compared to those taking NAC included dysthymia, auditory hallucinations, social withdrawal, ideas of reference, and grandiosity. There were no improvements more common in placebo-treated than in NAC-treated participants, and no persistent parameters were more common in NAC-treated than in placebo-treated participants. These data are presented in Table 1.
This study has generated 2 broad findings. First, it has verified the quantitative results of the randomized, placebo-controlled trial of NAC in schizophrenia using standard rating instruments, particularly with regard to improvements in negative symptoms and functionality.2 Second, this study has demonstrated that the use of qualitative research methods uncovered a number of improvements or persistence of symptoms that were not evident when only using a quantitative approach. These results provide valuable information on whether the pharmacologic interventions were successful.
The principal findings of the qualitative analysis were additional reports of improvements in insight, self-care, social interaction, motivation and volition, and mood reactivity, as well as more euthymia and psychomotor stability in NAC-treated than placebo-treated subjects. Persistent parameters that were more commonly reported in the placebo group were dysthymia, auditory hallucinations, social withdrawal, ideas of reference, and grandiosity. These results confirm the broad pattern of efficacy seen in the PANSS data, particularly with regard to negative symptoms. In the quantitative study, NAC treatment was associated with a significantly greater improvement compared to placebo treatment on the PANSS negative (−1.83 [−0.32, −3.33], P = .018), PANSS general (−2.79 [−0.2, −5.38], P = .035) and PANSS total (−5.97 [−1.51, −10.44], P = .009) scores at week 24 when compared to baseline. N-acetylcysteine additionally improved scores on the Barnes Akathisia Scale (BAS) (−0.54 [−0.08, −1.0], P = .032). Effect sizes were in the moderate range; 0.43 on the CGI-S, 0.52 for the PANSS negative subscale, 0.57 for the PANSS total, and 0.44 for the BAS. With regard to individual PANSS items, only passive/apathetic social withdrawal differed between groups. No formal measure of mood was used in this study. In contrast, themes relating to positive symptoms emerged in the qualitative data, particularly regarding changes in auditory hallucinations, ideas of reference, and grandiosity that were not consistently seen in the quantitative PANSS data. Of greater value, themes pertaining to parameters not assessed in the original design, in particular mood symptoms, emerged. It was this observation that led us to subsequently study NAC in individuals with mood disorders, in which large effect sizes for the treatment of depressive symptoms with NAC were found.9 This experience strongly confirms the efficacy in utilizing qualitative methods to explore the potential psychotropic profile of a novel agent. Additional clinical signals have since emerged from the bipolar disorder trial,9 which have given us additional clues of potential efficacy in unexpected conditions such as compulsive nail biting.10 In hindsight, there are data that suggest that cystine-glutamate exchange may play a role in the observed obsessive phenomena, an observed target of NAC,11 and there are also now pilot data of NAC in obsessive-compulsive disorder.12
There are examples in the literature of the use of qualitative methodologies to examine aspects of pharmacologic therapies,13,14 to understand the subjective experience of therapies15,16 as well as of the disorders themselves.17 Subjective experiences, such as issues of adherence and acceptance, lend themselves particularly to qualitative methodologies.18-20 There are also examples of the use of qualitative methods in the context of randomized designs, but to our knowledge, qualitative methods have not been used before in the study of the effect of pharmacologic agents on illness symptoms.21-23 Mixed-methods research is commonly used in health services research. This is consequent to the perceived limitations of quantitative methods alone in capturing the complexity of research in health care.24 We are, however, unaware of any prior research using qualitative methods to explore the potential range of efficacy of novel psychopharmacologic agents.
The integration of qualitative methods into clinical trials has its own characteristics and limitations. The efficacy of qualitative data is dependent on the completeness and level of detail available in the source documentation. Additionally, themes that may emerge in a population with a particular disorder are likely to differ from those with a different disorder. The identification of and classification of themes are dependent upon the quality of the narrative data and the clinical and interpretive skills of the clinician researchers. However, in the current study, the effect of this potential for bias is limited, considering that the themes apply to both groups that are compared. A strength of this study is that only 1 rater did the coding, eliminating interrater reliability issues. There may be subjective bias in reporting symptoms. A symptom might be influenced by insight or may not have been so bothersome or prevalent as to create a theme, being dependent on how a given subject perceives his/her illness and prioritizes symptoms. However, the pooling of subjective reports and observer-based reports has the potential to provide more dimensions to the clinical picture. This method of data collection is post hoc and, by definition, not a priori-hypothesis driven. Indeed, the main strengths of this approach is that it is not constrained by the need to have an a priori hypothesis and corresponding measure, and, as a consequence, it has an ability to generate a signal leading to discovery of serendipitous clinical effects. It is therefore better suited to theory generation than hypothesis testing.
The theory generated in this study—that the use of qualitative research methods can lead to an expanded scope of clinical outcomes in clinical drug trials—is grounded in the data and in our observation and interpretation of those data.25 In a sophisticated model of grounded theory, broad sensitizing questions, such as those we posed regarding the potential of this approach to provide broader and deeper insights into the efficacy and profile of a potential psychotropic agent, allow for this kind of inductive theory generation.25
In summary, this study confirms that the integration of qualitative methods into rigorous, placebo-controlled randomized designs provides additional insight into the profile of a potential psychotropic agent and broadens the scope of the trial process. The value of this information is more likely to be greatest in early phase clinical trials, in which the potential efficacy profile of an agent may be quite opaque.
Author affiliations: Department of Clinical and Biomedical Sciences: Barwon Health, University of Melbourne, Geelong (Drs Berk, Munib, Dean, and Hanna and Mss Kohlmann and Otmar); Orygen Research Centre, Parkville (Drs Berk and Conus); Mental Health Research Institute, Parkville (Drs Dean, Schapkaitz, Jeavons, Anderson-Hunt, and Bush and Mss Kohlmann and Katz); School of Psychological Science, La Trobe University—Bendigo Campus, Bendigo (Dr Jeavons); Discipline of Psychological Medicine, University of Sydney (Dr Malhi); Department of Psychiatry, Clinical Assessment Diagnostic Evaluation Clinic, Royal North Shore Hospital, Sydney (Dr Malhi); Monash University, Victoria (Drs Hanna and Copolov); and Discipline of Psychiatry, School of Medicine, University of Adelaide, Adelaide (Dr Ng), Australia.
Potential conflicts of interest: Dr Berk has received grant/research support from the Stanley Medical Research Foundation, MBF, National Health and Medical Research Council, Beyond Blue, Geelong Medical Research Foundation, Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Organon, Novartis, Mayne Pharma, Servier, and AstraZeneca; has been a paid consultant for AstraZeneca, Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Janssen-Cilag, Lundbeck, Servier, and Pfizer; and has been a paid speaker for AstraZeneca, Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Janssen-Cilag, Lundbeck, Organon, Pfizer, sanofi synthelabo, Servier, Solvay, and Wyeth. Dr Bush is a shareholder and consultant for Prana Biotechnology Ltd and a shareholder in Cogstate Ltd. Drs Munib, Dean, Malhi, Schapkaitz, Jeavons, Anderson-Hunt, Conus, Hanna, Ng, and Copolov and Mss Kohlmann, Katz, and Otmar have no personal affiliations or financial relationships with any commercial interest to disclose relative to the article.
Funding/support: The research team would like to thank the Stanley Medical Research Institute for providing funding for this trial.
2. Berk M, Copolov D, Dean O, et al. N-acetyl cysteine as a glutathione precursor for schizophrenia: a double-blind, randomized, placebo-controlled trial. Biol Psychiatry. 2008;64(5):361-368. PubMed doi:10.1016/j.biopsych.2008.03.004
9. Berk M, Copolov DL, Dean O, et al. N-acetyl cysteine for depressive symptoms in bipolar disorder: a double-blind randomized placebo-controlled trial. Biol Psychiatry. 2008;64(6):468-475. PubMed doi:10.1016/j.biopsych.2008.04.022
10. Berk M, Jeavons S, Dean OM, et al. Nail-biting stuff? the effect of N-acetyl cysteine on nail-biting. CNS Spectr. 2009;14(7):357-360. PubMed
11. Grant JE, Kim SW, Odlaug BL. N-acetyl cysteine, a glutamate-modulating agent, in the treatment of pathological gambling: a pilot study. Biol Psychiatry. 2007;62(6):652-657. PubMed doi:10.1016/j.biopsych.2006.11.021
12. Lafleur DL, Pittenger C, Kelmendi B, et al. N-acetylcysteine augmentation in serotonin reuptake inhibitor refractory obsessive-compulsive disorder. Psychopharmacology (Berl). 2006;184(2):254-256. PubMed doi:10.1007/s00213-005-0246-6
13. Munoz-Plaza CE, Strauss S, Astone-Twerell J, et al. Exploring drug users’ attitudes and decisions regarding hepatitis C (HCV) treatment in the US. Int J Drug Policy. 2008;19(1):71-78. PubMed doi:10.1016/j.drugpo.2007.02.003
14. Salter ML, Go VF, Celentano DD, et al. The role of men in women’s acceptance of an intravaginal gel in a randomized clinical trial in Blantyre, Malawi: a qualitative and quantitative analysis. AIDS Care. 2008;20(7):853-862. PubMed doi:10.1080/09540120701742300
15. Chalmers A, Mitchell C, Rosenthal M, et al. An exploration of patients’ memories and experiences of hyperbaric oxygen therapy in a multiplace chamber. J Clin Nurs. 2007;16(8):1454-1459. PubMed doi:10.1111/j.1365-2702.2006.01700.x
16. Phillips L, McCann E. The subjective experiences of people who regularly receive depot neuroleptic medication in the community. J Psychiatr Ment Health Nurs. 2007;14(6):578-586. PubMed doi:10.1111/j.1365-2850.2007.01145.x
18. Verbeek-Heida PM, Mathot EF. Better safe than sorry—why patients prefer to stop using selective serotonin reuptake inhibitor (SSRI) antidepressants but are afraid to do so: results of a qualitative study. Chronic Illn. 2006;2(2):133-142. PubMed
21. Murtagh MJ, Thomson RG, May CR, et al. Qualitative methods in a randomised controlled trial: the role of an integrated qualitative process evaluation in providing evidence to discontinue the intervention in one arm of a trial of a decision support tool. Qual Saf Health Care. 2007;16(3):224-229. PubMed doi:10.1136/qshc.2006.018499
22. Fitzsimons CF, Baker G, Wright A, et al. The ‘ Walking for Wellbeing in the West’ randomised controlled trial of a pedometer-based walking programme in combination with physical activity consultation with 12 month follow-up: rationale and study design. BMC Public Health. 2008;8(1):259. PubMed doi:10.1186/1471-2458-8-259
23. Murphy AW, Cupples ME, Smith SM, et al. The SPHERE Study: secondary prevention of heart disease in general practice: protocol of a randomised controlled trial of tailored practice and patient care plans with parallel qualitative, economic and policy analyses (ISRCTN24081411). Curr Control Trials Cardiovasc Med. 2005;6(1):11. PubMed doi:10.1186/1468-6708-6-11
24. O’ Cathain A, Murphy E, Nicholl J. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res. 2007;7(1):85. PubMed doi:10.1186/1472-6963-7-85