Secondary Outcome Bias Prevalent in Antipsychotic Randomized Controlled Trials

If evidence-based practice means making clinical decisions based on the best available evidence at the time, then those decisions, aside from the valuable contributions of clinical experience and judgment, are only as informed as the evidence is reliable. But what if more information about new evidence is available than most clinicians may realize?

It’s possible that understanding more about the original protocols of randomized controlled trials (RCTs) for various drugs may offer helpful insights into the clinical implications of those trials’ findings.

Results from a study published in Translational Psychiatry show that in RCTs examining antipsychotics, published findings differ from the original trial protocols more often than not, particularly with secondary and safety endpoints.

“As a clinical psychiatrist, you read a lot of RCTs and are constantly busy with all this new evidence coming up, but you never compare what authors describe as their initial protocols,” Jurjen Luykx, MD, PhD, an assistant professor at the University Medical Center Utrecht’s Brain Center Rudolf Magnus in The Netherlands, told Psychiatry Advisor.

“As we found out, a lot happens from the protocol to publication that may actually impact the results as they are published.”

The primary endpoints differed between original protocol and published results in just under 1 in 4 trials, but the secondary endpoints differed in about 4 in 5 trials, Dr Luykx’s team found. Interestingly, the differences did not appear to be influenced by funding sources or by the directionality of results.

Whereas research in other areas of medicine has sometimes found publication bias from positive findings and/or potential bias in industry-funded trials, the majority of differences Dr Luykx and colleagues found arose from additional safety endpoints in the final manuscripts. In fact, the authors found more than 5 times more published safety endpoints than those prespecified in the study’s original protocol at ClinicalTrials.gov.

“Either people don’t reflect that much on the secondary outcomes [initially] or they emerge as they conduct clinical trials,” Dr Luykx told Psychiatry Advisor.

It would likely be helpful in clinical practice if authors reflected on the reasons those additional safety endpoints became more relevant during the trial, Dr Luykx said, but it is up to journal editors to begin asking for that. Until then, clinicians can compare prespecified and published endpoints just as the authors did in their study.

Comparing Prespecified Endpoints With Published Endpoints

The researchers first looked for all clinical trial registrations at ClinicalTrials.gov for studies on antipsychotics for schizophrenia or schizoaffective disorder entered between January 2006 and December 2013. They limited their search to phase 2 and 3 trials no longer enrolling participants (“closed”) and then searched PubMed for matching published studies at the end of 2015.

They also excluded trials with “indications other than schizophrenia or schizoaffective disorder…antipsychotics not listed as main treatment, use of non-[US Food and Drug Administration-]approved antipsychotics, use of one or more concomitant interventional drug(s) besides antipsychotics…and ongoing and prematurely terminated RCTs and non-randomized RCTs.”

(Dr Luykx’s team started with antipsychotics trials because they had already been studying schizophrenia. They are now conducting a similar study for antidepressant trials.)

Of the 136 trials they identified, 72 had no published study in PubMed and 16 did not have a matching study, leaving 48 RCTs for analysis. These had a median enrollment of 319.5 participants and investigated 13 antipsychotics, including 23% of studies that studied paliperidone. Most of the studies (63%) were phase 3, multicenter (73%), and industry funded (81%).

The researchers then compared primary and secondary (or “other”) reported outcomes in ClinicalTrials.gov with those in PubMed. They allowed for slightly different phrasing (eg, “metabolic syndrome clinical findings were considered equal to BMI and weight circumference”) to be more conservative about identified differences. They also noted funding sources and whether primary outcomes were negative (null hypothesis) or positive.