How Accurate Must Cost-Effective Suicide Risk Prediction Models Be?

The researchers who developed an economic evaluation to estimate threshold values of sensitivity, specificity, and positive predictive value (PPV) that suicide risk prediction methods must reach to be cost-effective among primary care patients in the United States found that several statistical suicide risk prediction models exceed these accuracy thresholds. Their research was published in JAMA Psychiatry.

The study authors used decision analytic models to simulate use of suicide risk prediction methods, specifying predictive accuracy, to identify high-risk individuals who would either be offered an active contact and follow-up (ACF) intervention or a cognitive behavioral therapy (CBT) intervention, with various sensitivity and specificity values, and simulated clinical and economic outcomes. The outcomes they evaluated were fatal and nonfatal suicide attempts, quality-adjusted life years (QALYs) and health care sector and societal costs (both in 2016 US dollars). Societal costs included patient time and lost productivity. QALYs and future costs were discounted 3% annually.

They then calculated the incremental cost-effectiveness ratio (ICER) of suicide risk prediction and intervention, which they set at a maximum of $150,000, over a lifetime horizon among a population of US adults with a primary care physician.

They also developed a state-transition model of true annual suicide risk among population, calibrating this distribution to reproduce suicide attempt and suicide death rates of 175 per 100,000 person-years and 15 per 100,000 person-years derived from individuals 15 years or older in national emergency department and inpatient databases and fraction of suicide attempts preceded by a prior attempt of 54%.

In the ACF intervention, a 2015 meta-analysis of randomized clinical trials found a relative risk of suicide attempts of 0.83. The researchers assumed a cost of a structured suicide safety plan, a cost per nonphysician telephone call based on CMS reimbursement, and other baseline assessments to be a total health care cost of $96.

In the CBT intervention, relative risk of suicide attempt, found through another meta-analysis, was found to be .47 with a total health care cost of $1088 and a lost productivity cost of $385 due to patient time requirements.

Suicide risk prediction could be cost-effective for targeting a safety planning and telephone call intervention if specificity were at least 95% and sensitivity were at least 17%.

With usual care, the rate of suicide death was 15.34 and suicide attempt rate was 174.15 per 100,000 person-years. If used to target ACF, the risk prediction method reduced the population’s suicide death rate by .52 and the attempt rate by 5.90 per 100,000-person years, with threshold PPV of .8% for suicide attempt and .7% for suicide death. If used to target CBT, risk prediction reduced the suicide death rate by 1.56 and attempt rate by 17.76 person-years, with threshold PPV of 1.7% for suicide attempt and .2% for suicide death.

Limitations of the study included uncertainty regarding primary and secondary effects and lack of accounting for aspects involved in risk reduction interventions.

References

Ross EL, Zuromski KL, Reis BY, Nock MK, Kessler RC, Smoller JW. Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US. JAMA Psychiatry. Published online March 17, 2021. doi:10.1001/jamapsychiatry.2021.0089