We Run Clinical Trials for Medications, Why Not for Apps?

Give this a moment of thought: We are required to run clinical trials on medical treatments in order to not only demonstrate that they work (efficacy), but also to assess whether or not they are safe before rolling them out wholesale across the entire population. Safe for the users, safe for others (such as offspring, vis-à-vis teratogenicity studies, etc).

Safety is never assumed in clinical trials — indeed, safety is the principally necessary outcome of clinical trials before efficacy will even be weighed.

Now, how about a (global) population-wide research study — one for which there is no oversight, no Institutional Review Board (IRB) to guide or evaluate ethics, no studies of safety or efficacy? Shockingly, these now happen daily with smartphone apps: Tools for systematic behavioral modification which have no discernable governance. These mobile apps literally saturate the population and dramatically affect human behavior every day and we only have post hoc assessments in the form of a small number of frail studies with light, equivocal evidence, about their day-to-day impact on individuals.

The First Clinical Trial

James Lind is considered the father of the first clinical trial. He was a physician on board the HMS Salisbury in 1747 when scurvy was a serious deterrent to lengthy sea travel. He wondered how he could best evaluate different treatments in the men onboard, so he divided 12 men up into 6 pairs and gave each pair a different concoction, in an attempt to determine for which scurvy was treated or prevented. His armamentarium of treatments for the different pairs of seamen were:

·   A quart of cider per day

·   25 drops of elixir vitriol (sulfuric acid), 3x/day

·   2 spoonfuls of vinegar, 3x/day

·   Seawater

·   2 oranges and 1 lemon per day

·   Electary recommended by hospital surgeon¹

What he found was that only the group offered citrus fruits had their scurvy symptoms resolved — to the point where they actually returned to deck and work on the ship.

Lind wrote, “The consequence was, that the most sudden and visible good effects were perceived from the use of oranges and lemons; one of those who had taken them, being at the end of 6 days fit for duty.”¹ This became a dramatic example of the first clinical trial, which he documented in his book, A Treatise of the Scurvy (1753).

So we’ve known about the best ways to scientifically explore and compare treatments head-to-head for 270 years, yet one of our most innovative industries (tech sector) doesn’t do anything of the sort. They’ll conduct what’s called an “A-B Analysis” to compare consumer feedback on 2 different offerings (“A” vs “B”), but this is wholly unscientific without control groups nor a null hypothesis to test. Put another way: Our world of technology prides itself in making its users and their data “quantified,” but it is not itself quantified in a meaningful way. It seems both ironic and hypocritical to do something to others that you’re not actually practicing yourself.

And when the metrics driving app production are ones such as “number of downloads,” “number of repeat uses in the first 30 days,” and “number of in-app purchases,” it’s clear that these apps are not only directly driving behavior, but that “pulling the food lever” (BF Skinner, operant conditioning)² is all that matters to the app developers. None of the metricization has anything whatsoever to do with understanding how the apps impact or interfere with daily living, and to allostatic stress load, and so on. It’s well-known that online time can be associated with diminished feelings of worth, anxiety disorders, compulsive behaviors, and many other negative impacts.³

What Would Be Recommended?

While this list of assessing layered harms is designed to be nonexhaustive, it does provide a sort of litmus test which, if an app candidate answers “Yes” to all 3 of the above assessment questions, would almost certainly render a negative evaluation in a clinical trial context. Even if generalizability between the 2 aren’t 1:1, it’s clear that there are smart ways to evaluate these things which our friends, families, and children are downloading daily; Doing nothing is the exact wrong answer.

There are millions of apps, and the total of available apps continues to accrue at a blistering pace. While policing and user-testing all of them would be practically impossible, those apps which have the potential to be downloaded by over a million users and influence their daily behavior also have the implicit ability to do great harm. That’s not hyperbole, it’s patent truth.

It seems almost criminal that companies and individuals are able to unleash a blizzard of behavior — and thinking-modifying apps onto an unsuspecting public, without some types of smart controls on how and where these are being downloaded, consumed, and used.

Ben Locwin, PhD, is a health care executive, public health expert, and philanthropist.

References

1.  Lind, J. A treatise of the scurvy. James Lind Library. Published 1753. Accessed March 16, 2022. https://www.jameslindlibrary.org/lind-j-1753/

2. Richelle MN. Skinner, Burrhus Frederick (1904–90). International Encyclopedia of the Social & Behavioral Sciences. Published 2001. Accessed March 16, 2022. https://www.sciencedirect.com/science/article/pii/B0080430767003375

3.  The social dilemma: social media and your mental health. McLean Hospital. Published January 21, 2022. Accessed March 5, 2022. https://www.mcleanhospital.org/essential/it-or-not-social-medias-affecting-your-mental-health