A Framework for Health Guidance
Many digital health apps are providing some sort of health guidance. Blog posts, digital coaches etc. provide education to users, hopefully with the goal of empowering them to make better informed decisions regarding their health.
While some companies do a great job at this, others don't. A start-up that's strapped for cash will probably not hire a physician with deep knowledge of evidence-based medicine for blog articles that a copywriter can spew out at a fraction of the cost. Some companies might also turn the findings of obscure, low-quality studies into 'novel' educational content to provide the user with information that will set them apart from the competition. There are also people who are very convinced of their own opinion on a topic and look for studies to support it, without paying attention to the actual quality of the evidence that is presented.
Even if everybody on a team has the best intentions and is reasonably qualified, there can be disagreements as to what people consider sufficient evidence. However, it is crucial for an app to ensure that its content speaks with one voice and this in turn requires a shared understanding of what evidence-based actually means. Different perspectives can result in content that either sounds incoherent our outright conflicting which will cause users to lose trust in the app.
Therefore, I'd like to present a framework based on three principles for evidence-based health guidance in addition with my reasoning for choosing the respective levels of evidence. It's based on the way I approach evidence-based medicine for health guidance (especially in digital health apps) and I would love to hear in what way it fits to or differs from the way you do it!
Principle 1
'Any health guidance that recommends a lifestyle change in order to obtain a health benefit requires one of the following to support the alleged health benefit:
- A high-quality, randomized controlled trial (e.g. an RCT of exercise vs. no exercise for preventing fractures from falling in the elderly)
- High-quality - ideally prospective - observational data, that shows a strong, clinically relevant effect (e.g. increased risk of all-cause mortality from smoking or obesity)
If there is neither, there should be no recommendation based on alleged health benefits.'
I admit, these are pretty strict criteria, but I believe they are necessary.
If we tell people, that something is beneficial for their health, we have to be very certain that there is a causal relationship between the two, i.e. that changing the behavior will also change their health. While there are plenty of studies on the association or correlation of certain behaviors and health outcomes, only the two methods above give us good certainty to assume a causal relationship. For retrospective data or prospective observational/uncontrolled data that shows a marginal effect, the chance of some hidden confounder being at play will always be there.
Now some people may ask:
'Is it really that bad if there is no causal relationship and we give them wrong advice as long as the advice is not dangerous?'
I think it is.
- If we tell people that they should do something to get a health benefit, they might actually do it despite not liking it which reduces their quality of life. If the recommendation is wrong, people sacrifice quality of life for literally no benefit.
- We can only focus on a limited amount of things to change in our life. Self-control is a scarce resource. If we turn every retrospective garbage that associates some random micronutrient with health outcomes into a recommendation, we confuse people. Now it seems like there are thousands of options they have to chose from, when in reality, there are about a handful that will give them meaningful benefits (being active, losing some weight, smoking cessation etc.).
Principle 2
'For health guidance that suggests an activity to accomplish a lifestyle change, the level of evidence can be low as long as it is phrased as a suggestion and is unlikely to cause harm.'
Why can we allow ourselves to be less strict here? Let's look at an example: We want to help users be more physically active. We have good data for a causal relationship between the suggested lifestyle change (i.e. increase physical activity) and a health benefit (e.g. reduced cardiovascular mortality).
Now we suggest the user activities to accomplish this lifestyle change, e.g. 'pack your bag and prepare your outfit the day before going to the gym'.
Why don't we need high quality data to support any of the activity suggestions? Because the users can immediately see for themselves if the suggestion is actually increasing their likelihood of going to the gym or not! There is no 10 year delay like for most health outcomes. If it doesn't help them, they will stop right away.
To keep it fair and transparent, we should phrase those tips as suggestions and not instructions and try to make sure that the users don't get hurt when they try.
Principle 3
'Recommendations can also come in an implicit form. If there is an implicit recommendation to change a lifestyle for health reasons, the recommendation has to fulfill the data requirements mentioned in principle 1.'
Implicit recommendations can be very tricky to spot. Doing it requires you to go through your app with the mindset of someone who doesn't know nearly as much about the topic as you do. As a more obvious example, let's say your app helps people to live with and alleviate back pain. On your blog, there is an interview with someone who successfully recovered from back pain who describes his daily routine.
Due to the situation (blog for back pain, former person with back pain describing his routine), there is a high likelihood that some of the things in the routine will be perceived as an implicit recommendation. If the person has a 30 minute stretching routine he goes through every day, you should have a look at the data that supports the benefit of stretching for alleviating back pain. If there is good data, then there is no problem. However, if there isn't, you should:
- Try to remove the implicit recommendation or
- Provide explanations and context to the users
It should go without saying, but if you decide to provide an explanation, it is important to put it somewhere where the user is likely to see it at the moment he sees the implicit recommendation!
Principle 4
'When there is data to suggest that a behavior has a detrimental health effect, specific subgroups should not be declared as safe unless there is high quality data to support this. The same also applies for behaviors that have a positive health effect.'
This last point is somewhat of an edge case, but comes up surprisingly often. For example, when there is a study associating meat consumption with colon cancer, some people immediately jump in to say that this only applies to highly processed or high fat meats (cause that is what a lot of people in the study ate) and that nobody has ever demonstrated that effect in their favorite subgroup (chicken, wagyu etc.). By following this logic you could always define increasingly small subgroups and declare them as safe which doesn't make sense for health guidance. Instead we should be honest about the fact that the behavior in general comes with a health detriment and that some subgroups might be exempt, but that nobody knows for sure.
Closing thoughts
I might update this article if I see the need to add more or edit existing principles. If you have other principles, I'd love to know about them. I believe that every organization that is active in the field should have some principles written down as adhering to them is not easy when you have other factors like UI design or driving engagement that might sometimes create pressure to deviate from some of the principles. Having something to reference can help!