How Confident Are You in What You Know?

Ironically, when many people talk about science-based approaches to bodybuilding and powerlifting, they often do so in black and white terms: something is either “evidence-based”, or not. I say this is ironic, because the actual studies these approaches are based on always report the probability that their findings are correct. Not only that, but different types of scientific studies themselves exist on a continuum; some publications carrying much more “evidential weight”, while others serve as very weak evidence in comparison, just as a function of their design.

So why is there this disconnect? I believe some of it comes down to personality. Most people who are drawn to scientific inquiry as a means to inform their practice as an athlete or coach, also tend to be analytical thinkers. It is a good thing to be an analytical thinker, but in my experience most analytical thinkers are also most comfortable with clear cut, objective answers and thus, they stray towards viewing information in absolute terms. That’s the other side of the coin which is the Achilles heel for an analytical thinker which you have to be mindful to keep in check. Because in reality, science is an ever-evolving beast built on statistics that at best can provide probabilities of correctness and disprove falsehoods, but almost never provides immutable truth.

So how can we align our interpretation of science-based information with the science itself? One very useful way which I was originally exposed to in a presentation by Swedish science communicator Jacob Gudiol at the Academy For Personal Trainers 2017 convention in Oslo Norway, was the concept of having levels of confidence in your knowledge.

Adapting this concept and putting it in my own terms, this means that for something “evidence based” you might have a very high confidence level of 1 (the highest) or only a theoretical leaning with a confidence level of 5 (the lowest). Below I have outlined my take on this concept with 5 levels, their guidelines so you can assess your current knowledge and perhaps what level of confidence you should have for various aspects of it, and common training and nutrition examples to help you see how these levels can be applied.

Level 1 

This level of confidence is as close to an immutable truth as we get in science. When multiple meta-analyses (statistical analyses of many studies on the same topic combined to give a bird’s eye view of the literature on a topic) have been performed over multiple eras, all confirming the same outcome, you can be pretty damn confident in this knowledge. For example, the concept that if you are in an energy deficit you will lose mass is not even debatable (at least using facts). I challenge anyone to find a single study where subjects were placed on an energy restricted diet in a metabolic ward (ensures perfect adherence and gives a measurement of energy expenditure to ensure a deficit is imposed) where body mass losses didn’t occur. Another example would be the dose response curve of training volume with strength and hypertrophy: as volume of training increases, so too does the rate of strength gain and hypertrophy, until it begins to plateau, then slow, and then on the extreme far end of very high volumes of training, backslide. Not only does every adaptation to exercise occur in such a manner (that’s just how we work as a survival mechanism), but we have meta analyses spanning the early 2000’s to those being published just recently confirming this outcome [1-7].

Level 2 

Knowledge at this level can be discussed from a position of strong confidence. At this level, there might be only a single meta-analysis, or perhaps a group of controlled trials in which the majority have the same finding, and only an odd, underpowered study has a finding where the two compared variables in question are not significantly different. While not quite as strong as level 1, level 2 knowledge is certainly at a level of confident application for coaches, and promotion for science communicators. An example might be that a protein intake of at least 1.6g/kg is needed to confidently maximize the strength and hypertrophy response to resistance training in most individuals. While a previous meta-analysis in 2012 found that protein supplementation in and of itself improved these outcomes [8], this meta wasn’t able to provide a specific guideline for total daily intake. It wasn’t until recently that a meta-analysis was published that did just that [9]. Information at level 2 tends to be more specific than level 1, which is typically relegated to broader, more universal truths (such as the volume of exercise influences the adaptation, or that we lose body mass without the energy to support it). This specificity is partly why the confidence tends to be a bit lower, because in some situations and in some individuals, the relationship might change a bit. For example, it may be that protein intake should be higher when dieting [10] or for certain outlier individuals with a higher requirement during intense phases of training [11].

Level 3

At this level, you may have one or two studies supporting a finding, without any data to the contrary. Not many studies have been done, but the one or two studies that have been performed are based on a sound underlying theory, and their outcomes match what is seen in the field. As a practitioner, it is certainly reasonable to apply information at this level, just be aware that while it’s unlikely these findings will do a complete 180-degree flip, in the long run data might end up showing the findings aren’t all they were originally cracked up to be. For example, a recent study found that taking periodic 2-week diet breaks where you eat at controlled maintenance intake (not ad libitum) during a weight loss period enhances fat loss and curtails the adaptive reduction in resting energy expenditure that occurs due to dieting compared to a diet without breaks [12]. Prior to this 2017 study, we only had one study done back in 2002 on diet breaks showing that a similar system using uncontrolled breaks did not interfere with weight loss, but also did not slow it down [13]. As I said, it’s unlikely these findings will completely flip; i.e. I seriously doubt a study will be published showing that taking 2 weeks off from dieting and eating at maintenance results in a decrease in resting energy expenditure and makes weight loss harder when you go back to dieting. However, it is possible that diet breaks might end up only making a mental difference without helping fat loss physiologically…although I doubt that is the case given the outcome of the recent study and our field observations of clients using diet breaks.

Importantly, this is the first level where you really need to consider risk management; meaning, you have to consider the potential downside of applying this information if it ends up being false. For example, if diet breaks end up not helping fat loss, I have added to the length of time an athlete needs to prep for…granted the added time isn’t hard as it is at maintenance, and we account for this by using longer preps. But, that is exactly the type of consideration you should go through before implementing a strategy that you aren’t completely confident in. However, sometimes the downside is hardly worth worrying about. For example, contrary to what we used to believe in the scientific community, the benefits of post workout protein intake probably aren’t much to write home about if you are consuming a high protein diet already [14], but there is really no potential harm in having some of your daily protein intake within 2 hours of training.

Level 4

At this level, there is actually no high quality direct evidence supporting the knowledge. However, it is based on sound theory, there is some published indirect or low quality (not a controlled study) evidence that supports it, and it matches field observations. Knowledge at this level can certainly be communicated as evidence-based, but often requires disclaimers and context for when it would be useful or how it should be applied (i.e. only applicable to athletes, and only in certain situations), and may indeed change as more data is produced. For example, the claim that muscle group training frequencies higher than 2-3x/week are beneficial for advanced lifters to optimize strength and hypertrophy is a level 4 claim. This claim is based on the theory that as higher stress levels are needed to continue making gains as you progress your lifting career, it may be better to distribute that stress over more days per week so as to avoid any single day from becoming too taxing and diminishing the quality of the work performed. In addition, it’s been theorized that because the protein synthetic response to training stays elevated for shorter time periods in well-trained lifters, more frequent sessions may help to prevent the slowing of muscle gain [15]. In addition to these being reasonable theories, there is a conference abstract (a scientific study that was presented, but has not yet gone through peer review and been published in a journal) which found high level powerlifters gained more strength and muscle mass when splitting their volume over 6 vs 3 sessions [16]. In addition, there is data showing that splitting up the volume of a single session into multiple daily sessions can improve the adaptive response in well trained strength athletes [17-20]. However, systematic reviews and meta analyses on this topic have found that frequencies of training muscle groups and movements 2-3x/week consistently outperform lower frequencies for both hypertrophy and strength. Also important, in these publications it is specifically stated that there is not enough data to assess frequencies above 2-3x/week [4, 5, 21, 22]. Indeed, the claim that a 2-3x/week muscle group or movement pattern frequency is superior to 1x/week when all else is equal (and when using reasonable volumes and training loads), is arguably a level 1 or 2 claim. But claims made about even higher frequencies simply can’t be made as confidently. For example, a study on untrained lifters found no significant differences between a muscle group frequency of 2 vs 3x/week with matched volume [23], likewise a recent study on recreationally trained lifters (men with roughly a bodyweight bench press) found no significant differences between a 2 and 4x/week frequency with matched volume [24], and finally a conference abstract from 2016 found no significant differences between a 3 and 6x/week frequency with matched volume in recreationally trained lifters (men with a 1.25, 1, and 1.5x bodyweight squat, bench press, and deadlift respectively). Thus, it seems that not only is there no high-quality data showing frequencies beyond 3x/week could benefit high level lifters, there is indeed data showing it likely won’t benefit novice and intermediate lifters.

In the face of such data, you have to couch your conclusions and consider that you could be wrong. For example, while I am reasonably confident that frequencies higher than 3x/week could benefit high level physique and strength athletes, it’s also possible that some minimal level of total session stress (the stress from the combined volume and intensity in that session) has to be reached per session to optimize progress, and that the required weekly stress needed to succeed even at the highest levels of bodybuilding and powerlifting just doesn’t quite necessitate muscle group or movement frequencies higher than 3x/week in most competitors.  Additionally, there is a potential risk of harm if high frequency is misapplied. Even if muscle would theoretically adapt more quickly to high frequency training, it is possible that other soft tissues (tendon, ligament, etc) would not and this could predispose someone to an injury. This is even more likely if a very high frequency is used with higher risk movements. Some people who experience manageable discomfort and tightness when performing movements 1-3x/week might develop a full-blown overuse injury when moving to a higher frequency. Also, many people simply misapply such concepts, and run a high frequency program with a high volume and intensity, and get injured quite catastrophically (which I’ve seen happen too many times). In situations such as these, where your confidence level is low for a belief (but you still believe it to be true), and there is a risk to its application, you should be very conservative in its application and promotion.

Level 5 

Finally, there is level 5 information. At this level, there is sound theory and field observations, but not much else. Or, there may be some scattered indirect evidence, but it’s preliminary to say the least. For example, one or two studies on a supplement, done in vitro in a petri-dish or in animals showing the supplement positively impacts a marker that theoretically should be beneficial for hypertrophy, strength or fat loss (but the effect directly on hypertrophy, strength or fat loss wasn’t measured), would fit this category. Information at this level should not be completely dismissed, as it hasn’t been disproven, rather, it just hasn’t been directly studied. I wouldn’t advise buying a supplement with only this data, as it is just as likely to waste your money as it is to do anything useful (arguably more likely), and if it’s totally new its safety probably hasn’t been well established. Rather, just wait until it is studied.

On the other hand, for training and nutrition strategies (versus just taking a supplement), sometimes it’s unlikely it will ever be studied because of logistical issues. For example, a specific dietary or training strategy for bodybuilding or powerlifting is hard to rigorously study in a controlled setting because you would have to get two groups of athletes willing to let you manipulate and control their nutrition and training, and the placebo group has to be okay with not getting to do the potentially beneficial strategy. But, just because there isn’t good data doesn’t mean you should do nothing. If the risk of harm is low, the theory is sound, and it’s been field tested and seems to work well in practice, go for it! It’s certainly a better choice than doing nothing. For coaches and science communicators, information at this level should be promoted as anecdotal and/or theoretical, and it should be made clear that there is not really strong science to base this belief on. Indeed, you could say this is “evidence based”, but you should clarify that the evidence it is based on is your field observations and it is theoretical.

Thank you for sticking with me through this long article, I think it’s quite important for anyone interested optimizing their athletic careers or helping athletes through science. Remember, science is not black and white but those interested in it have a tendency to put things in those terms. Assess your confidence in a belief based on the available evidence, and assess the risk of harm before implementation. For coaches and communicators, state the level of confidence and the risks, the caveats and the context along with the information you put out to ensure you aren’t doing more harm than good!

Also, if you enjoyed this article about how to apply science, check out Monthly Applications in Strength Sport where I do this sort of thing every month with my colleagues Dr. Mike Zourdos and Greg Nuckols for specific studies related to strength and physique sport.

References

1. Krieger, J.W., Single versus multiple sets of resistance exercise: a meta-regression. J Strength Cond Res, 2009. 23(6): p. 1890-901.
2. Schoenfeld, B.J., D. Ogborn, and J.W. Krieger, Dose-response relationship between weekly resistance training volume and increases in muscle mass: A systematic review and meta-analysis. J Sports Sci, 2017. 35(11): p. 1073-1082.
3. Krieger, J.W., Single vs. multiple sets of resistance exercise for muscle hypertrophy: a meta-analysis. J Strength Cond Res, 2010. 24(4): p. 1150-9.
4. Peterson, M.D., M.R. Rhea, and B.A. Alvar, Maximizing strength development in athletes: a meta-analysis to determine the dose-response relationship. J Strength Cond Res, 2004. 18(2): p. 377-82.
5. Peterson, M.D., M.R. Rhea, and B.A. Alvar, Applications of the dose-response for muscular strength development: a review of meta-analytic efficacy and reliability for designing training prescription. J Strength Cond Res, 2005. 19(4): p. 950-8.
6. Rhea, M.R., et al., A meta-analysis to determine the dose response for strength development. Med Sci Sports Exerc, 2003. 35(3): p. 456-64.
7. Ralston, G.W., et al., The Effect of Weekly Set Volume on Strength Gain: A Meta-Analysis. Sports Med, 2017. 47(12): p. 2585-2601.
8. Cermak, N.R., et al., Protein supplementation augments the adaptive response of skeletal muscle to resistance-type exercise training: a meta-analysis. Am J Clin Nutr, 2012. 96.
9. Morton, R.W., et al., A systematic review, meta-analysis and meta-regression of the effect of protein supplementation on resistance training-induced gains in muscle mass and strength in healthy adults. Br J Sports Med, 2017.
10. Hector, A. and S.M. Phillips, Protein Recommendations for Weight Loss in Elite Athletes: A Focus on Body Composition and Performance. Int J Sport Nutr Exerc Metab, 2017: p. 1-26.
11. Houltham, S.D. and D.S. Rowlands, A snapshot of nitrogen balance in endurance-trained women. Appl Physiol Nutr Metab, 2014. 39(2): p. 219-25.
12. Byrne, N.M., et al., Intermittent energy restriction improves weight loss efficiency in obese men: the MATADOR study. Int J Obes (Lond), 2017.
13. Wing, R.R. and R.W. Jeffery, Prescribed “breaks” as a means to disrupt weight control efforts. Obesity Research, 2003. 11(2): p. 287-291.
14. Schoenfeld, B.J., A.A. Aragon, and J.W. Krieger, The effect of protein timing on muscle strength and hypertrophy: a meta-analysis. Journal of the International Society of Sports Nutrition, 2013. 10(1): p. 53.
15. Dankel, S.J., et al., Frequency: The Overlooked Resistance Training Variable for Inducing Muscle Hypertrophy? Sports Med, 2017. 47(5): p. 799-805.
16. Raastad, T., et al., Powerlifters improved strength and muscular adaptations to a greater extent when equal total training volume was divided into 6 compared to 3 training sessions per week, in 17th annual conference of the ECSS, Brugge 4-7 2012.
17. Hakkinen, K. and M. Kallinen, Distribution of strength training volume into one or two daily sessions and neuromuscular adaptations in female athletes. Electromyography and Clinical Neurophysiology, 1994. 34(2): p. 117-24.
18. Hakkinen, K. and A. Pakarinen, Serum hormones in male strength athletes during intensive short term strength training. European Journal of Applied Physiology and Occupational Physiology, 1991. 63(3-4): p. 194-9.
19. McLester, J.R., Bishop, E., Guilliams, M.E., Comparison of 1 day and 3 days per week of equal-volume resistance training in experienced subjects. Journal of Strength and Conditioning Research, 2000. 14(3): p. 273-281.
20. Hartman, M.J., et al., Comparisons between twice-daily and once-daily training sessions in male weight lifters. International journal of sports physiology and performance, 2007. 2(2): p. 159-69.
21. Schoenfeld, B.J., D. Ogborn, and J.W. Krieger, Effects of Resistance Training Frequency on Measures of Muscle Hypertrophy: A Systematic Review and Meta-Analysis. Sports Med, 2016. 46(11): p. 1689-1697.
22. Wernbom, M., J. Augustsson, and R. Thomee, The influence of frequency, intensity, volume and mode of strength training on whole muscle cross-sectional area in humans. Sports Med, 2007. 37(3): p. 225-64.
23. Candow, D.G. and D.G. Burke, Effect of short-term equal-volume resistance training with different workout frequency on muscle mass and strength in untrained men and women. J Strength Cond Res, 2007. 21(1): p. 204-7.
24. Fu, Y., et al., Comparison of Two Equated Resistance Training Weekly Volume Routines Using Different Frequencies on Body Composition and Performance in Trained Males. Appl Physiol Nutr Metab, 2017.