OK so now I have gotten your attention.
Have read of this comic. Lets talk after you finish reading it.
So what is all this about then ?
You mean what apart from highlighting how news articles cherry pick research findings and just confuse us to get a headline?
Often you see things like (p < 0.05) in a journal abstract. What does that mean? Well it is based on probability theory and relates to the confidence of one thing causing another.
What is a P-value?
The “P-value” is a measure of how likely something is to occur compared to just pure chance. For us who read medical abstracts, often we seem them used in statements like “Treatment X Significantly Reduced Symptom Y (p<0.05)”.
Here the value 0.05 means the probability that the result is related to pure chance is less than 5/100 or 5%.
This says that there was a strongly significant link between using Treatment X and seeing your Symptom Y reduced. The element of chance has been mostly eliminated as contributing to the result.
Once chance has been reduced to less than 5% researchers can say that there is strong significance in the results. This is the best result for a researcher looking for a new treatment; being able to use statistics to prove their treatment actually works.
The Punch Line
By the way, the punch line from the comic, visible if you hover over the graphic was “So, uh, we did the green study again and got no link. It was probably a “RESEARCH CONFLICTED ON GREEN JELLY BEAN/ACNE LINK; MORE STUDY RECOMMENDED!”
So there you go, our comic researcher just made a mistake 🙂
Want to ask a Question?
Statistics and probability theory can be quite complicated. Is there something you always wanted to ask about this topic? Leave it in the comments below and I’ll do my best to answer it.
I think you didn’t get at all what the comic is about and what the actual mistake of the scientists is.
This comic is about the fact, that if you test a lot of parameters regarding one hypothesis/research question, you increase the likely hood of getting a false positive statistically significant result.
What it actually means is, in this case you HAVE TO correct your analysis for so-called “multiple comparisons”. This test re-evaluates your cut-off p-value to make the conclusion that a result was actually statistically significant. If you do that, you don’t make the false assumption that you found a “real” result in the first place.