How to read a CI, p-value, and Cohen's d

If you have seen a nutrition headline and wondered whether a small p-value meant a big result, or whether a confidence interval crossing zero meant the study found nothing, this explainer is for you. After reading, you should be able to read confidence intervals, p-values, and Cohen’s d more carefully, using two sentences from Nutritious News coverage as worked examples.

What this means

A confidence interval shows the range of plausible values and precision. If it crosses zero, the data are compatible with no difference—but that is not proof of no effect. A p-value shows how unusual the data would be under a null; p<0.05 is a convention, not a measure of importance. Cohen’s d shows standardized size; rules of thumb do not tell you whether a change matters in real life.

Confidence intervals: a range, not a verdict

A confidence interval is a range of plausible values for an estimate under the study’s model. A 95% confidence interval is often described this way: if the same study were repeated many times, about 95% of such intervals would capture the true value. It is not a 95% probability that the true value sits inside this one interval. Narrow means more precise; wide means more uncertainty.

For a difference, zero means “no difference.” If a confidence interval crosses zero, the data are compatible with no difference—and with effects in both directions. That does not prove the effect is zero. It means the study cannot rule out zero at the conventional level.

Nutritious News coverage of a steps-and-weight-maintenance meta-analysis reported:

“Program groups ended maintenance 3.28% below their starting weight (95% CI −4.95 to −1.61), versus about 1% for controls — a 2.29-percentage-point between-group gap (95% CI 0.06 to 4.52).”

Here, 3.28% below starting weight is the weight-change figure for the program groups. The 95% CI from −4.95 to −1.61 is the range for that figure compatible with the data and model: a loss as large as 4.95% or as small as 1.61%. Both ends are negative, so the interval does not cross zero. Controls ended about 1% below starting weight. The between-group gap is 2.29 percentage points—roughly 3.28% minus 1%. It is a difference between two percentage changes, not a 2.29% of body weight.

The gap’s 95% CI is 0.06 to 4.52 percentage points. Because the whole interval is above zero, the data are not very compatible with no between-group difference. But the lower end, 0.06, is close to zero. The study supports a gap, but the smallest plausible gap is very small. Whether 2.29 percentage points matters in practice depends on starting weight, duration, and what maintenance means—details not in this sentence.

P-values: what p=0.03 and p=0.09 do and do not mean

A p-value is the probability of getting a result at least as extreme as the one observed if the null hypothesis were true. It is not the probability that the null is true. It is not the probability the result happened by chance. It is not a measure of effect size. By convention, p<0.05 is often called “statistically significant,” but that threshold is a rule of thumb.

Nutritious News coverage of an ultra-processed-food controlled feeding trial reported:

“It was significant (p<0.001, d=1.22): the 18-21-year-olds ate more after the ultra-processed diet (p=0.03, d=0.79); the 22-25-year-olds did not (p=0.09).”

The p<0.001 there belongs to the age-by-diet interaction flagged in the sentence — an exploratory subgroup finding about the difference in how the two age groups responded, not an overall effect: across the full sample, buffet intake did not differ between diets (all p>0.05). A result this extreme under the null would occur less than 1 time in 1,000. That is strong evidence against the null for that interaction by conventional standards, but it does not tell us how large or important the effect is. For the 18-21-year-olds, p=0.03 is below 0.05, so that difference is conventionally called significant. If there were no effect, a result this extreme would occur about 3 times in 100. It does not mean there is a 97% chance the effect is real. For the 22-25-year-olds, p=0.09 is above 0.05, so the result “did not” reach conventional significance. That does not prove there was no effect. It means the data are less incompatible with the null at that threshold. A p-value near 0.09 can reflect a smaller effect, more variability, or a smaller sample.

The word “significant” means statistical significance for the tested comparison. It does not mean the finding is large, important, or clinically meaningful. The age split also matters: those are separate age-group results, not a universal rule that all younger adults respond one way and all older adults another.

Effect sizes: what d=0.79 and d=1.22 mean

Cohen’s d is a standardized effect size. It expresses a difference between means in units of standard deviation. If d=0.79, the groups differ by 0.79 standard deviations on that outcome. If d=1.22, they differ by 1.22 standard deviations. Standard deviation measures how spread out the data are, so d helps compare effects across studies that use different scales. A larger d means the groups are more separated relative to variability.

By common rules of thumb, d=0.2 is small, 0.5 is medium, and 0.8 is large. So d=0.79 is near the large threshold, and d=1.22 is large by that convention. But rules of thumb are not practical meaning. A d of 1.22 does not tell us how many calories, grams, or meals differed. It does not tell us whether the change would matter for health over time. In the ultra-processed-food sentence, d=1.22 describes the age-by-diet interaction’s effect, while d=0.79 describes the younger age group’s own contrast. The smaller d in the 18-21 group means less separation than in that interaction, but it still came with p=0.03. The p-value and the effect size answer different questions: p asks about compatibility with a null; d asks about standardized size.

Related coverage: NN’s live explainer What Happens When People Stop Weight-Loss Medications covers the regain baseline behind the first worked example.

Sources

Nutritious News coverage of the steps-and-weight-maintenance meta-analysis. Underlying study: https://doi.org/10.3390/ijerph23040522.
Nutritious News coverage of the ultra-processed-food controlled feeding trial. Underlying study: https://doi.org/10.1002/oby.70086.