> In our predominant use with a 0-difference null hypothesis, the p-value is the probability of seeing a result as extreme as the experiment’s results under the hypothetical circumstances where nothing actually changed, purely due to random chance.
No, it's the chance of seeing one *at least* as extreme, not *as* extreme. Seeing exactly any given result is extremely unlikely, approaching 0 as the sample size increases.
> I didn’t write “false positive rate” because technically I’m describing the false discovery rate.
No, you're describing the false positive rate. The false positive rate is FP/(FP+TN). (The fraction of results that should have been negative that accidentally came up positive.) The false discovery rate is FP/(FP+TP). (The fraction of results that came up positive that did so incorrectly.)
Don't worry, everyone else finds this confusing too. :)
Thanks for the detailed read and writing up a clear comment.
On the first point, we have the same understanding of the p-value and I read both phrases as meaning the same thing. It's normal to use "as" in the way that I did here without meaning strict equality. For example, one could write "Is the boy as tall as his father?" or "Is her new car as fast as her last one?", in both cases meaning "at least as". Using "at least as" would reduce ambiguity but seems wordy and unnecessary when the meaning is clear. I would expect someone who wanted to ask about strict equality to phrase those more like "Is the boy the same height as his father?" and "Is her new car the same speed as her last one?"
On the second point, thanks for noticing. I've corrected the footnote and credited you. Please continue reading more posts and correcting other mistakes.
> In our predominant use with a 0-difference null hypothesis, the p-value is the probability of seeing a result as extreme as the experiment’s results under the hypothetical circumstances where nothing actually changed, purely due to random chance.
No, it's the chance of seeing one *at least* as extreme, not *as* extreme. Seeing exactly any given result is extremely unlikely, approaching 0 as the sample size increases.
> I didn’t write “false positive rate” because technically I’m describing the false discovery rate.
No, you're describing the false positive rate. The false positive rate is FP/(FP+TN). (The fraction of results that should have been negative that accidentally came up positive.) The false discovery rate is FP/(FP+TP). (The fraction of results that came up positive that did so incorrectly.)
Don't worry, everyone else finds this confusing too. :)
https://x.com/IsaacKing314/status/1759310122977755557
Thanks for the detailed read and writing up a clear comment.
On the first point, we have the same understanding of the p-value and I read both phrases as meaning the same thing. It's normal to use "as" in the way that I did here without meaning strict equality. For example, one could write "Is the boy as tall as his father?" or "Is her new car as fast as her last one?", in both cases meaning "at least as". Using "at least as" would reduce ambiguity but seems wordy and unnecessary when the meaning is clear. I would expect someone who wanted to ask about strict equality to phrase those more like "Is the boy the same height as his father?" and "Is her new car the same speed as her last one?"
On the second point, thanks for noticing. I've corrected the footnote and credited you. Please continue reading more posts and correcting other mistakes.