Interpretation of Alpha and p-Value

Quick Links

SPC for Excel Software

Visit our home page

SPC Training

SPC Consulting

Ordering Information

Thanks so much for reading our publication. We hope you find it informative and useful. Happy charting and may the data always support your position.


Dr. Bill McNeese
BPI Consulting, LLC

View Bill McNeese

Connect with Us

Comments (14)

  • DaleWApril 7, 2017 Reply

    When you wrote "This means that there is a 57.34% probability of obtaining a mean of 101.3 or larger if the null hypothesis is true" you were overlooking the low tail of the distribution.  The probability of a mean of 101.35 or larger is only 28.67% if the null hypothesis is true, so the probability of that large an absolute deviation from the null hypothesis can be 57.34%.

    • billApril 7, 2017 Reply

      You are correct as usual.     I should have applied it to the upper portion of the distibution only once you know what the sample aveage is.  I revised the wording to reflect your comments.

  • THANK YOU!April 13, 2020 Reply

    Wow, thanks for your super clear explanation!

  • KyleneApril 19, 2020 Reply

    "If the p-value is greater than alpha, you accept the null hypothesis."Oh no, accepting the null hypothesis is a big 'ol no-no. Just because your sample isn't extreme enough for you to reject the null hypothesis doesn't mean that there isn't another sample that exists that /is/ extreme enough for you to reject the null.To avoid making a Type II error, we would usually say that we "fail to reject the H0" or "do not reject the H0.""The p-value measures the probability of getting a more extreme value than the one you got from the experiment."Close! You said it better earlier in the page, but it is still best to say "The p-value measures the probability of getting a more extreme value than the one you got from the experiment (assuming the null is true)." I'm sure you were trying to keep things short, but stating the assumption is still important.

  • osman alawiyeJune 30, 2020 Reply

    Is the a better way to conclude when the P value is less than or equal to the alpha in a meaningful way to the average educator like a teacher?

    • billJune 30, 2020 Reply

      Not sure what you mean, but I think Figure 1 makes it clear if there is a significant difference or not – good visual picture.

  • Thank youJanuary 2, 2021 Reply

    This was a well written and thorough explanation of both concepts. Thank you so much!!

  • Andrew AtamanyukApril 11, 2021 Reply

    “What do you do if the two values are very close? For example, maybe the p-value is 0.06 and alpha is 0.05. It is your call to make in those cases. You can always choose to collect more data.”This is an absolute abuse of the experiment, called p-hacking, because after you continue collecting data and get *new* p-value, you have to use all knowledge about both p-values you got when you decide to reject / not reject H0. To make life easier, run only one experiment with enough sample size determined by pre-chosen statistical power and make only one conclusion. You cannot fully explain the nature of statistical tests without mentioning probabilty of type 2 error “beta”, which tells about statistical power.

    • billApril 12, 2021 Reply

      Collecting more of the same data is not p-hacking i don't believe.  My understanding of p-hacking is that you perform many statistical tests on the data and only report those that come back with significant results.  This is one statistical dataset  that you are adding more data to.
      I agree that you should set the alpha and beta values before you run your experiemnt.  But in real life, you can't sometimes take all those samples.  

  • Nate H.November 29, 2021 Reply

    I believe you used the word "accept the null hypothesis" when instead you should have said "retain the null hypothesis". It's a small thing, but you can't "accept" a null hypothesis.I knew what you were trying to say though (I was just chewed out for making the same mistake in my sociology course).

    • billNovember 30, 2021 Reply

      I have not heard it quite like that using retained.  I think he makes a big deal out of nothing.   I usually frame it like:
      The null hypothesis is accepted.  There is no evidence that the difference in means is not equal to 0.
      The null hypothesis is rejected.  There is evidence that the difference in means is not equal to 0.

      • YZJune 6, 2022 Reply

        well not really, in every statistic course I took, especially the entry-level ones, we were taught not to use "accept the null", you either "reject" or "fail to reject", but you never "accept"

  • nate H.November 29, 2021 Reply

    thank you for your thorough answer

  • Iryna W.July 24, 2022 Reply

    Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *