"The effect of the level of α is the least intuitive factor affecting power. Recall that α is the pre-determined threshold for rejecting the null hypothesis (H0) that there is no difference between groups, customarily set at 0.05. If the p-value of a study is below 0.05, then H0 is rejected; if the p-value of a study is above 0.05, the conclusion is that there is insufficient evidence to reject H0. If power is analogous to sensitivity, then α is analogous to 1-specificity or the false positive rate. By setting α at 0.05, we accept a false positive rate of 5%. If we decrease α to 0.01, we decrease our false positive rate to 1%, but in doing so, we increase the β or false negative rate, decreasing power. A decrease in α means that we are less likely to reject H0. This protects us from false positive results but will increase the number of times we fail to reject H0 when it is incorrect, decreasing the number of true positives. Increasing α makes it easier to reject H0, increasing the number of false positives and increasing the number of true positives and thus power."
"Consider the 2 extremes. If α=0, H0 will never be rejected and all study results will be negative: there will be no false positives (good) but also no true positives (bad), resulting in a power of 0. If α=1, H0 will always be rejected and all study results will be positive: the only possible outcomes will be false positives (bad) and true positives (good), resulting in a power of 1. Increasing the number of negative results increases both true negatives and false negatives while increasing the number of positive results increases both true positives and false positives. More false negatives decrease power, and more true positives increase power."
"Consider the 2 extremes. If α=0, H0 will never be rejected and all study results will be negative: there will be no false positives (good) but also no true positives (bad), resulting in a power of 0. If α=1, H0 will always be rejected and all study results will be positive: the only possible outcomes will be false positives (bad) and true positives (good), resulting in a power of 1. Increasing the number of negative results increases both true negatives and false negatives while increasing the number of positive results increases both true positives and false positives. More false negatives decrease power, and more true positives increase power."
"A more intuitive analogy compares the level of α to the amount of evidence required to convict a defendant at trial. The lower the α, the more evidence is needed to convict, resulting in fewer false positive convictions but also fewer true positive convictions, decreasing power. The higher the α, the less evidence is needed to convict, resulting in more false positive convictions and more true positive convictions, increasing power."
"A more intuitive analogy compares the level of α to the amount of evidence required to convict a defendant at trial. The lower the α, the more evidence is needed to convict, resulting in fewer false positive convictions but also fewer true positive convictions, decreasing power. The higher the α, the less evidence is needed to convict, resulting in more false positive convictions and more true positive convictions, increasing power."
Expected headings
"Effect size"
"Effect variability"
"Level of α"
"Sample size"
"Caveat"
"The effect of the level of α is the least intuitive factor affecting power. Recall that α is the pre-determined threshold for rejecting the null hypothesis (H0) that there is no difference between groups, customarily set at 0.05. If the p-value of a study is below 0.05, then H0 is rejected; if the p-value of a study is above 0.05, the conclusion is that there is insufficient evidence to reject H0. If power is analogous to sensitivity, then α is analogous to 1-specificity or the false positive rate. By setting α at 0.05, we accept a false positive rate of 5%. If we decrease α to 0.01, we decrease our false positive rate to 1%, but in doing so, we increase the β or false negative rate, decreasing power. A decrease in α means that we are less likely to reject H0. This protects us from false positive results but will increase the number of times we fail to reject H0 when it is incorrect, decreasing the number of true positives. Increasing α makes it easier to reject H0, increasing the number of false positives and increasing the number of true positives and thus power."
"Consider the 2 extremes. If α=0, H0 will never be rejected and all study results will be negative: there will be no false positives (good) but also no true positives (bad), resulting in a power of 0. If α=1, H0 will always be rejected and all study results will be positive: the only possible outcomes will be false positives (bad) and true positives (good), resulting in a power of 1. Increasing the number of negative results increases both true negatives and false negatives while increasing the number of positive results increases both true positives and false positives. More false negatives decrease power, and more true positives increase power."