Statistical significance in research methodology

Statistical Significance

The author of this response is asked to answer to two major questions. The first question asks the author to explain the relationship between statistical significance and effect size. The second question asks the author to explain the importance of effect size in the statistical significance of the studies that were reviewed during this course.

Questions Answered

The statistical significance has two basic meanings but the one that is probably more applicable to this course and the documents reviewed is the Type I error rate for a study. The Type I rate is the false positive rate for a study. Stated another way, statistical significance is a measurement of whether all of the actions and behaviors being studied are a result of simple chance or if they are were induced improperly and/or accidentally by the program. Basically, if the p-value of a study's sample and statistical significance is less than the alpha value, then the results of the test are statistically significance. If the opposite is true, than the results are NOT statistically significant. A value of 0.05 or less is the widely accepted standard for many studies and this includes social sciences like criminology (MEERA, 2013).

Statistical significance is not always a big deal. It can be but this is not always the case. This is where effect size comes in. The effect size is garnered by subtracting the mean of the control group from the mean of the treatment group and dividing by the standard deviation of the control group. If the answer to that equation is less than 0.1, then the effect size is trivial. If it is between 0.1 and 0.3, the effect is small. If it's between 0.3 and 0.5, it is moderate. If the answer is more than 0.5, then the effect is large.

The relationship between the two variables, those being statistical significance and effect size is clear. A study's result may or may not be statistically significant. If it is not, then one need not really go any further. However, if it is statistical significant, the effect size should be calculated to determine whether the statistical significance is of any import. If there is statistical significance but the effect size is less than 0.1, than the statistical significance is not all that extraordinary.

This distinction is important in the studies reviewed during this course because just because one or more studies had statistical significance does nto mean that the significance is all that big a deal. The source for this report covers such an instance where a pre-test and a post-test of an examination are off by a full point (on a scale of 100) and this is deemed to be statistically significant given the size of the sample. However, since the difference is only one point, that difference is not nearly as significant as it may seem.