This program examines the extent that a given set of results is either improbably significant or improbably non-significant.
For a given power level, studies should only be significant a fraction of the time (e.g. 80% if the type II error rate is.20)
Enter in the values from a series of studies below, and then press "Calculate" to determine how incredible the results are.
First, enter the statistical test result from a study into the appropriate box.
Then click "add"
The observed power, and sample size from the study should appear in the "Observed Power" and "Sample Sizes" text areas on the right side of the page.
Repeat this process until all relevant studies have been entered.
Note: You can also directly enter the power and samples in to the text areas.
The incredibility of the results are shown in this area.
The first row displays the average power of the studies. The column next to it shows the average power, weighted by sample size.
The next row shows the probability of obtaining at least as many significant results as you observed. Values less than .10 indicate an implausibly high number of significant results.
The next row shows the probability of obtaining as many non-significant results as observed. Values less than .10 indicate an implsauibly high number of non-significant results.
The last row indicates how many studies in the current sample would be expected to be non-significant.
Results of Analyses
Probabilities < .10 suggest highly incredible data