Extreme values warnings

In power analysis, sooner or later the computation fails. The main reason of this failure is the size of the effect size, too big or too small, or the size of the sample. Different software handles this issue in different ways. G*Power, for instance, does not allow inserting effect sizes that would produce a computation failure. Focusing on required N, for instance, we see that asking the required N for a correlation \(\rho=.93\), with \(power=.99\) halts the calculation.

R pwr package, for another example, simply fails.

pwr::pwr.r.test(r=.99,power=.80,sig.level=.05)

## Error in uniroot(function(n) eval(p.body) - power, c(4 + 1e-10, 1e+09)): f() values at end points not of opposite sign

In both examples, the combination of effect size, power, and type I error results in a required sample size so small that it falls outside the feasible range. The same issues happens when the effect size is too small, and the required N would be simply too large.

In R pwr package we can observe this issue.

pwr::pwr.r.test(r=.00001,power=.80,sig.level=.05)

## Error in uniroot(function(n) eval(p.body) - power, c(4 + 1e-10, 1e+09)): f() values at end points not of opposite sign

Admittedly, these are extreme cases, but they still pose a problem when users experiment with such values, whether to learn something new or investigate emerging issues.

computes the required parameters regardless, but when the values fall outside the feasible range, a warning is issued, ensuring that the user is not left confused by an technical error. Here is an example for Correlation power analysis.

We asked the required N for \(power=.80\) and \(\pho=.99\). The required N is produced ($N=4) with a warning.

This basically says that given the power and the type I error, the effect size \(\rho=.99\) is too large, because it is larger than \(\rho=.986\), an effect size that guarantees the required minimum power (\(.80\)) with the minimum feasible sample size, in this case \(N=4\). Thus, any effect size larger than \(\rho=.986\) needs only 4 cases to achieve the required power.

Furthermore, another warning is issued:

This means that if you request sensitivity analysis plots (e.g., contour, by N, or by ES plots), you may encounter some unusual results. In this example, the by N

A similar warning appears when the effect size is too small.

The reasoning is as follows: PAMLj calculates the smallest effect size (given power and type I error) that would require a sample size of 10 million cases. If the input effect size is smaller than this threshold, the computation is likely to fail, so the smallest feasible effect size is used instead. This informs us that any effect size smaller than the reported one would require more than 10 million cases to achieve the desired power. The warning uses the term “around” because, in this range of values, the computation is not highly accurate, meaning that even slight changes in decimals could significantly affect the effect size and the required N.

Finally, the module conducts a series of checks to guarantees that the values in input are correct (correlation or eta-squared between 0 and 1, proportions less than 1, etc.).