E2P Simulator

Understanding predictive value of effect sizes

This tool allows you to perform predictive utility analysis: translating effect sizes into expected predictive performance while accounting for measurement reliability and base rates

Outcomes
Effects
True Obs.
Cohen's d
Odds Ratio
Log Odds Ratio
Point-biserial r
η²
Group 1 reliability, ICC1
Group 2 reliability, ICC2
Grouping reliability, κ
Base rate (%)

AUC: 0.00

Accuracy: 0.00

Sensitivity: 0.00

Specificity: 0.00

BA: 0.00

NPV: 0.00

PPV: 0.00

F1-score: 0.00

MCC: 0.00

Cohen's d attenuation by reliability

\[d_{obs} = d_{true} \sqrt{\frac{2ICC_1ICC_2}{ICC_1+ICC_2}\sin\left(\frac{\pi}{2}\kappa\right)}\]

d to Log Odds Ratio

\[\log(OR) = \frac{d\pi}{\sqrt{3}}\]

d to AUC

\[AUC = \Phi\left(\frac{d}{\sqrt{2}}\right)\]

d to Point-biserial r

\[r_{pb} = \frac{d}{\sqrt{d^2 + \frac{1}{p(1-p)}}}\]

d to Eta squared

\[\eta^2 = \frac{d^2}{d^2 + \frac{1}{p(1-p)}}\]

Mahalanobis D Calculator

Instead of using a single large effect, it is often necessary to combine multiple smaller effects to achieve the desired level of predictive performance. This calculator shows how many predictors are needed, given their individual effect sizes and collinearity, to achieve a desired level of group separation. The separation is quantified using Mahalanobis D, which generalizes Cohen's d to multivariate settings.

A simplified Mahalanobis D formula when all predictors have the same effect size and collinearity:

\[D = d \sqrt{\dfrac{p}{1 + (p-1)r_{ij}}}\]
Needed Mahalanobis D
Effect size of each predictor (d)
Collinearity among predictors (r_ij)
Number of predictors (p)