requiredPairedSampleSize({ effect: dz }) takes a standardised effect. In practice dz comes from a preregistered raw-scale effect divided by an sd(delta) estimated from a small pilot (discovery's E1 has one run per arm and no variance at all). Plugging the pilot's point-estimate sd into the row does not preserve the power target: on 5 degrees of freedom an underestimated sd turns a true dz 0.5 into an apparent 1.0 and selects 8 confirmatory pairs instead of 32.
The obvious fix, the chi-square upper confidence bound on sd (times 1.461 at 6 pairs, 80% one-sided), assumes normal pilot deltas, which differences of verified-contributions-per-token ratios need not be. Discovery's E1 gate (tangle-network/discovery#218) implemented that chi-square bound locally and is retiring it, because the capability belongs here.
Requested export, shape open to you:
pairedSampleSizeFromPilot(opts: {
rawEffect: number; // preregistered minimally useful effect, outcome units
pilotDeltas: number[]; // within-pair deltas from an excluded pilot
alpha?: number; power?: number; confidence?: number; // confidence for the variance bound
varianceMethod?: 'chi-square' | 'bootstrap' | 'assurance';
}): { sdPoint: number; sdBound: number; dz: number; pairsNeeded: number; method: string }A reviewer's requirement worth honouring: the bound must be valid for the pilot's actual distribution, or the caller must be made to register a normal model explicitly; a bootstrap or assurance option covers the non-normal case.
requiredPairedSampleSize({ effect: dz })takes a standardised effect. In practice dz comes from a preregistered raw-scale effect divided by an sd(delta) estimated from a small pilot (discovery's E1 has one run per arm and no variance at all). Plugging the pilot's point-estimate sd into the row does not preserve the power target: on 5 degrees of freedom an underestimated sd turns a true dz 0.5 into an apparent 1.0 and selects 8 confirmatory pairs instead of 32.The obvious fix, the chi-square upper confidence bound on sd (times 1.461 at 6 pairs, 80% one-sided), assumes normal pilot deltas, which differences of verified-contributions-per-token ratios need not be. Discovery's E1 gate (tangle-network/discovery#218) implemented that chi-square bound locally and is retiring it, because the capability belongs here.
Requested export, shape open to you:
A reviewer's requirement worth honouring: the bound must be valid for the pilot's actual distribution, or the caller must be made to register a normal model explicitly; a bootstrap or assurance option covers the non-normal case.