Confidence intervals of the Mann-Whitney parameter that are compatible with the Wilcoxon-Mann-Whitney test

Abstract

For the two-sample problem, the Wilcoxon-Mann-Whitney (WMW) test is used frequently: it is simple to explain (a permutation test on the difference in mean ranks), it handles continuous or ordinal responses, it can be implemented for large or small samples, it is robust to outliers, it requires few assumptions, and it is efficient in many cases. Unfortunately, the WMW test is rarely presented with an effect estimate and confidence interval. A natural effect parameter associated with this test is the Mann-Whitney parameter, φ = Pr[ X<Y ] + 0.5 Pr[X = Y ]. Ideally, we desire confidence intervals on φ that are compatible with the WMW test, meaning the test rejects at level α if and only if the 100(1 - α)% confidence interval on the Mann-Whitney parameter excludes 1/2. Existing confidence interval procedures on φ are not compatible with the usual asymptotic implementation of the WMW test that uses a continuity correction nor are they compatible with exact WMW tests. We develop compatible confidence interval procedures for the asymptotic WMW tests and confidence interval procedures for some exact WMW tests that appear to be compatible. We discuss assumptions and interpretation of the resulting tests and confidence intervals. We provide the wmwTest function of the asht R package to calculate all of the developed confidence intervals.

Keywords: Mann-Whitney U test; Wilcoxon rank sum test; area under the curve; probabilistic index; receiver operating characteristic curve.

Published 2018. This article is a U.S. Government work and is in the public domain in the USA.

Figures

Figure 1:

Comparison of VLA.PH(ϕ) from Equation 14 (thick gray line), with VPO(ϕ) from the proportional odds model (thin black line) (see Supplement Section S2), when m = 10 and n = 20. The lines are equal at ϕ = 1/2 and nearly indistinguishable otherwise.

Figure 2:

Two-sided p-values for x = [1, 2, 3, 4, 5] and y = [6, 7, 8, 9, 10, 11]. Black dots are the absolute value method and the gray line is the central method. The 95% confidence region for the absolute value method is (0.6500, 0.6505) ∪ (0.6667, 1), while the one for the central method is (0.6897, 1).

Figure 3:

Simulated error for 95% “exact” confidence interval procedures (Section 7.1) when m = 5 and n = 6. Dotted lines are at 0.05 and 0.025. The lower and upper simulated errors for the central method appear bounded at 2.5%, while those errors appear bounded at 5% for the absolute value method.

Figure 4:

Simulated error for 95% asymptotic confidence interval procedures for the continuous proportional odds model. Dotted lines are at 0.05 and 0.025.

Figure 5:

Simulated error for 95% asymptotic confidence interval procedures for the grouped proportional odds model with k = 6 categories. Dotted lines are at 0.05 and 0.025. The first column uses asymptotic confidence intervals for ϕ and calculates error with respect to ϕ. The second column uses asymptotic confidence intervals for ϕ but calculates error with respect to ϕ*. The third column uses asymptotic confidence intervals for ϕ transformed to the ϕ* (i.e. latent Mann-Whitney) scale, then calculates error with respect to ϕ*.