Rank Correlation Test for Agreement in Multiple Judgements

This tests the significance of the correlation between $n$ series of rank numbers, assigned by $n$ judges to $K$ subjects. The $n$ judges give rank numbers to the $K$ subjects and we compute:

$$S = \frac{nK(K^2-1)}{12}$$

and $S_{D}$, the sum of squares of the differences between subjects' mean ranks and the overall mean rank. Let:

$$D_1 = \frac{S_D}{n}, D_2 = S - D_1, S^2_1 = \frac{D_1}{K-1}, S^2_2 = \frac{D_2}{K(n-1)}$$

The test statistic is:

$$F = \frac{S^2_1}{S^2_2}$$

which follows the $F$ distribution with $K-1,K(n-1)$ degrees of freedom.