Table 7.
Rater 2 intra-rater agreement: Cohen’s Kappa with 95% confidence intervals and overall criteria percentage agreement (n = 20 except *n = 19, **n = 18).
Test
Criteria
Left-hand side
Kappa (95% CI)
Right-hand side
Kappa (95% CI)
Criteria agreement (%)
1
A
.83
(.50-1.00)
1.00
(1.00-1.00)
97.5
B
.89
(.69-1.00)
.76
(.45-1.00)
92.5
C
.90
(.71-1.00)
.90
(.71-1.00)
95.0
D
.79
(.52-1.00)
.89
(.69-1.00)
92.5
E
.90
(.70-1.00)
.60
(.26-.94)
87.5
2
A
.86
(.59-1.00)
.83
(.50-1.00)
95.0
B
.69
(.38-1.00)
.31
(-.08-.71)
75.0
C
.70
(.39-1.00)
.68
(.35-1.00)
85.0
D
N/A
N/A
N/A
N/A
100.0
E
N/A
N/A
.00
N/A
97.5
3
A
1.00 (1.00-1.00)
100.0
B
N/A N/A
100.0
C
N/A
N/A
N/A
N/A
N/A
4
A
1.00 (1.00-1.00)
100.0
B
N/A
N/A
N/A
N/A
N/A
C
1.00
(1.00-1.00)
1.00
(1.00-1.00)
1.00
D
.60
(.28-.92)
.60
(.28-.92)
.60
5
A
.64 *
(.00-1.00)
.77 *
(.35-1.00)
94.7
B
.83 *
(.50-1.00)
.44 **
(-.21-1.00)
91.9
C
N/A
N/A
N/A
N/A
100.0
D
-.08 *
(-.23-.08)
.83*
(.50-1.00)
89.5
7
A
.69
(.29-1.00)
.53
(.14-0.92)
85.0
B
.00
N/A
.00
N/A
90.0
C
.79
(.53-1.00)
.67
(.34-1.00)
87.5
D
0.77 (0.35-1.00)
95.0
E
1.00
(1.00-1.00)
.69
(.31-1.00)
95.0
8
A
-0.053 (-0.18-0.07)
90.0
B
1.00
(1.00-1.00)
1.00
(1.00-1.00)
1.00
C
N/A
N/A
N/A
N/A
N/A
D
.00
N/A
.00
N/A
.00
9
A
.89
(.67-1.00)
.88
(.64-1.00)
95.0
B
.88
(.64-1.00)
.88
(.64-1.00)
95.0
C
.77
(.35-1.00)
.77
(.35-1.00)
95.0
D
.83
(.50-1.00)
.83
(.50-1.00)
95.0
E
1.00
(1.00-1.00)
.64
(.01-1.00)
97.5
10
A
.89 *
(.69-1.00)
.90
(.71-1.00)
94.9
B
.64
(.01-1.00)
N/A
N/A
97.5
C
.64
(.01-1.00)
.64
(.01-1.00)
95.0
D
.64
(.01-1.00)
.00
N/A
95.0
E
1.00
(1.00-1.00)
N/A
N/A
100.0