The formula to calculate the inter-rater reliability is:
\[ IRR = \frac{TA}{TR \times R} \times 100 \]
Where:
For two raters, the formula simplifies to:
\[ IRR = \frac{TA}{TR} \times 100 \]
Inter-rater reliability is defined as the ratio of the total number of agreements between raters and the total number of ratings. The term is used to describe how often a set of judges or raters agree on a certain score.
Let's assume the following values:
Using the formula:
\[ IRR = \frac{80}{100 \times 3} \times 100 = 26.67\% \]
The inter-rater reliability is 26.67%.
Let's assume the following values:
Using the simplified formula for two raters:
\[ IRR = \frac{90}{100} \times 100 = 90\% \]
The inter-rater reliability is 90%.
