Inter-Rater Reliability Calculator

Calculate Inter-Rater Reliability





Formula

The formula to calculate the inter-rater reliability is:

\[ IRR = \frac{TA}{TR \times R} \times 100 \]

Where:

For two raters, the formula simplifies to:

\[ IRR = \frac{TA}{TR} \times 100 \]

Inter-Rater Reliability Definition

Inter-rater reliability is defined as the ratio of the total number of agreements between raters and the total number of ratings. The term is used to describe how often a set of judges or raters agree on a certain score.

Example Calculation 1

Let's assume the following values:

Using the formula:

\[ IRR = \frac{80}{100 \times 3} \times 100 = 26.67\% \]

The inter-rater reliability is 26.67%.

Example Calculation 2

Let's assume the following values:

Using the simplified formula for two raters:

\[ IRR = \frac{90}{100} \times 100 = 90\% \]

The inter-rater reliability is 90%.