Under such conditions, we suggest that ICC values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability.

What are raters for ICC?

The Intraclass Correlation Coefficient (ICC) is a measure of the reliability of measurements or ratings. For the purpose of assessing inter-rater reliability and the ICC, two or preferably more raters rate a number of study subjects.

What tools increase inter-rater reliability?

Survey Research Methods Interrater reliability is enhanced by training data collectors, providing them with a guide for recording their observations, monitoring the quality of the data collection over time to see that people are not burning out, and offering a chance to discuss difficult issues or problems.

Is an intraclass correlation a good measure of inter-rater reliability?

An intraclass correlation (ICC) can be a useful estimate of inter-rater reliability on quantitative data because it is highly flexible. A Pearson correlation can be a valid estimator of interrater reliability, but only when you have meaningful pairings between two and only two raters. What if you have more? What if your raters differ by ratee?

How is inter-rater reliability of ICC computed?

Inter-rater reliability (one-way random effects model of ICC) was computed using SPSS (v.17.0). One-way random effects model was used instead of Two-way random effects model because the judges are conceived as being a random selection of possible judges, who rate all targets of interest.

How reliable is the ICC?

One of the driving factors of the uptake of the ICC in many fields is its ease of interpretation [ 9, 10 ]. The ICC is a value between 0 and 1, where values below 0.5 indicate poor reliability, between 0.5 and 0.75 moderate reliability, between 0.75 and 0.9 good reliability, and any value above 0.9 indicates excellent reliability [ 14 ].

What are the ICC values for the intraclass correlation?

The ICC values for these classes are respectively called ICC (1, 1), ICC (2, 1) and ICC (3, 1). Each of these measures the reliability of a single rater. We can also consider the reliability of the mean rating. The intraclass correlation for these are designated ICC (1, k), ICC (2, k) and ICC (3, k).