Evaluateurs évalués: évaluation diagnostique des compétences en évaluation des correcteurs d'une épreuve d'expression écrite à forts enjeux

Raters are a key element of language proficiency assessment systems. They ought to be monitored cosely to keep the quality of assessments at a high level. This article compares two methods in order to determine the raters' severity profiles using real rating data, for a writing test of French as a foreign language. The first method makes use of the Classical Test Theory while the second is based on the Item Response Theory using the multifaceted Rasch Model (MFRM). The concurring results of both methods show that in order to improve the reliability of a test, it is important to take into account a rater's severity at each cut-off score, even if this consideration only explains a relatively small portion of the error variance. This information also allows for drawing up the raters' individual evaluation profiles, which could prove useful for more focused training activities.