Rater-mediated Judgment with or without Rating Scales
Scoring Second Language Spoken and Written Performance - Issues, Options and Directions - Ute Knoch
Ute Knoch [+ ]
University of Melbourne
Ute Knoch is the Director of the Language Testing Research Centre at the University of Melbourne. Her research interests are in the area of writing assessment, rating processes, assessing languages for academic and professional purposes, and placement testing. She was the Co-president of the Association for Language Testing and Assessment of Australian and New Zealand (ALTAANZ) between 2015-2016 and currently serves on the Executive Board of the International Language Testing Association (ILTA). In 2014, Dr Knoch was awarded the TOEFL Outstanding Young Scholar Award by the Educational Testing Service (Princeton, US), recognizing her contribution to language assessment.
Judith Fairbairn [+ ]
British Council
Judith Fairbairn leads on assessment development projects for the British Council’s Global Assessments team, which includes project management, assessment design, item writing, rating scale development, test quality assurance and developing bespoke assessment solutions for clients. Areas of expertise include research into rating speaking and writing tests, online rater training, rating scale development, and equality, diversity and inclusion in testing. Judith has an MA in Language Testing (Lancaster University) and is studying a PhD at University College London, Institute of Education focusing on rater decision making processes when marking speaking tests.
Yan Jin [+ ]
Shanghai Jiao Tong University, China
Dr. Yan Jin is a professor of linguistics and applied linguistics at the School of Foreign Languages, Shanghai Jiao Tong University, China. Her research interests include the development and validation of large-scale, high-stakes language assessments. She is currently chair of the National College English Testing Committee of China and co-president of the Asian Association for Language Assessment. She is co-editor-in-chief of the Springer open-access journal Language Testing in Asia and is also on the editorial boards of Language Testing, Language Assessment Quarterly, International Journal of Computer-Assisted Language Learning, and a number of academic journals published in China.
Description
In Chapter 5, we look at scoring, in particular by using rating scales. We introduce various rating scale types and also examine rating scale development methods described in the literature. The chapter concludes by looking at one method increasingly used when scoring without specific scale descriptors – comparative judgement.