Validity
Theoretical Development and Integrated Arguments
Micheline Chalhoub-Deville [+–]
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
Barry O’Sullivan [+–]
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality.
The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
Series: British Council Monographs on Modern Language Testing
Table of Contents
Prelims
List of Figures [+–] viii
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality. The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
List of Tables [+–] ix-x
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality. The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
Chapter 1
Introduction [+–] 1-12
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter focuses on the background of validity as an area of scholarly activities. It also describes commonly referenced terms and ideas commonly used with regard to validity.
Chapter 2
Historical Overview of Validity [+–] 13-58
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter provides a contrastive historical account of the validity and testing scholarship on each side of the Atlantic divide. The account highlights the divergent UK and USA perspectives and shows how the two traditions have continued to evolve.
Chapter 3
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter discusses key models of validity that have been influential in shaping thinking and language testing operations. The analysis of these models takes into account their views on consequences and their role in validity research.
Chapter 4
Validity and Consequences [+–] 128-145
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter urges researchers to consider broad—more than technical, e.g., differential test and item analysis—aspects of consequences in validity models. Given the limited state-of-knowledge in the field with regard to consequences, the undertaking, the authors contend, is massive. They present areas and issues to consider as researchers build their research programmes.
Chapter 5
An Integrated Argument-based Approach to Validation [+–] 146-157
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter affirms that the traditions represented in the USA and the UK are both critical to validity conceptualisation. The authors argue that these test development and measurement arguments, however, are not sufficient. Validity documentation should also consider arguments related to consequences and stakeholder engagement.
Chapter 6
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
This chapter reviews the validity models that have guided the field and have been discussed in the present book. The authors also build on expositions, e.g., validation in the context of higher-education admission testing and in reform-based educational accountability testing contexts, delineated in previous chapters to push for localised validity research that embraces an integration of the test development, the measurement, the theory of action, and the communication engagement arguments
End Matter
References [+–] 168-188
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality. The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
Appendix [+–] 189-200
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality. The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
Index [+–] 201-206
University of North Carolina at Greensboro
Micheline Chalhoub-Deville (Ph.D., The Ohio State University, USA) is Professor of Educational Research Methodology at the University of North Carolina at Greensboro (UNCG), USA http://erm.uncg.edu/. At UNCG, she also serves as Director of a research group, Testing and Evaluation in Educational Accountability Models (TEEAM) https://www.teeamaccountability.com/. Professor Chalhoub-Deville is the President of the International Language Testing Association (ILTA) http://www.iltaonline.com/.
Professor Chalhoub-Deville is the recipient of the Outstanding Young Scholar Award by the Educational Testing Service—TOEFL Program, the ILTA Best Article Award, and the UNCG School of Education Outstanding Senior Scholar Award. Professor Chalhoub-Deville has published widely especially in second language testing and validity. She has also directed federally-funded projects to design and develop foreign language proficiency-based, computer adaptive tests, and a K-12 academic English language assessment battery.
Professor Chalhoub-Deville is the founder of the Mid-West Association of Language Testers (MwALT). She is also a founding member of the British Council Assessment Advisory Board and the Duolingo English Test (DET) Advisory Board. She is a former Chair of the TOEFL Committee of Examiners as well as a member of the TOEFL Policy Board. She currently serves on MetaMetrics Technical Advisory Committee, the DET Advisory Board as well as the Language Testing and Language Assessment Quarterly Editorial Boards.
British Council
Barry O’Sullivan is Head of Assessment Research and Development, the British Council, London, Visiting Professor of Language Assessment at the University of Reading, and Advisory Professor at Shanghai Jiao Tong University.
Validity: Theoretical Development and Integrated Arguments provides a historical overview of validity, targeting developments in both the UK and the US. It explores theoretical notions of validity as well as pragmatic validation practices and expands the arguments that need to be attended to document quality. The authors examine the need to consider, in addition to the psychometric evidence, which has continued to prevail especially in the US, other critical sources of quality evidence. They call attention to principled design and the evidence accumulated from various departments/groups involved in test design and development. They also promote the concept of impact by design, which places consequences at the top of the evidence chain to guide all testing efforts and quality documentation. They envision validity scholarship to attend to consequences at the individual, aggregate/group, and larger educational/organisational/societal levels. Concomitant with this attention to consequences are considerations of stakeholders and the tailoring of communication to engage intended groups. Such an approach yields a more convincing validity argument. The monograph ends by calling on professionals in the field to publish case studies which showcase localised validity arguments in practice. Local case studies represent critical endeavours to illustrate how evidence and arguments are pulled together to support the quality of a testing programme and all that it entails.
ISBN-13 (Hardback)
9781781799895
Price (Hardback)
£75.00 / $100.00
ISBN-13 (Paperback)
9781781799901
Price (Paperback)
£24.95 / $32.00
ISBN (eBook)
9781781799918
Price (eBook)
Individual
£24.95 / $32.00
Institutional
£75.00 / $100.00
£24.95 / $32.00
Institutional
£75.00 / $100.00
Publication
08/10/2020
Pages
216
Size
234 x 156mm
Readership
language testers and scholars
Illustration
12 figures
