The Multimodal Context of Phonological Learning
Debra M. Hardison [+–]
Michigan State University
View Website
Debra M. Hardison (PhD, Indiana University) is a faculty member in the Department of Linguistics, Languages, and Cultures at Michigan State University. She has directed language and academic programs, and taught courses on second-language phonetics and phonology, linguistics for language teachers, second language acquisition, and research methods. Her research focuses on auditory-visual integration in spoken language processing, co-speech gesture, applications of technology in perception and production training, and the relationship between learner variables and oral communication skill development. She has published in numerous handbooks, edited collections, and journals such as Language Learning & Technology, Applied Psycholinguistics, and Language Teaching Research.
This book fills a need in applied linguistics by providing a comprehensive view of the multimodal context in which speech is perceived, produced, taught, and learned. It is both authoritative and practical, combining research review and a pedagogical orientation presented by a researcher and teacher whose work in second language acquisition has centrally involved the wider context in which pronunciation is situated.
The book reviews the early foundation established by speechreading studies that explored the extraction of speech information from a talker’s facial movements and the interaction of auditory-visual cues in perceptual illusions. It then focuses on the role of auditory, visual, and tactile information in second-language perceptual learning and the interactional functions of eye gaze within the complex of nonverbal communication that incorporates head movements and manual gestures. The book also details an original mixed-methods study of the eye-gaze behavior of perceivers when viewing the face of a speaker producing their first language (English) and a speaker producing their second language (French) in different stimulus conditions.
The issue of variability in speech emerges as a central theme throughout the book and one which researchers and teachers are encouraged to exploit through their selection of multimodal materials and their approach to pronunciation teaching.
Table of Contents
Chapter 1
Chapter 1 addresses the roles that visual information from a talker’s face can play in a communicative interaction, including the establishment of a visual connection between interlocutors, and the improvement of perceptual accuracy. In contrast to early notions that speech is perceived as an auditory signal, complemented when needed by information from other modalities (e.g., to deal with noise and/or hearing loss), research has underscored the important role of multimodal input, and emphasized the impacts of stimulus, talker, and contextual factors on the contribution of sensory cues to the perceptual outcome.
Chapter 2
Chapter 2 outlines the role of visible speech cues in perception across the lifespan. Research on the integration of auditory and visual cues in speech perception flourished after the initial publication of the perceptual illusion known as the McGurk-McDonald effect in 1976, and ultimately, included research on the effect as experienced by native speakers of several languages in addition to English, and second-language learners. This illusory effect has contributed to our understanding of the neurophysiological processing of speech.
Chapter 3
Chapter 3 discusses the development of high variability perception training (HVPT), which initially focused on auditory input. Successful training resulted in the development of perceptual categories robust to stimulus and talker variability, with generalization of improved abilities to the perception of novel stimuli and unfamiliar voices. This milestone set the stage for the addition of visual input in segmental perception training with transfer of improved skills to greater production accuracy and earlier identification of words in connected speech. Neurophysiological findings are reviewed that support auditory-visual integration, the link between speech perception and production, and the use of multimodal input in language instruction.
Chapter 4
Chapters 4 and 5 address the question of where perceivers look on the face of a talker and how eye gaze patterns change as a speech event unfolds. Chapter 4 provides some background on eye-movement research with an emphasis on speech. Chapter 5 presents original data from a study that tracked the eye movements of L1 English intermediate-level L2 French learners during L1 and L2 vowel identification tasks in different stimulus conditions. Identification accuracy and several eye gaze measures were calculated for both languages, and eye movements were visualized through heatmaps and gaze plots. This study was able to assess how much of the variability in each measure was accounted for by the independent variables of language and stimulus condition, and how much could be attributed to variability across the learners. In post-study interviews, the learners reflected on their use of facial cues in phonological learning.
Chapter 5
Chapters 4 and 5 address the question of where perceivers look on the face of a talker and how eye gaze patterns change as a speech event unfolds. Chapter 4 provides some background on eye-movement research with an emphasis on speech. Chapter 5 presents original data from a study that tracked the eye movements of L1 English intermediate-level L2 French learners during L1 and L2 vowel identification tasks in different stimulus conditions. Identification accuracy and several eye gaze measures were calculated for both languages, and eye movements were visualized through heatmaps and gaze plots. This study was able to assess how much of the variability in each measure was accounted for by the independent variables of language and stimulus condition, and how much could be attributed to variability across the learners. In post-study interviews, the learners reflected on their use of facial cues in phonological learning.
Chapter 6
Chapter 6 reviews a range of nonverbal cues used in communication such as eye gaze in its interactional role (e.g., directing turn-taking), head movements, and manual gestures, in addition to the technologies available to examine the co-occurrence of head nods, eyeblinks, brow raise, lip movements, manual gestures, and features of speech (e.g., pitch-accented vowels). As with other components of a speech event, variability exists across speakers and cultures in their gestures, and familiarity plays a role in the contribution of a speaker’s gestures to comprehension as it does for accent and topic.
Chapter 7
Chapter 7 emphasizes that multimodal input in phonological teaching and learning is not a recent concept. It is among several issues in the field that can be found in the writings of Daniel Jones from the early 1900s. Jones was also an advocate of the importance of intelligibility as the goal of language learning, and the development of positive attitudes toward accents. These concepts form part of a recommended foundation for teacher education, which should also address pre- and in-service teachers’ concerns, such as insufficient confidence and training to teach pronunciation, among others.
Chapter 8
Chapter 8 outlines the basics of a multisensory approach to the teaching and learning of speech sounds in American English as they are produced in isolation and in connected speech. Teachers and learners should know how consonants and vowels look, sound, and feel in order to maximize the value of visual cues to guide perception and production, and of proprioceptive feedback to guide production. The chapter was designed for teacher educators, and pre- and in-service teachers to address the gap that they often report in their knowledge base with regard to phonetics and its application to language teaching and learning.
Chapter 9
Chapter 9 concludes the book with a summary of the importance of multimodal speech input and the key theme of variability that emerges throughout the chapters in such areas as speechreading ability, the occurrence of perceptual illusions, success of perception training, attitudes toward gestures, and eye gaze behavior. Variability characterizes our speech and should be addressed in the classroom to ease learners’ transition from citation forms to understanding connected speech.
ISBN-13 (Hardback)
9781000000000
Price (Hardback)
£75.00 / $100.00
ISBN-13 (Paperback)
9781000000000
Price (Paperback)
£24.95 / $32.00
ISBN (eBook)
9781000000000
Price (eBook)
Individual
£24.95 / $32.00
Institutional
£75.00 / $100.00
£24.95 / $32.00
Institutional
£75.00 / $100.00
Publication
01/09/2025
Pages
224
Size
234 x 156mm
Readership
students, teachers and scholars
