| Asia Pacific Journal of Corpus Research Vol. 2, No. 1, pp. 35-45 |
| Abbreviation: APJCR |
| e-ISSN: 2733-8096 |
| Publication date: 31 August 2021 |
| Received: 29 October 2020 / Received in Revised Form: 30 May 2021 / Accepted: 28 July 2021 |
| DOI: https://doi.org/10.22925/apjcr.2021.2.1.35
|
Predicting CEFR Levels in L2 Oral Speech, Based on Lexical and Syntactic Complexity
|
| Xiaolin Hu (Tokyo University of Foreign Studies) |
| Copyright 2021 APJCRThis is an open access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted, distribution, and reproduction in any medium, provided the original work is properly cited. |
Abstract |
| With the wide spread of the Common European Framework of Reference (CEFR) scales, many studies attempt to apply them in routine teaching and rater training, while more evidence regarding criterial features at different CEFR levels are still urgently needed. The current study aims to explore complexity features that distinguish and predict CEFR proficiency levels in oral performance. Using a quantitative/corpus-based approach, this research analyzed lexical and syntactic complexity features over 80 transcriptions (includes A1, A2, B1 CEFR levels, and native speakers), based on an interview test, Standard Speaking Test (SST). ANOVA and correlation analysis were conducted to exclude insignificant complexity indices before the discriminant analysis. In the result, distinctive differences in complexity between CEFR speaking levels were observed, and with a combination of six major complexity features as predictors, 78.8% of the oral transcriptions were classified into the appropriate CEFR proficiency levels. It further confirms the possibility of predicting CEFR level of L2 learners based on their objective linguistic features. This study can be helpful as an empirical reference in language pedagogy, especially for L2 learners’ self-assessment and teachers’ prediction of students’ proficiency levels. Also, it offers implications for the validation of the rating criteria, and improvement of rating system. |
Keywords |
| CEFR, Proficiency Predicting, Speaking Assessment, Lexical Complexity, Syntactic Complexity |
References |
| Bailey, K. (2005). Practical English Language Teaching: Speaking. New York: McGraw-Hill.
Bhat, S., & Yoon, S. (2015). Automatic assessment of syntactic complexity for spontaneous speech scoring. Speech Communication, 67, 42-57. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Council of Europe (2018). Common European Framework of Reference for Languages: Learning, Teaching, Assessment Companion Volume with New Descriptors. Strasbourg: Council of Europe. Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238. Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: the roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35(2), 115-135. Crossley, S. A., Salsbury, T., & McNamara, D. S. (2012). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29(2), 243-263. Ferris, D. R. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414-420. Hawkins, J., & Buttery, P. (2010). Criterial features in learner corpora: theory and illustrations. English Profile Journal, 1(1).1-23. Hawkins, J., & Filipovic, L. (2012). Criterial Features in L2 English: Specifying the Reference Levels of the Common European Framework. Cambridge: Cambridge University Press. Izumi, E., Uchimoto, K., & Isahara, H. (2004). The NICT JLE corpus: Exploiting the language learners’ speech database for research and education. International Journal of the Computer, the Internet and Management ,12(2), 119-125. Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12(4), 377-403. Kang, O., & Yan, X. (2018). Linguistic features distinguishing examinees’ speaking performances at different proficiency levels. Journal of Language Testing & Assessment, 1, 24-39. Kim, J. (2014). Predicting L2 Writing Proficiency using linguistic complexity measures: A corpus-based study. English Teaching, 69(4), 27-51. Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4),474-496. Lu, X. (2011). A corpus‐based evaluation of syntactic complexity measures as indices of college‐level ESL writers’ language development. TESOL Quarterly, 45(1), 36-62. Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190-208. Milanovic, M. (2009). Cambridge ESOL and the CEFR. Research Notes, 37, 2-5. Tono, Y. (Ed.). (2013). The CEFR-J Handbook: A Resource Book for Using CAN-DO Descriptors for English Language Teaching. Tokyo: Taishukan Publishing. Wisniewski, K. (2018). The empirical validity of the Common European Framework of Reference Scales. An exemplary study for the vocabulary and fluency scales in a language testing context. Applied Linguistics, 39(6), 933-959. Wolfe-Quintero, K., Inagaki, S., & Kim, H. Y. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy, & Complexity. Honolulu: University of Hawaii Press. Yoon, H. (2017). Linguistic complexity in L2 writing revisited: Issues of topic, proficiency, and construct multidimensionality, System, 66, 130-141. |
The Author |
| Xiaolin Hu is a PhD student at Tokyo University of Foreign Studies. Her principal research lies in the field of corpus linguistics and identification of CEFR criterial features. |
The Author’s Address |
| First and Corresponding Author Xiaolin Hu PhD Student Graduate School of Global Studies Tokyo University of Foreign Studies 3-11-1, Asahi-cho, Fuchu-city, Tokyo 183-8534, JAPAN E-mail: hu.xiaolin.t0@tufs.ac.jp  DOI QR Code |
