Paper short abstract:

This presentation reports preliminary results of the project which aims at (1) identifying features of good academic speaking in Japanese; (2) developing an assessment scale for academic speaking proficiency in order to develop a task-based assessment tool.

Paper long abstract:

The number of international students in Japan increased from 140,000 to 270,000 in the past ten years. However, none of the large-scale tests used for college admission purposes include speaking component, and an emphasis is placed on either reading (Noguchi et al., 2008) or listening (Kadokura, 2005). This has created a problem of admitting a large number of students with a gap between reading/listening and speaking proficiency.

In order to deal with this problem, the current project aims at (1) identifying features of good academic speaking in Japanese; (2) developing an assessment scale for academic speaking proficiency and (3) developing an online task-based assessment of academic speaking to be used for admission and placement in the Japanese university. The data consisted of speech samples from thirty native speakers (NS) and sixty non-native speakers. All were college students. There were thirty argumentative quest ions. Each participant talked about four topics that were randomly presented on the computer screen. The rating scale was developed by adopting IELTS, TOEFL, and ACTFL OPI Scale as well as referencing CEFR (Council of Europe, 2001), and used to assess participants' speaking proficiency.

The results showed that the proficiency level of NNS differed from A2 to C1 in terms of CEFR while that of NS ranged from C1 to C2. Their speech samples were also analyzed in terms of complexity, accuracy, fluency (CAF), and functional adequacy (FA) (Kuiken & Vedder, 2018). The results indicated that all measures of CAFFA correlated with the rating scale, but that NS an NNS differed in all of the CAFFA measures. Also, advanced NNS's speech was more coherent, complex and accurate than that of intermediate NNS. Qualitative analysis showed that intermediate NNS were unable to use conjunctive particles and deletion effectively to control the topic shift compared to NS or advanced NNS. Also, they were unable to use low-frequency words compared to advanced NNS.