Analysis of essays by European learners of Japanese as a second language: examining results of holistic rating and multiple-trait rating [JP]

Mari Tanaka (Nagoya University of Foreign Studies)
Shin Abe (Tokyo University of Foreign Studies)
Michiyo Fuchs-Shimizu
Japanese Language Education
Torre B, Piso 3, Auditório 2
Start time:
2 September, 2017 at 9:00
Session slots:

Short abstract:

This panel reports findings from an analysis of essays by European JSL learners, drawn from the I-JAS corpus. We performed holistic and multiple-trait rating using six levels, and three multiple-trait perspectives: "Japanese Proficiency," "Purpose and Content," and "Organization and Cohesion."

Long abstract:

 Nowadays, the corpus of Japanese language learners has been widely opened to the public. However, there have been few corpora with evaluations, and most of those have been limited to data for speakers whose L1s are Chinese, Korean, and English.  In this study, three raters applied a holistic rating of levels 1-6 to essays (covering 611 written samples) from the International Corpus of Japanese as a Second Language (I-JAS). Of these, we found data for 193 samples of European learners, and we applied multiple-trait rating (MT) to 55 of the essays with the highest rates of concordance of holistic rating. The traits of MT are "Japanese Proficiency," "Purpose and Content," and "Organization and Cohesion." This panel reports the findings of this data analysis.  For "Japanese Proficiency," using holistic ratings for all 193 samples, we divide them into three groups: high (scores of 4 & 5), middle (score of 3), and low (scores of 1 & 2). We calculate the average value of text features of each group, and specify statistically significant linguistic features such as particles and verbs, etc. We analyze differences found in writings of each group.  For "Purpose and Content," we provide representative examples of writing for each level, and show a relationship between holistic rating and MT rating. The distribution of writing samples shows that, of the 55 samples we examined, most were at Level 2, followed by Level 4, and then Level 3. Even for writing samples at MT Level 2, we could see high scores for holistic rating. Having done this analysis, we then perform qualitative analysis of the writing.  For "Organization and Cohesion," we analyze macro-organizational essay patterns seen in European learners' essays, along with the meta-language, referring to the levels. The use of meta-language is related with cohesion between paragraphs. Of the 55 writing samples, about half were at Level 3. We compare these to higher-level essays, and perform a qualitative analysis.  This study aims to contribute to future developments in Japanese writing education in Europe. We hope to have a discussion with the floor based on our results.