Click the star to add/remove an item to/from your individual schedule.
You need to be logged in to avail of this functionality.
Log in
- Convenor:
-
Michiko Takagi
(The Japanese Saturday School of Brussels)
Send message to Convenor
- Stream:
- Japanese Language Education
- Location:
- Torre B, Piso 3, Auditório 2
- Sessions:
- Friday 1 September, -
Time zone: Europe/Lisbon
Short Abstract:
Interested in honing your skills in assessing the oral ability of JFL users? Let's work on this theme using the OJAE test, which has been researched and developed for 10 years by a team of former OPI testers.[JP]
Long Abstract:
OJAE (AJE-SIG since 2008) is a method of oral testing and assessment that has been researched and developed for 10 years. The members of the OJAE team, originally trained as OPI testers in the American ACTFL-OPI methodology, wished to create a European solution for JFL-users in accordance with the Can-do-descriptors of the CEFR and its JFL adaptation, JF Standard, as well as preceding studies. The OJAE team wishes to share the insights obtained through its R&D, providing an alternative to relying on one's "gut instinct" or sheer empiricism. It furthermore hopes that the propagation of the OJAE method throughout Europe is going to lead to fairer and more objective scores in the assessment of oral competence.
The OJAE panel will consist of three main parts, followed by a Q&A session. The first presenter introduces the OJAE testing, focusing on the B-levels: (1) Format: 2-on-2 (2 assessors, one doubling as the interlocutor with the test script/prompts); (2) test scenario and script; (3) assessment-sheet with its five aspects: ① Scope/Range ② Correctness ③ Fluency ④ Coherence ⑤ Interaction. She emphasizes that the assessment competence can be developed through the endeavor of the test-production itself.
OJAE can be applied multi-facetedly: Partly and wholly, for various tests of proficiency, achievement, pro-chievement, placement etc. The second speaker exemplifies actual applications of the OJAE method, such as in the final examination of the BA-qualification.
The third speaker elucidates the teachers' seminar of "JFL Oral Assessment 2016 in Brussels", conducted by the OJAE Team, on the Project: Good Dialogue Testers (co-hosted with the Belgian NK and supported by the JF Cologne). It is hoped that activities of OJAE as a AJE-SIG, such as aforementioned seminar, are to be expanded and conducted throughout Europe.
Accepted papers:
Session 1 Friday 1 September, 2017, -Paper short abstract:
The first part focuses on the testing of B-levels, since these are most frequently practiced and rated as subtle and difficult, being transitional from A to C. In addition, it will be emphasized that the assessment competence can be developed through the endeavor of the test-production itself.
Paper long abstract:
The first speaker presents the CEFR-based OJAE-Tests, focusing on the B-level. She introduces the characteristics of this testing method, dealing at the same time with her own acquisition of better competence in assessing the oral skill of JFL-users.
1. OJAE-Test: A testee applies for the level according to the self-assessment grid in his/her own language offered in the CEFR " Table 2. Common Reference Levels: self-assessment grid" (CEFR, Engl. 25-26), out of the levels consisting of six: A1-A2-B1-B2-C1-C2. Different prompts as cues to inspire verbal acts are employed and the test scenarios of the whole procedure have been written. For the B1-level, e. g., two pictures of different apartments are chosen in order to give the task to select one of the choices. For the B2-level, somewhat more abstract question will be given, requiring prompts that could bear corresponding ideas. In the first part of the task, e.g., a testee will be asked to elucidate one rite of the passage out of the several examples of a prompt. The next question is a more abstract one such as why human beings in various cultures do achieve those kinds of rites. As is clear in the examples above-mentioned, to produce the tasks on-target itself can contribute to train one´s assessment skill.
2. Test-format: As a rule, 2-on-2 (2 assessors, one doubling as the interlocutor with the test script/prompts). It includes dialogue-parts embodying the ideals of the CEFR. Each task should be achieved within the time-limitation, i. e., the competence of speaking concisely is also required. The interlocutor is restricted to explain the tasks; the test duration varies 10 - 20 minutes.
3. Pass-Fail level-assessment: Via the assessment-sheet defined by the stratified standardization of six levels the performance is assessed. The assessment occurs concurrently with the on-going test.
Paper short abstract:
OJAE can be applied multi-facetedly. I exemplify actual applications of the OJAE method, such as in the final examination of the BA-qualification. It will be presented how class activities could be carried out concretely. We strive for the objective and transparent oral test.
Paper long abstract:
In order to verify the OJAE testing method we have for over a decade already taken more than 100 videos. Subsequently, I have tried to apply this method to the final BA exams at universities in Germany. It is required for this final exam, the subjects that have been taught during the semester should be dealt with. Furthermore, the examination should be objective and transparent as an oral test.
From the beginning of the BA courses, the class repeatedly viewed the OJAE level-illustrating DVDs (OJAE 2010). It was decided in consensus to set B2 as the target levelof speaking competence. The self-assessment table (CEFR 2: Self-Assessment Grid) played a crucial role. It was necessary for the students to be capable of assessing their own level.
For this type of testing, following the OJAE method, I chose matching partners for every student. These pairs proceeded to independently practice for the duration of the semester. Furthermore, during lessons, what I wanted to promote was the students cooperating on the task instead of working against each other. In my opinion, this point has a great effect on their motivation. In addition, the abilty to talk concisely was important. I often had the students practice summarizing their point of view within a limited time.
A strong point of the assessment method basing on the CEFR-OJAE, was the fact that it quickly became apparent which of the 5 skill categories were lacking and required more practice. Nonetheless, more of the R&D on the benchmarks individually as well as institutionally are necessary to define a clear common "assessment-scale" for an oral competence throughout the university-courses within the EU so that the students can be examined and assessed more objectively and fairly.
Paper short abstract:
In 2016 in Brussels the OJAE Team as an AJE-SIG co-hosted a teachers' seminar with the Belgian NK on the project Good Dialogue Testers, supported by the JF Cologne. This part exemplifies how such seminars in various EU regions can contribute to the sharing of a fair, transparent assessment method.
Paper long abstract:
The 3rd speaker presents the recent activities of the OJAE-Team as the AJE-SIG, inviting the JFL people to join the endeavor of OJAE, which in turn would contribute to the upskilling in assessing of the oral competence of JFL-users:
1. The OJAE Team conducted a teachers´ seminar in March 2016 in Brussels on its "Project: Good Dialogue Testers" (co-hosted with the Belgian NK; supported by the JF Cologne). The seminar with 4 groups (11 participants in 3 Groups +1 OJAE G with 4 members) proceeded as follows:
Step I:
(1) The presentation of the OJAE´s R&D
(2) Confirming speaking competence of B2 according to the CEFR-Table 3 "Common Reference Levels: qualitative aspects of spoken language use" (Jpn Translation, Yoshijima/Ōhashi 2004: 30-31)*
(3) OJAE-B2 test, conducted in Berlin in 2016, Testees N & S, 18-min. Film;
(4) Assessment Nr.1: individually > Common counting.
Step II:
(5) (brand-new) OJAE-Standard**
(6) OJAE-Test live on two testees, prior to the seminar recruited, B&A-S, students of the Univ. B;
(7) Assessment Nr.2: individually > common counting;
(8) Discussion → Results, being often conform, though with some "overt" differentiation.
2. Results and reflections:
a. OJAE-test with rather high validity, though exact analysis is still to come.
b. The gaps in assessment appear to be caused by the difference in the individual/inner scales as well as in the institutional benchmarking -> Efforts in standardization are needed definitely.
3. Towards the vitalization of the OJAE-SIG; Adaptation to each JFL-front lines
a. Planning and conducting of the similar seminars in other regions in EU with various languages & cultures in the background.
b. Improvement of the assessment (Creative Test/Task production; Prompt selection; hands-on workshops for Assessment-sheet, Feedback paper etc.)
*"Old Scale" in terms of the OJAE study
** "Brand-new" OJAE Table, incrementally-stratified