This article is sourced from the WeChat public account “BangBang English Words”. For sharing only.
The essence of exams is evaluation. Evaluation acts as a guiding force that influences and leads teaching. The type of evaluation determines the teaching methods. If English tests focus solely on dictation and precise memorization of words, teachers will have students repeatedly practice dictation in class. However, if the exams assess the comprehensive use of vocabulary in context, it will encourage frontline teachers to adopt the practices of Teacher Yan from Shenzhen University experiment, integrating vocabulary into narratives, recording their experiences with learned vocabulary, and closely linking language learning to their lives, learning through practice, and using practice to promote learning.
On May 1, 2024, Teacher Yan from Shenzhen University experiment shared the following in her friend circle:
Teacher Yan wrote this diary to help students review the phrases appearing in the Foreign Language Teaching and Research Press high school English textbooks, compulsory volumes one to three (the editors listed a total of 45 phrases in the appendix at the back of the first three volumes). In her diary, she chose to use ten of them, recording her trip to Seoul, South Korea during the May Day holiday from a threefold perspective of what she saw, heard, and felt. This 159-word diary entry is well-structured, uses authentic language, and vividly illustrates the principle that learning a language is to better record one’s life and express inner thoughts.
Teacher Yan’s approach is influenced by exam orientation. Starting from this summer vacation’s citywide research exam for high school freshmen and sophomores, we imitated the vocabulary testing methods in the Shanghai college entrance examination English test: in a text, assessing students’ vocabulary mastery through a fill-in-the-blank format with 11 options and 10 blanks. Teacher Yan happened to design a text involving ten phrases (marked in red). This is a classic example of how exam evaluation reform drives and guides frontline teachers to carry out teaching reform.
In reading comprehension tests, if we always use questions like “Which of the following is true?” to assess students’ judgment of the correctness of detailed information, and rarely or never use questions like “Why does the author mention… in paragraph…?” and “Which of the following best describes …?” and “What is paragraph 2/3/4 or the text mainly about?” to assess students’ logical thinking, reasoning ability, and summarization skills, then teachers will only ask students factual questions that capture detail information in their reading instruction, neglecting analytical questions. Over time, students’ development of thinking quality will stagnate. Because they do many true or false type questions, they will also believe that all people and things in the world have absolute right or wrong distinctions, resulting in weakened critical thinking and dialectical thinking skills.
Since exams are so important, ensuring the high quality of test items becomes crucial. We need to ensure that test items can deeply assess students’ language abilities and thinking qualities, that the selected points and item design can reflect the author’s writing ideas, foster students’ awareness of discourse, and guide frontline teachers’ teaching. In this critical aspect, generative artificial intelligence technology represented by ChatGPT can assist us in designing and improving test items. How to use GPT to assist and improve the design of English reading comprehension tests? What important procedures and useful instructions are there? The author will illustrate with examples below.
The following is a test item found on the internet:
The following is a version of the test item provided online:
My criticisms of this version of the test item:
Question 24, the point of testing pronoun reference is repeated with the reading comprehension seven-choice question, which constitutes a redundant test, violating the principle that test items should sample knowledge and abilities broadly;
Question 25, the information in the original text is simply “Look at fossils”; the option design is still “look at fossils”, which does not effectively assess students’ language abilities;
Question 26, the options lack homogeneity; both A and B start with “why”, and the correct answer C is inaccurately summarized, missing the key term CGI;
Question 27, the question lacks focus and direction, which is not conducive to the development of students’ awareness of discourse, equivalent to letting students make four T/F choices, which cannot effectively assess students’ thinking quality or promote deep reading.
After polishing the original text using GPT, I obtained the following test item:
The modified version of the test item I obtained with GPT’s assistance:
These four test items effectively avoided the four issues I mentioned earlier, especially the improvement of question 25, which replaced the original word “looking at” with “observing”, effectively assessing students’ language ability (being able to express the same idea in many different ways). The language in the text “fossils are windows into the past” exposed students to authentic and vivid language expressions. The improvement of question 27 fostered the development of students’ deep reading abilities and cultivated their awareness of discourse. GPT’s analysis of this text indicated that through the last paragraph, the author intends to express future research prospects. The content of the correct answer C is precisely the author’s outlook: with the progress of paleontologists’ research, the depiction of these extinct animals in future dinosaur films is expected to continue improving.
During the process of improving the test items and text, my operational procedures and basic instructions used are as follows:
「READING」
1
Step one: Copy the original text into the GPT-4 dialogue box (if GPT cannot be used, you can use domestic Kimi as a substitute):
Input instruction: Please help me improve or polish the text, correct the diction or grammar mistakes and make the language more idiomatic, without making the text become more difficult. After obtaining the result, replace the advanced vocabulary that appears with simpler synonyms, using the instruction: Please help me replace the words “… and …” with their simple synonyms while keeping the rest of the text the same.
Step two: Perform discourse analysis on the obtained text using the instruction: please help me analyze how the text is structured. This instruction allows GPT to summarize the author’s writing ideas. When designing test items, we should follow this thought process, ensuring that the keywords in the questions align with the keywords summarized by GPT. Remember: our questions are not meant to confuse students, but to help them reconstruct the structure of the text, clarifying their understanding of the text’s structure.
2
3
Step three: Design correct answers and distractors. You can ask GPT to paraphrase the original text; many expressions in GPT’s paraphrase will inspire our design of the correct answer language. Alternatively, we can design a correct answer and a distractor ourselves, demonstrate to GPT, and then use GPT’s powerful language capabilities and reserves to create other distractors. The instruction used at this time is as follows: Help me finish making the test by creating three distractors (wrong answers). The wrong answers should also be related to the text. They should align with the correct answer in terms of structure.
Use secondary instructions for feedback, such as: Choice C is too long. Make it more concise. If the distractors are not satisfactory, extract the corresponding paragraph from the article and ask GPT to lengthen it slightly to include more nouns or verbs, which can serve as a basis for designing distractors. Repeat the above process to obtain high-quality distractors.
Step four: Input both the article and the questions to GPT, asking it: Do you have any suggestions for me to improve the text and the questions and answers? Give me your version. Review the results provided by GPT, selecting the aspects you feel it performed well in to learn from and improve your test design. Sometimes, if we think of a good question, we can input that question into GPT, ask it to answer, and limit its response to a specific word count, such as answering the question with no more than 8 words. We can then use its generated answer as our correct answer and have it mimic the language structure of this correct answer to create three distractors using the instructions mentioned earlier.
4
In summary, in enhancing the quality of English test item design, GPT and Kimi, with their powerful language abilities and vast language reserves, are excellent assistants in creating high-quality English test items. Frontline teachers can seek close collaboration with them in their daily test design practices to provide students with high-quality test items, better achieving the fundamental goals of moral education, selecting talent for the country, and guiding teaching.
END
For premium book purchases, please click the image to enter the mall for viewing:
Original books are not easy to come by. The Sanhao Teacher Alliance calls for choosing formal channels to purchase authentic books, protecting authors’ copyrights, and thanks to all teachers and parents for their support!
Sanhao Teacher Alliance
Ten Thousand Teacher Community
