AI Education for Teens with Large Language Models

Chu Leyang, Wang Hao, Chen Xiangdong

Abstract: Large Language Models (LLMs), as an advanced AI technology, are profoundly impacting human life. Compared to traditional AI technologies, LLMs can handle vast amounts of text data and play a significant role in areas such as natural language processing. The rapid development and application of LLMs have significantly influenced the curriculum content, teaching models, and learning platforms in AI education, necessitating timely reforms in AI education for teenagers. To address this change, this study designs a youth AI curriculum framework oriented towards LLMs from three dimensions: curriculum content framework, LLM-based teaching models, and LLM-assisted learning platforms. Using teaching activity design as an entry point, it explores how to align this curriculum framework closely with core subject competencies. Based on this framework, a high school LLM demonstration course has been designed and developed, focusing on how to utilize a self-developed platform (LLM 4 Kids) for human-machine collaborative teaching. The unit content of “Prompts and Evaluation for LLM” from the demonstration course is selected to explain how to effectively apply and integrate LLM technology in the teaching process. By providing a youth AI curriculum framework oriented towards LLMs and explanatory cases, this research offers AI education practitioners a framework system and curriculum reference for LLMs, promoting the latest cutting-edge knowledge of artificial intelligence into primary and secondary school classrooms, thus helping teenagers adapt to the rapidly developing era of AI.

Keywords: Large Language Models; AI Curriculum; Human-Machine Collaboration; LLM 4 Kids

Classification Number: G434 Document Identification Code: A

∗ This article is a phased research achievement of the National Education Science Planning General Project “Research on Youth AI Education Based on Large Language Models” (Project No.: BCA230276).

Please cite the following information:

Chu Leyang, Wang Hao, Chen Xiangdong. AI Education for Teens with Large Language Models [J]. China Electric Education, 2024, (4): 32-44.

1. Introduction In recent months, Large Language Models (LLMs) have sparked a new wave of technology in society due to their fluency and coherence in generating text, closely resembling human-like dialogue. Their influence is gradually becoming evident in fields such as education, healthcare, media, and science. These leading LLMs are self-supervised learning models based on the Transformer architecture. By utilizing vast amounts of text data for pre-training and further optimizing performance through fine-tuning methods such as Reinforcement Learning Through Human Feedback, these models exhibit strong capabilities in various language and knowledge processing tasks. Because they can capture rich semantics and text patterns, and excel in language generation, information retrieval, machine translation, natural language understanding, and other tasks, new applications like ChatGPT have quickly become hot topics in the public domain. Notably, LLMs are changing the technological paradigm of artificial intelligence, especially in the field of text data processing: on one hand, through self-supervised learning, models can learn from large amounts of text data without explicit labeling or human intervention; on the other hand, these models interact with users by generating natural language text, resembling human conversational partners. These changes pave the way for the application of LLMs in more scenarios. With the continuous emergence and application of LLMs such as GPT-4, PaLM2, and LLaMA, AI education in primary and secondary schools also faces new challenges and opportunities. Firstly, traditional AI education often focuses more on programming, logical reasoning, and task-specific model training, with little emphasis on interacting with AI models in natural language or using AI models to acquire and understand information. In the current context of LLM prevalence, understanding how these models work, how to interact effectively with them, and how to responsibly utilize their capabilities to solve practical problems requires helping students learn new skills to adapt to and leverage the self-learning and self-improvement abilities of these models. Secondly, the emergence of LLMs has brought about a series of new risks and challenges, including the potential for models to generate incorrect or harmful information, as well as serious privacy violations and data security issues associated with their use. Identifying and managing these risks, as well as how to use these models safely and responsibly, has become increasingly important. Thirdly, the development of LLMs has expanded many new human-machine collaborative learning methods. LLM-based programming assistance plugins like CodeX/Copilot can be directly applied in AI programming learning, providing students with instant code suggestions and helping them understand complex code structures and algorithms. Students can enhance their programming skills through interaction with the models, while teachers can use the feedback from these models to understand students’ learning progress and difficulties, thereby providing more targeted guidance and assistance. In response to the rapid development of LLMs, the learning content, teaching methods, and learning tools of AI courses urgently need to be transformed. It is worth mentioning that countries around the world are promoting research on AI curricula for primary and secondary schools oriented towards large models. For example, the National Science Foundation (NSF) in the United States issued an urgent research initiative on May 8, inviting the research community to quickly study the use and teaching of AI in formal and informal K-12 educational environments in response to the unprecedented pace of progress in LLMs, particularly emphasizing the need to clarify the essential knowledge structures that K-12 students need to master and how to use large language models for AI-related teaching. Singapore’s AI Singapore Goes to School program has already launched related courses to introduce students to the basics of AI and its applications, including an introduction to AI and machine learning, the basics of AI ethics, and how to use ChatGPT. China has always attached great importance to the popularization of AI at the basic education stage. Whether it is the 2017 version or the 2020 revised “General High School Information Technology Curriculum Standards” or the 2022 version of the “Compulsory Education Information Technology Curriculum Standards,” artificial intelligence occupies an important position. It should be noted that previous research related to AI courses in China focused on exploring how students’ core subject competencies can be implemented within the traditional AI knowledge system. However, as mentioned above, large language models have brought new knowledge content, teaching models, and platform tools to AI education. Although research on the impact of large language models on youth AI education is still scarce, these new features have been explored more deeply in the public domain. In this context, to help youth quickly adapt to the changes brought about by LLMs, the construction of AI education curricula in China also needs to keep pace with the times. In summary, this article aims to systematically sort out and analyze the impact of large language models on AI education, designing a youth AI curriculum framework oriented towards LLMs from three dimensions: curriculum content framework, LLM-based teaching, and LLM-assisted learning platforms, and exploring how to align teaching activity design with core subject competencies. Based on this, a high school demonstration course is presented as an explanatory case to illustrate how to design LLM courses and carry out teaching practices. By deeply exploring the practical impact of the development of large language models on artificial intelligence education, this research will provide a framework system and demonstration courses for the new generation of youth AI curricula.2. Literature Review(1) Current Status of AI General Education Oriented Towards Large Language Models Large language models have become an undeniable trend, and due to their differences from traditional artificial intelligence, updating public AI general education is particularly important. Some social institutions and MOOC platforms have responded quickly, launching general courses for the public. EDX has released an official series of courses on ChatGPT, covering an introduction to ChatGPT, prompt engineering and advanced usage methods, specific applications of ChatGPT in education, business, healthcare, and other fields, as well as case analyses on data, coding, and technology dimensions across business scenarios. In addition, EDX has also launched a professional certification program in “Large Language Models,” focusing on how to use popular LLM technology frameworks to solve practical problems. The Coursera platform has also included several courses on ChatGPT, such as Vanderbilt University’s “Prompt Engineering for ChatGPT,” which mainly introduces effective use of ChatGPT and other LLM applications through prompt engineering; developing applications based on complex prompts using prompt templates; and creating applications for personal management and business scenarios. The University of Michigan has offered a free public course on Coursera titled “Teaching with ChatGPT,” focusing on the principles of ChatGPT, its advantages and limitations, as well as the social impacts and ethical issues arising from its use. Moreover, Coursera includes self-built course projects such as “Using AI for Market Research,” “Writing Sci-Fi Novels with Dall-e,” “Using ChatGPT for Machine Learning: Image Classification Models,” and “Using GPT-2 for New Methods.” Andrew Ng, the founder of DeepLearning.ai, has opened four short courses on LLMs on his platform, guiding learners from beginner to advanced levels in mastering skills such as prompt engineering for ChatGPT, building systems using APIs of models like the GPT series, implementing richer functionalities for LLM applications with a framework called LangChain, and quickly fine-tuning customized large language models. Notably, learners can also experience different development projects using the built-in programming modules of the learning platform. Correspondingly, starting from the spring of 2023, renowned universities such as Princeton University, Stanford University, Carnegie Mellon University, and ETH Zurich have begun to implement general courses related to LLMs in their formal curricula. These courses focus on the foundational knowledge of LLMs, emphasizing principles, applications, and practices. From a foundational knowledge perspective, Princeton University’s COS-597G course includes a deep review of representative LLMs such as BERT and GPT-3, while Stanford University’s CS224n course continues the development of traditional natural language processing and adds LLM knowledge modules from a linguistic perspective. From an application perspective, these courses cover prompt engineering for LLMs, applications and potential risks of LLMs, retrieval-based language models, and multimodal language models, among others. Additionally, many courses explore innovation and problem-solving using LLMs. For example, Carnegie Mellon University has initiated a generative AI innovation incubator project alongside setting up specific LLM courses, aiming to encourage multi-disciplinary learners to leverage LLMs for social innovation through free tutorials, lectures, group discussions, and hackathons. Certainly, some courses emphasize the systematic nature of content. For instance, ETH Zurich’s LLM-s23 course covers a complete knowledge system from probability basics and modeling fundamentals to fine-tuning and inference. Notably, these newly added courses around LLMs generally emphasize the ethics of LLM applications, integrating it as an important teaching content throughout the curriculum, covering issues such as data privacy breaches caused by the technical limitations of LLMs, biases and toxicity, hallucinations, and the social harms of model misuse. It is evident that the current AI education oriented towards LLMs exhibits several characteristics: firstly, the knowledge coverage of relevant courses is broad. Current general education courses not only focus on the foundational theories of LLMs, such as in-depth analysis of GPT series models, but also explore application methods in a wide range of downstream tasks. This broad knowledge coverage helps learners comprehensively understand the potential and application scope of LLMs. Secondly, the emphasis is on the application of LLMs in real-world scenarios. In the aforementioned general education courses, training for practical applications appears particularly important. This includes teaching prompt engineering, using commercial LLM APIs to build applications, and using specific frameworks to develop richer LLM applications. The accompanying resources and activities involved in these courses particularly encourage learners to leverage LLMs for multi-disciplinary innovation. Thirdly, there is additional focus on the ethics of LLM applications. As LLMs penetrate various fields, their potential social impacts and ethical issues are increasingly being recognized. This is particularly evident in general courses in higher education, which have begun to embed the ethics of LLM applications into different aspects of the curriculum, fostering students’ systematic understanding of LLM application ethics and encouraging responsible use of this technology. The aforementioned public and higher education LLM general education can serve as important references for youth AI education. Compared to the rapid responses in the public domain and higher education institutions, the impact of LLMs on basic education has not yet been systematically addressed. Although there have been discussions on how to implement AI education oriented towards LLMs in primary and secondary schools, the emergence of ChatGPT has been too rapid, and currently, apart from some small-scale attempts initiated by the NSF and certain countries, large language models have not yet entered the realm of AI education in primary and secondary schools.(2) The Transformation of Programming Education Due to Large Language Models The impact of large language models on artificial intelligence education is not only reflected in changes to the curriculum content framework but also brings about new transformations in various aspects of programming education, a crucial component of AI education. LLMs, having been pre-trained on vast amounts of source code and natural language data, have demonstrated exceptional capabilities in understanding code structures and generating code or text. LLM-based tools such as OpenAI’s Codex (Copilot), DeepMind’s Alpha Code, and Amazon’s CodeWhisperer have begun to drive transformation in various programming education settings. Existing research indicates that large language models are changing programming education methodologies from dimensions such as expanding teaching methods, optimizing teaching processes, generating learning resources, and improving learning assessments. Firstly, LLMs are changing the teaching methods of programming education. By using LLMs, teachers can more effectively address inherent challenges in traditional programming teaching activities, such as handling Programming Error Messages (PEM). For a long time, PEM has been a significant challenge for both teachers and novice programmers. Teachers often need to spend considerable time and effort explaining PEM information, while students may lose motivation due to the poor readability of PEM. Researchers have found that using large language models like GPT-3 can effectively transform PEM information into easily understandable natural language, thus becoming a powerful tool for teaching, helping students learn more effectively from PEM. Secondly, many explorations in LLM programming education attempt to optimize conventional algorithm teaching processes, allowing teachers to focus more on teaching advanced algorithms. Algorithm teaching is often delayed due to students’ difficulties in mastering specific programming language syntax and programming principles, making it challenging to ensure teaching effectiveness. By using LLMs, teachers can initially focus on teaching algorithms and problem-solving skills, leaving the teaching of syntax and programming principles for later. Additionally, LLMs can assist learners in generating some initial code, giving them the opportunity to extend code blocks and thus boosting their confidence in learning. It should be noted that research has shown that accurately describing algorithm steps in prompts will assist learners in generating more effective initial code; other studies have also indicated that when more complex programming statements are decomposed into smaller tasks, utilizing specific prompting techniques can improve the quality of the initial code generated by programming assistance plugins like Copilot, thereby facilitating more effective programming learning. This also suggests that future programming education may require additional focus on developing learners’ prompting skills, allowing them to describe the computational steps they wish to achieve in natural language as a means of guiding the model to produce effective outputs. Thirdly, LLMs assist in generating new learning resources. Existing research has found that learning resources created using LLMs, such as targeted exercises, abstract code explanations, and illustrative examples, can further enhance the effectiveness of computer education. For instance, researchers have discovered that the Codex model can generate new learning resources, including programming exercises and code explanations from a single introductory example. Other studies have explored how to utilize LLMs to generate high-quality code explanations, finding that LLMs can generate stylistically diverse code explanation texts based on specific requirements. As LLMs continue to evolve, teachers are expected to create more systematic explanatory cases using advanced LLMs, reducing cognitive load and facilitating more effective learning for students. Finally, LLMs improve the evaluation methods used in programming teaching processes. In introductory programming courses, assessment methods often focus solely on the correctness of the code, neglecting the quality and style of the code. However, by submitting code snippets to LLMs, learners can evaluate not only the correctness of the code but also compare differences among multiple correct codes and make judgments about the style and quality of solutions. Research suggests that this evaluation method, which emphasizes comparison and identifying variations, helps programming novices appreciate the efficiency and methodological differences in code writing. In summary, some researchers have made revolutionary attempts regarding LLMs in information technology education, particularly in programming education. This LLM-assisted teaching has exhibited characteristics of human-machine collaboration. Similar to how Python programming has always been an important component of AI education, an increasing number of programmers are using LLMs like Codex and Copilot to generate code in many open-source communities (such as GitHub), which also presents new challenges for AI education at the basic education level. Existing evidence has shown that programming education in AI education has shifted towards placing greater emphasis on reading and evaluating code rather than merely writing code. Primary and secondary AI education needs to pay attention to this industrial transformation, and both teachers and students need to learn how to utilize LLMs for teaching and learning.(3) Learning Platforms for Large Language Models As a transformative technology, LLMs allow users to communicate using natural language and enable the development of applications that were previously impossible to construct. However, the realization of these features is based on users’ ability to construct effective prompts. For general users, apart from using public testing chat tools like ChatGPT, direct access to the LLM’s API is often unavailable, typically requiring specific applications or prompt tools. Therefore, for novice learners of large models or general users, various prompt testing tools are the primary means of understanding large models. For example, the GPT-3 Playground is the web interface for OpenAI’s GPT-3 API and is often the first point of contact for many learners with large language model APIs. The GPT-3 Playground provides a user-friendly interface with several different parameter dials that can modify the corresponding parameters of the GPT-3 model and debug the respective outputs. In contrast, some researchers have developed dedicated prompt testing tools like Prompt IDE to assist learners in testing variations of prompts. These specialized prompt evaluation tools visually display the effects of different wording and structures of prompts on the accuracy of model outputs, allowing learners to assess the effectiveness of different prompting methods and iteratively optimize prompt templates for specific tasks. The roles of these tools can be summarized as follows. On one hand, they aim to let learners test the effectiveness of prompts for different downstream tasks, exploring how widely proven prompting strategies can be applied to various tasks, such as “Chain of Thought Prompting,” “Generated Knowledge Prompting,” and “Least to Most Prompting.” On the other hand, they help learners debug the parameters of LLM models, adjusting parameters such as Temperature (the degree of divergence in generated content) and Top-k (ranking the probabilities of each possible next word at each step and randomly selecting from the top k words) for models like GPT-3/4 to obtain high-quality LLM output content. Compared to learning support focused on basic content like natural language prompt construction, there are relatively few learning tools aimed at developers. A typical approach is to embed a coding environment within the learning platform to assist learners in practicing based on the content of instructional videos. For example, the DeepLearning.ai learning platform includes a Python compilation environment based on Jupyter Notebook, allowing learners to familiarize themselves with how to implement various application development practices step by step under the guidance of instructional videos. However, for specialized learning content with a high degree of specialization, such as fine-tuning large language models, even providing a compilation environment may not guarantee that novices can learn effectively, as they may face common difficulties such as interpreting PEM. Unlike platforms that focus on systematically learning LLM knowledge from scratch, some tools have also been developed to assist learners in building their applications using popular LLM technology frameworks. The design of these tools is top-down, providing learners with a visual interactive interface to intuitively learn how to construct LLM applications. For example, LangFlow and FlowiseAI are visual user interfaces for LangChain, supporting node-based programming, allowing learners to drag and drop sidebar components onto a canvas and connect these components to create demonstrations of custom LLM software workflows. Similarly, the Dust platform offers similar functions, building a series of prompt scaffolds for large language model applications to call external models and providing an easy-to-use graphical interface to construct prompt chains, parsing and processing language model outputs using a set of standard blocks and a custom programming language. Lastly, some tools can assist in learning how to fine-tune models locally using open-source LLMs: further training LLMs on smaller datasets for specific tasks to improve their performance on specialized tasks. For instance, the HuggingFace platform provides access to thousands of pre-trained LLMs and various training datasets, allowing learners not only to interact directly with various pre-trained LLMs but also to fine-tune these open-source LLMs using the dataset resources available on the platform. Another platform is H2O LLM Studio, a no-code graphical user interface for LLM developers and learners aimed at providing a framework for fine-tuning large language models. These platform tools typically come with more guidance and resources to help learners address complex issues they may encounter during the fine-tuning process. It can be seen that the development of LLMs has had a significant impact on multiple dimensions of artificial intelligence education, reshaping learning content and transforming teaching and learning methods. Simultaneously, effective teaching in the context of large language models also relies on advanced resources and tools for support. Based on this understanding, this paper starts with designing the LLM curriculum content framework and further constructs a human-machine collaborative teaching model based on LLMs, systematically explaining the changes in teaching and learning methods brought about by the development of LLM technology. Furthermore, to effectively support the implementation of teaching activities, it is necessary to design and develop an AI-assisted learning platform oriented towards the LLM knowledge structure. This platform can not only facilitate the construction of teaching management and learning environments but also be used for evaluating and providing feedback on teaching effectiveness. Therefore, this study will also demonstrate how to utilize the self-developed auxiliary learning platform for human-machine collaborative teaching through a chapter case of a demonstration course, better serving youth’s artificial intelligence education.3. AI Curriculum Framework for Youth Oriented Towards Large Language Models To systematically outline the impact of LLMs on youth AI curricula and effectively guide AI curriculum development and practice, it is necessary to develop a comprehensive curriculum framework, which includes the aforementioned curriculum content framework, LLM-based teaching models, and LLM-assisted learning platforms as three core dimensions. It should be noted that these three dimensions are interrelated: the curriculum content framework provides knowledge modules and corresponding core knowledge points for human-machine collaborative teaching, while the implementation of teaching will set teaching objectives, learning activities, and evaluation methods based on the content framework. At the same time, teaching activities also rely on the support of auxiliary learning platforms. For example, when the content framework involves prompt evaluation, teaching activities can be designed to encourage students to use LLMs to solve specific language tasks, while the auxiliary learning platform provides the necessary resources and tools to support these activities. It is noteworthy that when developing the curriculum framework, the compatibility with core subject competencies must be considered, and teaching activities should be designed to cultivate students’ comprehensive abilities, ensuring the quality of the curriculum and the achievement of educational objectives.(1) Curriculum Content Framework By sorting out the knowledge system of large language models and various curriculum cases mentioned in section 2.1, and referring to the 2017 version and the 2020 revised “General High School Information Technology Curriculum Standards” as well as the 2022 version of the “Compulsory Education Information Technology Curriculum Standards,” this research has planned and organized a curriculum content framework suitable for youth learning oriented towards LLMs. The entire content framework follows the conventional learning stages of “recognition, application, understanding, development, and training,” taking into account the logicality and comprehensiveness of the learning content. Overall, the preliminary designed framework divides LLM knowledge into eight core knowledge modules. Based on this, the knowledge structure, ability requirements, learning resources, and learning suggestions required for each module can be detailed. It should be noted that in the content framework, some modules are divided into two parts, “A” and “B.” The “A” part is the foundational module of the subject, emphasizing that students, as users of large language models, learn necessary conceptual knowledge and master basic application skills; the “B” part is the advanced module, which, although belonging to a specific subject, usually relies on the knowledge of the preceding module and includes more advanced technical and development knowledge. Specifically, as shown in Figure 1, Module 1 is “Experiencing LLMs,” aimed at providing students with experiences of LLM applications (similar to ChatGPT, Wenxin Yiyan, etc.) to help them initially understand LLMs; Module 2 is “Principles of LLM A,” which primarily aids students in deeply understanding the foundational knowledge of LLMs, including their development history, training methods, and model architecture; Module 3, “Prompts and Evaluation for LLM,” emphasizes practical operations to help students understand the importance of prompt engineering in maximizing the potential of large language models; Module 4, “Programming and Project Development A,” incorporates tools for LLM-assisted programming teaching into traditional programming learning, focusing on utilizing large language models for programming education; Module 5, “Ethics of LLM Applications A,” encourages students to reflect on various ethical issues arising from the application of large language models.

Modules 1 to 5 cover foundational knowledge learning oriented towards LLMs, especially focusing on how to “make good use of” large language models in organizing various learning activities. Modules 6 to 8 are advanced learning modules, focusing on enhancing students’ deep understanding and development capabilities of LLMs from a developer’s perspective. Taking Modules 4 and 5 as examples, Module A develops students’ foundational understanding of algorithms, while Module B relies on the foundational knowledge covered in Module A, placing more emphasis on enhancing students’ practical skills in building applications using various LLM APIs or technology frameworks. Another characteristic of the content framework lies in the special consideration of LLM application ethics. In response to the increasingly prominent issues of content bias, toxicity, and deep forgery associated with LLMs, two corresponding knowledge modules have been specifically set up, covering ethical issues at all learning stages from usage to development; furthermore, relevant ethical content is also integrated into other knowledge modules to ensure that ethical concerns are consistently addressed. It should be noted that these modules are not linear; specific content selections should be combined based on different educational stages and student characteristics. For example, for upper-grade primary school AI learning, focusing on experience is more suitable for Modules 1 and 3 (partially), emphasizing attempts to use simple prompt strategies and interact with various LLM chat applications, while verifying the accuracy of LLM-generated content through peer discussions to recognize the basic functions and limitations of LLMs. For junior high school students, the curriculum content should balance experience and practical development, possibly being more suitable for Modules 1, 2 (partially), 3, and 5 (partially). This would guide students to recognize the impact of LLMs on human life, learning, and future development while enabling them to utilize interactive learning resources to initially understand the basic principles of LLMs in text generation. For high school students with a solid programming foundation, after having studied “Introduction to Artificial Intelligence,” Modules 2, 6, 7, and 8 could form a more systematic content combination. Additionally, given that LLM knowledge is still evolving, the content framework should maintain a degree of openness, allowing teachers and curriculum developers to make appropriate adjustments and supplements based on the latest research findings to ensure the timeliness of the content framework.(2) LLM-Based Teaching Models The development of LLMs has brought new changes to the teaching models of artificial intelligence courses, as reflected in the previous overview, particularly in the exploration of the impact of LLMs on programming education. It is noteworthy that in the current information technology or technology curriculum systems of primary and secondary schools, modules related to artificial intelligence often center around Python teaching. Considering this situation, it is crucial and practically feasible to introduce LLM-based programming assistance tools into primary and secondary programming classrooms. At the same time, discussions on LLMs in the education sector reveal that the positive impacts of LLMs on AI education are not limited to programming teaching; they can also play a significant role in learning other knowledge areas, enhancing learners’ engagement and learning quality. In summary, to fully leverage the potential of LLMs, it is necessary to construct a new human-machine collaborative teaching model based on the characteristics of LLMs, embedding LLMs into key aspects of classroom teaching. The human-machine collaborative teaching model proposed in this research is illustrated in Figure 2. The core of this model is to embed LLMs as a core functional module of an auxiliary learning platform into the three key aspects of teaching, learning, and evaluation. For example, LLMs can be used to help students solve complex programming problems or serve as an interactive chatbot to assist students in mastering abstract conceptual knowledge, prompting strategies, and related practical tasks. The specific functions are as follows: Firstly, LLMs can act as proxies for teachers (AI Teachers). By utilizing LLMs to construct a virtual teacher role, they can deeply participate in practical teaching scenarios such as student-teacher communication, problem explanations, and error analysis. They can dynamically adjust teaching content and methods based on student feedback and data to meet the needs of different students. For human teachers, the virtual teacher role can better collaborate with real teachers to improve course quality: by sharing data and feedback with human teachers, LLMs can generate targeted teaching resources, including learning materials and assessment tools. This can help human teachers save time and focus on interacting with students, providing personalized support. Secondly, LLMs also support programming teaching in the form of programming learning plugins, helping students deeply understand programming concepts. With the powerful analysis and generation capabilities of LLMs, programming assistance plugins such as GitHub Copilot and Codeium can be applied in programming teaching environments to provide learners with instant feedback on code errors, automatically generate code snippets, and even offer algorithmic frameworks for complex problems. Building on this, students can be guided to generate, analyze, and optimize code through human-machine interaction, shifting the focus of learning towards evaluating algorithm structures and code styles, thereby training higher-level programming thinking. At the same time, LLMs can also assist in learning other knowledge areas in the form of AI study partners, such as continuously engaging in human-machine dialogue to help students train prompting skills. Evaluation is an important component of this human-machine collaborative teaching model. For instance, prompt text writing and program application development are essential components of the content framework, and evaluating the quality of prompt text and program code helps understand learners’ mastery of knowledge and skill development in the course. Given LLMs’ excellent performance in text analysis and code analysis, deploying platform-based teaching applications can record various data from the teaching process, enabling evaluations of learners’ course learning outcomes under the assistance of LLMs and assessments of the performance of virtual teachers and AI study partners in helping and guiding learners. In summary, this LLM-supported human-machine collaborative teaching model can support AI education from the dimensions of teaching, learning, and evaluation, forming a complete, mutually supportive system. In this system, LLMs serve not only as auxiliary tools but also as active participants, playing a role in all aspects of teaching. However, to enable educators to utilize LLM functionalities more flexibly and provide students with richer learning experiences, support from auxiliary learning platforms is essential, integrating learning resources, various experiments, assessment tools, and process data systematically.(3) LLM-Assisted Learning Platforms As previously mentioned, learning about large language models requires the use of various LLM learning platforms. These platforms represent a new learning model—learning about LLMs through LLMs themselves; in other words, using LLMs to learn and understand how they operate and are applied. Although many AI learning platforms based on LLMs exist that help learners experience, apply, understand, develop, and train LLMs, considering the characteristics of primary and secondary students and the rules of cognitive development, existing public LLM learning platforms are not entirely suitable for primary and secondary curricula. Moreover, ethical issues arising from LLMs (such as data privacy protection, data security, and deep forgery) cannot be entirely avoided on public platforms. Therefore, it is necessary to develop a large language model learning platform suitable for basic education AI curricula based on the characteristics of the curriculum content framework and teaching models. As shown in Figure 3, the development of such an LLM-assisted learning platform for youth education needs to incorporate four core functional modules: resources, experiments, evaluation, and data, to support the implementation of human-machine collaborative teaching activities in LLM courses: Firstly, the experiment module. The learning platform can call APIs of LLMs such as GPT-3.5-turbo through backend proxy services to achieve rich experimental functionalities, meeting various inquiry-based learning activities. By invoking these APIs, the learning platform can structure online experience features such as role-playing dialogues, real-time emotion recognition, text summarization, and writing essays in specified styles. These experience-oriented foundational functional modules help students establish preliminary recognition of LLMs, enhance their learning interest, and lay the groundwork for further teaching. Secondly, the resource module. The learning platform can provide students with rich and diverse learning resource packages. These resources include but are not limited to documents explaining the foundational principles of LLMs, instructional videos on prompt text writing, practical teaching videos on plugin development, pre-trained datasets for models, examples of prompts suitable for different downstream tasks, and professional domain corpora for fine-tuning demonstrations. These learning resources can enrich and extend the teaching content, helping students get started more quickly, experience advanced LLM usage techniques, and deepen their understanding of the operating principles of LLMs during practice. Thirdly, the evaluation module. The learning platform can integrate plugin services provided by LLM applications like GitHub Copilot and Codeium. By fully utilizing the functionalities of these collaborative programming plugins, it can transform the existing programming teaching methods in AI courses: transitioning from a focus on existing programming syntax knowledge and norms to a teaching pathway that integrates code generation, analysis, error correction, and optimization, further cultivating students’ abilities to discover and solve problems using LLMs, reinforcing computational thinking training, and enhancing digital learning and innovation capabilities. Additionally, by integrating these collaborative programming plugins with other modules, the platform can leverage the multi-modal data generated during the learning interactions to provide personalized feedback and suggestions for students; at the same time, teachers can adjust their teaching methods based on information that reflects students’ needs and progress. Finally, the data module. The LLM-assisted learning platform also serves as a learning analytics platform. It can save various data generated during learners’ interactions with LLMs, primarily including prompt text data and model output texts. By further analyzing prompt text data, personalized feedback can be provided to improve the quality and skills of students’ prompt writing. Furthermore, evaluations can be made by comparing outputs generated by different models, such as using the GPT-4 model to assess the text generated by the GPT-3.5-turbo model, allowing students to reflect on their learning status based on diverse evaluation information, further cultivating their information awareness and critical thinking. Overall, by designing and developing a supportive learning platform for LLMs, the aforementioned human-machine collaborative teaching model can be effectively implemented, further enhancing student engagement and participation in learning, and providing new possibilities for transforming AI education.(4) Teaching Activity Design Oriented Towards Core Subject Competencies In recent years, China has increasingly emphasized AI education at the basic education stage. The “Compulsory Education Information Technology Curriculum Standards” issued in 2022 clearly identifies core subject competencies, including information awareness, computational thinking, digital learning and innovation, and social responsibility, as the overall goals of information technology courses. The youth AI curriculum framework oriented towards large language models not only needs to focus on the timeliness of the curriculum but also must consider its scientific nature. Specifically, it is necessary to explore ways to design teaching activities under the guidance of core subject competencies, helping students master foundational knowledge and skills of LLMs while developing and nurturing core subject competencies. Overall, the alignment of the youth AI curriculum framework oriented towards LLMs with core subject competencies needs to be reflected in the design of teaching activities, including inquiry-based learning for theoretical content, in-depth practice based on projects, and attention to ethical issues and social responsibilities. Firstly, inquiry-based learning for theoretical content. Through interdisciplinary inquiry-based learning activities, students can deeply understand the implementation principles, training methods, and model architectures of LLMs. For example, teachers can design a debate activity on LLMs, where students can create a fine-tuned chatbot program to engage in debates with various styled virtual agents to understand how LLMs work, cultivating students’ computational thinking in interdisciplinary activity contexts. Another typical example is that teachers can have students input basic personal information and then let the LLM generate personal stories for each student, combining personal experiences to preliminarily assess the quality of LLM outputs and stimulate students’ reflections on the psychological feelings involved in inputting personal sensitive information, enhancing their sense of social responsibility regarding the use of LLMs. Secondly, in-depth practice based on projects. The rapid promotion of LLM applications provides a wide range of thematic references for practical activities in the curriculum system. Through these comprehensive practical project learnings, students will learn how to effectively utilize these technologies to solve real-world problems. For instance, by calling specific LLM APIs in a Python environment, teachers can guide students to create a conversational robot supporting internet information retrieval and encourage them to think about possible application scenarios. This deep programming practice can enhance students’ digital learning and innovation capabilities. Additionally, students can enhance their creativity by using LLMs for content creation and editing. For example, they can use the text generation capabilities of LLMs to assist in content creation and editing, or design automated programs to detect and correct errors in texts they have written. Finally, focusing on ethical issues and social responsibilities. The widespread application of LLMs has sparked deeper ethical concerns. Incorporating these contents into the curriculum will reinforce students’ social responsibility in discussing the societal impacts of LLMs. For example, prompting students to discuss a social event caused by the misuse of LLMs, such as cheating in exams, deep forgery, or over-reliance on LLMs, will help them develop ethical reasoning and critical thinking capabilities for responsible use of new technologies. Furthermore, through studying dedicated ethical content modules, students can understand the technical roots of issues such as biases, discrimination, hallucinations, and privacy concerns that LLMs may provoke, and through group activities and case studies, they can explore how to formulate corresponding usage strategies and norms to mitigate these issues. In summary, by designing a curriculum framework for large language models in youth AI education, constructing a content framework for youth LLM learning, exploring human-machine collaborative AI teaching models using LLMs, and developing corresponding auxiliary learning platforms, core subject competencies can be further implemented in various teaching activities, providing teachers and students with a more in-depth and participatory AI education environment.4. Explanatory Case: High School LLM Demonstration Course Based on the preliminary constructed youth AI education system oriented towards LLMs, the research team selected portions of knowledge modules from the content framework to attempt to construct a demonstration course suitable for ordinary high schools: “Fundamentals of Artificial Intelligence Oriented Towards Large Language Models.” On this basis, the unit content of “Prompts and Evaluation for LLM” is selected to demonstrate how to carry out human-machine collaborative teaching using the self-developed LLM auxiliary learning platform.(1) Course Content Considering that high school students already possess foundational programming and algorithm knowledge, this demonstration course will select content from Modules 2, 3, 4, 6, and 8 of the content framework, forming five teaching units. Overall, the course content design balances the use of LLMs and advanced development, with the following unit introductions: Unit 1 focuses on the principles of large language models. This unit emphasizes teaching LLM principles to help students deeply understand LLM technology, including its development history, trends, model architecture, and characteristics. Since this module involves some specialized AI knowledge, auxiliary teaching resources are needed to elaborate on these conceptual understandings. Specifically, when explaining the architecture of LLMs, a teaching activity designed as a text fill-in-the-blank game will be used to showcase the probability distributions of words generated by the model to students, allowing them to compare their predictions with those of the language model, thereby introducing relevant knowledge of model training methods. Additionally, the unit includes interactive learning resources that demonstrate training datasets of varying quality, sources, and scales, helping students explore how differences in training datasets can affect model output biases. Through this module, students will gain a basic understanding of how LLMs use the Transformer architecture and potential attention mechanisms to predict relationships between words in input texts; at the same time, relevant ethical knowledge, such as how training datasets can lead to biases in LLM outputs, will be integrated into the module. Unit 2 focuses on prompts and evaluation for LLMs. This unit centers on a key skill of LLMs: prompt engineering, including its basic concepts and applicable strategies for mainstream LLMs. The learning in this unit emphasizes practical operations, enabling students to fully understand the importance of prompt engineering in influencing the output quality of LLMs (such as GPT-3). More importantly, evaluation tools are embedded in the module to assess the quality of students’ prompts and corresponding LLM outputs during their practice, encouraging them to improve their prompting strategies and understand the limitations of LLMs. By comprehensively utilizing explanations of prompting strategies, training in practical projects, and evaluations of prompts, students will master the foundational knowledge of prompt engineering and gradually recognize its importance in various industry fields in the future. Units 3 and 4 focus on programming and project development. Unit 3 highlights the main knowledge of LLMs in assisting programming learning. The distinctive feature of this unit is that it teaches programming through human-machine collaboration using large language models: introducing how to use mainstream LLM plugins to assist in programming learning, including PEM interpretation, constructing code frameworks, and evaluating code quality, guiding students to gradually establish programming skills through various project developments. In this process, teachers also need to rethink the content and logic of programming teaching, for instance, by advancing core knowledge related to computational thinking, such as algorithms and data structures, and emphasizing the cultivation of students’ innovative thinking and computational capabilities through practical project exercises and collaborative teaching with LLMs. Correspondingly, Unit 4 places more emphasis on enhancing students’ development capabilities oriented towards LLMs. Some practical training designs include calling APIs of hosted or commercial large language models on platforms like Hugging Face for secondary development and utilizing technology frameworks like LangChain to build applications with more functions. Unit 5 aims to explore the social impacts and ethical issues arising from the rapid development of LLMs. By tracking negative social events caused by the misuse, misapplication, or malicious use of LLMs, this unit combines these events with the technical details and application methods of LLMs, guiding students to contemplate the ethical knowledge behind various social events. Students will delve into issues such as privacy breaches in training datasets, biases and toxic content in model outputs, and adverse consequences due to model hallucinations through a series of interactive learning resources. This unit will also extend the teaching content to the developer’s perspective, prompting students to understand common attack patterns in LLM applications, such as prompt injection attacks, thereby gradually establishing awareness of privacy protection mechanisms for applications and how to control biased content outputs from LLMs.(2) Design of LLM 4 Kids To effectively support the teaching of the demonstration course, a dedicated auxiliary learning platform—LLM 4 Kids—is required. The design of this platform considers the needs of human-machine collaborative teaching and aims to provide students with rich learning resources and interactive tools to support them in mastering the foundational principles and application skills of LLMs throughout the course. It should be noted that the LLM 4 Kids platform is still in the development stage, and this paper only presents the design and implementation ideas for the prompt engineering module that supports the unit of “Prompts and Evaluation for LLM.” Figure 4 illustrates the specific technical roadmap of the platform’s prompt engineering module: Firstly, the learning platform provides learners with access to LLM APIs that are functioning normally, such as gpt-3.5-turbo, through backend proxy services, enabling learners to interact directly with large language models for learning and practicing prompt strategies. For instance, students can complete practice activities using the least-to-most prompting strategy or solve more complex reasoning experimental tasks using chain-of-thought prompts. Secondly, this experimental environment can also track and record data from the teaching process, such as the prompt texts sent by learners to the large language model for specific tasks and the returned content, and conduct evaluations with the assistance of LLMs. For example, using the Perspective API (a toxicity detection API for generated texts) to assess whether the generated content contains bias or discriminatory elements; or employing the GPT-4 model to evaluate the quality of texts generated by the GPT-3.5-turbo model; and assessing the quality of prompt texts written by learners using LLMs. Additionally, the tracking of these process data requires integration with a pre-developed evaluation module, implementing lightweight functionalities to recognize learners’ emotions during the learning process, converting process data into SQL outputs, and structuring text data. Lastly, the prompt engineering experimental environment needs to be combined with course teaching resources. The platform obtains some electronic resources from open-source project management platforms, online websites, and electronic literature databases, such as foundational principle documents for LLMs, instructional videos for API calls, and pre-trained datasets for models. These resources are then filtered and rearranged to form materials applicable for prompt teaching.(3) Unit Demonstration Case: “Prompts and Evaluation for LLM” To further illustrate the changes in youth AI education in the context of LLMs, this research selects the unit content of “Prompts and Evaluation for LLM” from the demonstration course as a case to demonstrate how to carry out human-machine collaborative teaching using the prompt engineering experimental module of the LLM 4 Kids platform. It should be noted that the learning content generated around “prompts” occupies an important position in the knowledge system of large language models. As mentioned above, public LLM education has set up diverse courses around prompts, covering multiple scenario use cases; in the field of programming education, the accuracy of natural language prompts will also impact the effectiveness of the code generated or explained by LLMs; various platforms have emerged in LLM-assisted learning that focus on optimizing and debugging prompt texts. The importance of prompt learning in LLM education is reflected in several aspects. Firstly, the use of large language models relies on high-quality natural language prompts: LLMs can be used to solve a wide range of ad-hoc tasks through zero-shot prompting, but variations in wording and style of prompt statements can lead to differences in the accuracy of model outputs, and the public needs to learn how to utilize different prompting strategies to enhance the quality of interactions with LLMs. Secondly, prompt engineering also impacts the effectiveness of LLM application development. In practical application scenarios, a series of prompt statements forming a prompt template will interact with LLMs in a specific sequence, determining the interaction quality and stability of various automated applications. Finally, prompt learning is also key to understanding the ethics of LLM applications. Ethical risks arising from prompts, such as prompt injection attacks and deep forgery content production, have garnered significant attention from various sectors of society, making prompt learning an important content for understanding these risks and formulating response strategies. Based on this understanding, prompt learning needs to be emphasized in high school youth AI courses. After mastering prompting skills, students can not only learn how to effectively apply LLMs in different subjects and real-life situations but also establish a preliminary understanding of knowledge related to LLMs; simultaneously, prompt engineering involves complex problem-solving and innovative thinking, which helps cultivate students’ computational thinking abilities. By learning how to optimize and adjust prompts, students will learn methods for abstract thinking, logical reasoning, and problem-solving. Moreover, practical applications of prompt engineering, such as generating creative content and providing intelligent suggestions in scenarios closer to students’ lives, are more likely to engage students’ interests. Incorporating these engaging applications into teaching activities can stimulate students’ interest and participation in learning LLMs. Therefore, based on this, the teaching of “Prompts and Evaluation for LLM” can have several objectives, including helping students understand the basic concepts and terminology of prompts; through practical activities, enabling students to master the skills of optimizing and improving prompts; fostering students’ creativity and problem-solving abilities through teamwork; and using the evaluation functions of the learning platform to assess students’ learning outcomes and provide feedback. Based on this, the teaching can be divided into the following stages: Stage 1: Introduction of Basic Concepts. Through the course resource construction section of the platform, teachers can introduce students to the basic concepts of prompts, familiarizing them with related terminology such as “prompt,” “chain of thought,” and “zero-shot/few-shot.” Teachers can demonstrate a simple prompt, such as “Tell me some healthy fruits,” using the platform’s prompt engineering experimental environment, explaining how this prompt leads to LLM outputs like “apple, banana,” etc. Stage 2: Hands-on Practice Activities. These practice activities can be divided into three categories based on teaching objectives: analyzing prompts, optimizing prompts, and collaboratively designing prompts in groups. Firstly, students can analyze different prompts using the prompt engineering experimental environment, discussing their effectiveness in terms of the desired outputs from the LLM. For example, students can compare the effects of two prompts: “Tell me some healthy foods” and “List five fruits rich in vitamin C.” Secondly, teachers can introduce prompt templates applicable to LLMs, and students can optimize existing prompts using these templates. For instance, in the chain-of-thought prompt template, students can attempt to optimize the prompt “Tell me some healthy foods” to “Tell me some fruits that can help teenagers improve their immunity. Please think step by step.” Lastly, teachers can divide students into groups, assigning each group a specific task or domain, and each group will use the platform to create and optimize prompts related to their assigned tasks, such as one group focusing on healthy eating prompts while another group concentrates on environmental protection themes. Stage 3: Interaction and Evaluation. As illustrated in Figure 5, the prompt engineering module provides evaluation strategies for students during interactions. The platform allows students to interact with LLM applications (similar to ChatGPT) using their designed prompts in the prompt engineering experimental environment. During this process, the platform’s teaching evaluation section will collect data from students’ practice, assessing both the prompt texts and the model-generated responses, enabling students to adjust and improve their prompts based on the evaluation results. Stage 4: Reflection and Application. In this stage, students are encouraged to reflect on the knowledge and skills they have learned in the previous stages and discuss how to apply these to real-life problem-solving. During this process, teachers can organize a sharing session where students present their practical projects completed using prompting strategies and share the challenges they faced while optimizing prompts and strategies for overcoming those challenges. Teachers can guide students to think about how to apply the learned prompting techniques to achieve progress in their studies or use them to solve practical problems in their lives. The aforementioned teaching activities only outline the key stages of teaching. Through this structured teaching activity, students will gain a comprehensive understanding of prompt engineering through both theory and practice, thereby cultivating their critical thinking, creativity, and collaboration skills; simultaneously, the evaluation functions of the learning platform ensure that students receive valuable feedback regarding their progress and achievements. It is also important to note that this explanatory case exploring human-machine collaborative teaching in the context of LLMs is only a preliminary attempt. The presentation of teaching segments primarily focuses on students’ participation, while the changes in teachers’ requirements and capabilities have not been deeply considered. In practical applications, the role and capabilities of teachers are crucial. They need to have a certain technical background to understand and effectively teach the concepts of prompt engineering. Furthermore, teachers must adapt to the new human-machine collaborative teaching model and be able to handle complex issues related to large language models.5. Conclusion As the coupling between technological development and social impact becomes increasingly tight, factors such as technology, society, economy, and culture are interwoven, collectively influencing individuals’ understanding and application of technology. As Klaus Schwab emphasized in his “Fourth Industrial Revolution,” we are in an era of rapid transformation driven by artificial intelligence, machine learning, and other digital technologies. The rapid development of large language models confirms this: in the context where such advanced AI systems are quickly influencing all areas of human existence, to enable teenagers to quickly adapt to the ever-changing future survival world, substantive reforms in AI education at the basic education stage are required. To quickly respond to the impact of the development of large language models on AI education, the primary goal of this research is to develop a new curriculum framework oriented towards large language models. This curriculum framework includes a content framework, human-machine collaborative teaching, and an LLM-assisted learning platform, characterized by: (1) constructing a mutually interconnected and progressively layered LLM curriculum content framework, which involves eight core knowledge modules that can meet the learning needs of youth at different educational stages; (2) forming a human-machine collaborative teaching model that embeds LLMs into AI course teaching; (3) proposing an implementation idea for an LLM-assisted learning platform that includes four core functional modules: resources, experiments, evaluation, and data, based on the characteristics of the curriculum content framework and human-machine collaborative teaching. At the same time, guided by core subject competencies, this research also points out the alignment between the design of teaching activities and the curriculum framework. Another core goal of this research is to provide frontline teachers with reference guidelines for designing and implementing artificial intelligence courses oriented towards large language models. By showcasing a specific high school demonstration course and unit case, this research reveals how to carry out human-machine collaborative teaching activities using the self-developed LLM auxiliary learning platform “LLM 4 Kids.” However, it is worth noting that the course case developed in this paper is only a preliminary small-scale application. To comprehensively evaluate the effectiveness of this curriculum framework, it is necessary to conduct in-depth analyses of the implementation status across different educational stages and regions in future research, continuously assessing the applicability of the curriculum, the cultivation of students’ core competencies, and teachers’ adaptability. It should also be noted that this research provides design and implementation approaches for LLM-assisted development platforms but primarily focuses on the prompt engineering experimental module. Future research should further develop the “LLM 4 Kids” platform to effectively integrate AI programming assistance education modules, enabling students to utilize these technologies more effectively to solve complex programming problems while mastering LLM application methods.References:[1] OpenAI. GPT-4 Technical Report [EB/OL]. https://arxiv.org/abs/2303.08774, 2023-06-01. [2] Tom B. Brown, Benjamin Mann, et al. Language models are few-shot learners [J]. Advances in Neural Information Processing Systems, 2020, 33: 1877-1901. [3] Luciano Floridi, Massinmo Chiriatti. GPT-3: Its nature, scope, limits, and consequences [J]. Minds and Machines, 2020, 30(4): 681-694. [4][25] Paul Denny, James Prather, et al. Computing Education in the Era of Generative AI [EB/OL]. https://arxiv.org/abs/2306.02608, 2023-06-15. [5] Roberto Navigli, Simone Conia, et al. Biases in Large Language Models: Origins, Inventory and Discussion [J]. ACM Journal of Data and Information Quality, 2023, 15(2): 1-21. [6][19][24] Brett A. Becker, Paul Denny, et al. Programming is hard—or at least it used to be: Educational opportunities and challenges of AI code generation [A]. Maureen Doyle. Proceedings of the 54th ACM Technical Symposium on Computer Science Education [C]. New York: Association for Computing Machinery, 2023. 500-506. [7] U.S. National Science Foundation. Rapidly Accelerating Research in Artificial Intelligence [EB/OL]. https://new.nsf.gov/funding/opportunities/rapidly-accelerating-research-artificial, 2023-06-01. [8] AI Singapore. AI Singapore Educators [EB/OL]. https://learn.aisingapore.org/educators/#goes-to-school, 2023-06-01. [9] Xiong Zhang, Li Zhengfu. Research on the Path of Information Technology Curriculum Construction in the Compulsory Education Stage [J]. China Electric Education, 2023, (1): 127-132. [10] Primary and Secondary School AI Curriculum Guidelines Research Group, Jiang Bo. Guidelines for AI Curriculum in Primary and Secondary Schools [J]. Journal of East China Normal University (Education Science Edition), 2023, 41(3): 121-134. [11] Edx. Search Results for “chatgpt” [EB/OL]. https://www.edx.org/search?q=chatgpt&tab=course, 2023-06-01. [12] Coursera. prompt-engineering [EB/OL]. https://www.coursera.org/learn/prompt-engineering, 2023-06-01. [13][39] DeepLearning.AI. Short Courses [EB/OL]. https://www.deeplearning.ai/short-courses/, 2023-06-01. [14] Princeton University. COS 597G: Advanced Topics in Computer Science: Generative Models [EB/OL]. https://www.cs.princeton.edu/courses/archive/fall22/cos597G/, 2023-06-01. [15] Stanford University. CS224N: Natural Language Processing with Deep Learning [EB/OL]. https://web.stanford.edu/class/cs224n/index.html#schedule, 2023-06-01. [16] Carnegie Mellon University. Generative AI Events [EB/OL]. https://www.cs.cmu.edu/generative-ai/events, 2023-06-01. [17] ETH Zürich. LLM S23: Large Language Models [EB/OL]. https://rycolab.io/classes/llm-s23/, 2023-06-01. [18] Enkelejda Kasneci, Kathrin Seßler, et al. ChatGPT for good? On opportunities and challenges of large language models for education [EB/OL]. https://doi.org/10.1016/j.lindif.2023.102274, 2023-03-01. [20] Brett A. Becker, Keith Quille. 50 years of CS1 at SIGCSE: A review of the evolution of introductory programming education research [A]. Elizabeth K. Hawthorne. Proceedings of the 50th ACM technical symposium on computer science education [C]. New York: Association for Computing Machinery, 2019. 338-344. [21] James Finnie-Ansley, Paul Denny, et al. The robots are coming: Exploring the implications of OpenAI Codex on introductory programming [A]. Judy Sheard. Proceedings of the 24th Australasian Computing Education Conference [C]. New York: Association for Computing Machinery, 2022. 10-19. [22] Andreas Stefik, Susanna Siebert. An empirical investigation into programming language syntax [J]. ACM Transactions on Computing Education, 2013, 13(4): 1-40. [23] Yujia Li, David H. Choi, et al. Competition-level code generation with AlphaCode [J]. Science, 2022, 378(6624): 1092-1097. [26] James Prather, Brent N. Reeves, et al. “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers [J]. ACM Transactions on Computer-Human Interaction, 2023, 31(1): 1-31. [27] Sami Sarsa, Paul Denny, et al. Automatic generation of programming exercises and code explanations using large language models [A]. Jan Vahrenhold. Proceedings of the 2022 ACM Conference on International Computing Education Research [C]. New York: Association for Computing Machinery, 2022. 27-43. [28] Stephen MacNeil, Andrew Tran, et al. Generating diverse code explanations using the GPT-3 large language model [A]. Jan Vahrenhold. Proceedings of the 2022 ACM Conference on International Computing Education Research [C]. New York: Association for Computing Machinery, 2022. 37-39. [29] Edwin M. Knorr. Reforming a Database Course to Address Cognitive Load by Using Worked Examples [A]. Ben Stephenson. Proceedings of the Western Canadian Conference on Computing Education [C]. New York: Association for Computing Machinery, 2019. 1-6. [30] Anna Eckerdal, Michael Thuné. Novice Java programmers’ conceptions of “object” and “class”, and variation theory [A]. Jose Cunha. Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education [C]. New York: Association for Computing Machinery, 2005. 89-93. [31] Roberto Minelli, Andrea Mocci, et al. I know what you did last summer-an investigation of how developers spend their time [A]. Andrea D. Lucia. 2015 IEEE 23rd international conference on program comprehension [C]. Los Alamitos: Wiley-IEEE Computer Society Press, 2015. 25-35. [32] Errol Thompson, Jacqueline L. Whalley, et al. Code classification as a learning and assessment exercise for novice programmers [EB/OL]. https://cseweb.ucsd.edu/~bsimon/pubs/papers/NACCQQ11analysisPaperFinal.pdf, 2023-06-01. [33] Editorial Board, Zhang Rong. The Impact of Generative Artificial Intelligence Technology on Education—An Interview about ChatGPT [J]. Electric Education Research, 2023, 44(2): 5-14. [34] OpenAI. GPT Playground [EB/OL]. https://platform.openai.com/playground, 2023-06-01. [35] Hendrik Strobelt, Albert Webson, et al. Interactive and visual prompt engineering for ad-hoc task adaptation with large language models [J]. IEEE Transactions on Visualization and Computer Graphics, 2022, 29(1): 1146-1156. [36] Shizhe Diao, Pengcheng Wang, et al. Active prompting with chain-of-thought for large language models [EB/OL]. https://arxiv.org/abs/2302.12246, 2023-06-15. [37] Jiacheng Liu, Alisa Liu, et al. Generated knowledge prompting for commonsense reasoning [EB/OL]. https://arxiv.org/abs/2110.08387, 2023-06-15. [38] Denny Zhou, Nathanael Schärli, et al. Least-to-most prompting enables complex reasoning in large language models [EB/OL]. https://arxiv.org/abs/2205.10625, 2023-06-15. [40] Langflow. Project Langflow [EB/OL]. https://pypi.org/project/langflow/, 2023-06-01. [41] Flowise AI. Flowise AI [EB/OL]. https://flowiseai.com/, 2023-06-01. [42] DUST. Document Understanding and Synthesis Toolkit [EB/OL]. https://dust.tt/, 2023-06-01. [43] Hugging Face. Hugging Face [EB/OL]. https://huggingface.co/, 2023-06-01. [44] H2O.ai. H2O Documentation [EB/OL]. https://docs.h2o.ai/, 2023-06-01. [45] Tianjia Wang, Daniel Vargas-Diaz, et al. Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives [EB/OL]. https://arxiv.org/abs/2306.03289, 2023-06-15. [46] Xiong Zhang, Zhao Jian, et al. The Timeliness and Scientific Nature of Information Technology Curriculum in the Compulsory Education Stage—Interpretation of the “Compulsory Education Information Technology Curriculum Standards (2022 Edition)” [J]. Journal of Teacher Education, 2022, 9(4): 63-69. [47] Liang Yunzhen, Liu Ruixing, et al. Interdisciplinary Integration Teaching of “Artificial Intelligence + X” in Primary and Secondary Schools: Theoretical Framework and Practical Strategies [J]. Electric Education Research, 2022, 43(10): 94-101. [48] Su Qing, Zhang Wenlan, et al. Research on Project-Based Learning in Information Technology Courses for Cultivating Computational Thinking for High School Students [J]. Electric Education Research, 2022, 43(8): 109-115+122. [50] Pengfei Liu, Weizhe Yuan, et al. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing [J]. ACM Computing Surveys, 2023, 55(9): 1-35. [51] Fabio Perez, Ian Ribeiro. Ignore previous prompt: Attack techniques for language models [EB/OL]. https://arxiv.org/abs/2211.09527, 2023-06-15. [52] [Switzerland] Klaus Schwab. Li Jing Trans. The Fourth Industrial Revolution: Forces of Transformation [M]. Beijing: CITIC Press, 2016.Author Information: Chu Leyang: PhD candidate, research interests include AI education, computer-supported collaborative learning, and shared regulation learning theory. Wang Hao: Master’s candidate, research interests include AI education, computer-supported collaborative learning, and shared regulation learning theory. Chen Xiangdong: Professor, PhD, Vice Dean, research interests include AI education, computer-supported collaborative learning, and shared regulation learning theory.

END·Produced by: Jiao YangProofread by: Zhao YunjianReviewed by: Song Lingqing