Author | Zhi Zhenfeng, Researcher at the Institute of Law, Chinese Academy of Social Sciences.
Source | Political and Legal Forum, 2023, Issue 4. For reading convenience, annotations are omitted; it is recommended to read the original text.
Abstract：Based on large computing power and strong algorithms to process massive amounts of data, generative AI large models have shown excellent performance in natural language processing, computer vision, and speech processing. They can now provide services such as content creation, digital humans, conversational search, and code generation, with great application prospects in fields such as autonomous driving, financial risk control, healthcare, and the Internet of Things. As a significant transformation in internet information technology, the logical reasoning ability of large models and their “understanding ability” of humans have greatly improved, making them a powerful tool for humans to produce creative information content. However, they may also significantly change the ecological environment of online information content, leading to the proliferation of low-quality information, contamination of initial information sources, and challenges to social ethics. It is necessary to balance development and security and explore governance paths that are compatible with incentives.
Keywords：Generative AI; Large Language Models; Information Content; Incentive Compatibility; Governance
Enhancing the integration of human knowledge and understanding of intentions, expanding human intellectual boundaries, and achieving smoother human-computer interaction has always been an important direction for information technology efforts. With the explosive popularity of the Chat Generative Pre-trained Transformer (ChatGPT) launched by the American AI research company OpenAI, many technology giants have continuously increased their investment in the generative AI race. After the chatbot Bard, Google released the multimodal “Second Generation Channel Language Model” (PaLM 2) that can “understand” and generate audio and video content. Microsoft’s New Bing search engine has integrated the multimodal “Generative Pre-trained Transformer 4” (GPT-4), and Amazon has also announced its entry into the race by releasing Titan. Chinese large models such as Baidu’s “Wenxin Yiyan”, Huawei’s “Pangu”, Tencent’s “Hunyuan Assistant”, Alibaba’s “Tongyi Qianwen”, SenseTime’s “Riri Xin”, Kunlun Wanwei’s “Tiangong”, and iFLYTEK’s “Xinghuo Cognition” are continuously emerging. Various generative AI large language models (LLMs) have experienced explosive growth, and the technological application boom has swept the globe.
Based on large computing power and strong algorithms to process massive amounts of data, AI large models trained on large-scale unlabelled data learn certain features or patterns to predict future outcomes. The number of parameters has increased from hundreds of millions to hundreds of billions, achieving a leap from supporting single tasks under single modalities such as images, text, and speech to supporting multiple tasks across various modalities, thus becoming a model library with generalization and certain general capabilities. The large model “achieves miracles through great strength”; it has shown excellent performance in natural language processing, computer vision, and speech processing, and is already able to provide content creation, digital humans, conversational search, and code generation services, with great prospects in autonomous driving, financial risk control, healthcare, and the Internet of Things.
The large model already has the capability to serve “thousands of industries.” However, as a significant transformation in internet information technology, the logical reasoning ability of large models and their “understanding ability” of humans have greatly improved, bringing revolutionary changes to the generation of information content in text, images, speech, and video. This will truly bring the production and dissemination of information content into a new era of artificial intelligence-generated content (AIGC), triggering a knowledge revolution in human society. By learning the characteristics of objects from massive data, AI content creation no longer simply compares and matches but attempts to understand people’s thoughts, utilizing existing text, images, or audio files to generate content based on large datasets. This will not only become a powerful tool for humans to produce creative information content but may also significantly change the ecological environment of online information content, bringing new risks and challenges to information governance.
Due to the high technical specialization of generative AI large models in information content production and dissemination, which has far removed them from people’s existing common sense, this article will primarily outline the typical functions, application scenarios, and important features of AI large models in content generation as concisely as possible in the first part. On this basis, the second part of the article will argue that due to the significant impact of large models and their inherent limitations that are difficult to overcome, they may pose significant risks to information content governance. Furthermore, after briefly summarizing domestic and international information content governance, the article will attempt to propose governance paths for generative AI information content.
1. Generative AI Opens a New Era of Information Content Production
and Dissemination
Language holds a special significance for humanity. Heidegger proposed, “Language is the house of being”; Wittgenstein stated, “The limits of my language mean the limits of my world.” Throughout the development of AI technology, natural language processing has always been hailed as the “jewel in the crown of artificial intelligence.” How to enable computers to understand and process human language is a crucial point in human-computer interaction. The natural language processing framework adopted by generative AI large language models has made significant progress in dialogue and content generation capabilities, allowing them to learn and train on large text datasets to produce complex and intelligent writing, which can even be transformed into images or videos.
(1) Revolutionary Changes in Information Content Production and Dissemination
The history of humanity is a history of information production, exchange, and dissemination. From the oral tradition of primitive society, the bamboo slips and silk of agricultural society, to the broadcasting and television of the industrial age, and then to the development of the internet, especially mobile communication technology, the production and dissemination of human information content have mainly unfolded in two modes: user-generated content (UGC) and professionally generated content (PGC). Before the internet era, whether it was bamboo slips, books, newspapers, or broadcasting and television, the most widely disseminated and enduring content was mainly professionally generated content, produced primarily by intellectuals, officials, and certain professionals in various fields. In the mass media era, content producers and gatekeepers such as journalists and editors also emerged. Overall, professionally generated content tends to be more authoritative, reliable, and of better quality. In contrast, content passed through oral tradition and street gossip is mainly user-generated, where producers may not be professionals, and there generally won’t be quality gatekeepers. The so-called “rumors” are mostly “self-produced and self-sold,” flourishing quickly but also fading fast. However, in the internet era, especially with the widespread application of social media technology, everyone has a microphone, everyone has a camera, and the “street gossip” in cyberspace can also be widely disseminated and long-lasting, with short videos giving everyone the opportunity to be “seen.” In cyberspace, user-generated content naturally dominates in quantity. On the WeChat platform alone, there are hundreds of millions of audio and video calls and hundreds of billions of messages sent daily. As of the end of 2022, the number of internet video users (including short videos) in China reached 1.031 billion, and the number of live-streaming users reached 751 million. The production and dissemination of information content in human society have achieved a revolutionary shift from a focus on professional production to a focus on user production.
The emergence of generative AI large language models has opened a new era of AI-generated content, marking another revolutionary change in the production and dissemination of human information content. The main body of information content production has undergone tremendous change, with artificial intelligence capable of replacing human labor in the entire process of information collection, filtering, integration, and reasoning, greatly liberating human resources. The efficiency of information content production has undergone disruptive changes, with large computing power driving strong algorithms to process large data sets. In natural language processing tasks such as text classification, sentiment analysis, machine translation, question-answering systems, and text generation, as well as in computer vision tasks such as image classification, object detection, image segmentation, and facial recognition, and in autonomous driving tasks such as vehicle control, road recognition, and traffic flow prediction, as well as in financial risk control tasks such as fraud detection, risk assessment, and market prediction, and in healthcare tasks such as disease diagnosis, pathology analysis, and medical image analysis, and in various tasks in the Internet of Things such as smart homes, smart manufacturing, and environmental monitoring, high-quality judgments and efficient content generation can be achieved. The dissemination of information content has undergone disruptive changes, making the production and dissemination of information more convenient, especially lowering the threshold for acquiring professional knowledge. The forms of information content have become more diverse, utilizing AI generative technology, allowing for more free conversion between images, text, and code, enabling the one-click generation of “digital human” avatars, and “opening the era of intelligent interconnection.”
(2) Content Generation Functions of Large Models
Large models already possess the capability for multimodal and cross-modal information content production. Currently, looking at the large models released domestically and internationally, the core architecture for information content generation is mainly based on natural language processing, using transformers as universal modules/interfaces, relying on deep learning models with self-attention mechanisms to generate text or image content similar to human creation. GPT-4, through pre-training on multimodal corpora that include various data such as text data and any interleaved images and text, enables the model to acquire native support for multimodal tasks.
Based on reinforcement learning from human feedback (RLHF) technology, large language models like ChatGPT can learn and improve output content based on user input. They can also achieve alignment between the model’s expressions and intrinsic values with human common sense and values. ChatGPT can also apply instruction tuning technology to better adapt to users’ language habits and communication styles, understanding user questions and enhancing the system’s adaptability and performance in specific tasks and scenarios.
In terms of output forms of information content, generative AI large models can now achieve various modalities including text, images, video, audio, digital humans, and 3D content. For example, SenseTime’s “Riri Xin” large model series includes “Miao Hua SenseMirage,” a text-to-image creation platform that can generate images with realistic lighting, rich details, and diverse styles, capable of producing 6K high-definition images. “Shangliang SenseChat” is an efficient chat assistant that can quickly solve complex problems, provide customized suggestions, and assist in creating top-quality text, exhibiting continuous learning and evolution capabilities. “Mingmou” is a data labeling platform that houses more than ten general large models and industry-specific models, supporting intelligent labeling for various scenarios in smart driving, smart traffic, and smart cities through 2D classification, detection, and 3D detection. “Ruying SenseAvatar” is an AI digital human video generation platform that can generate natural-sounding and accurately lip-synced digital human avatars proficient in multiple languages from just five minutes of real human video material. Scene generation platforms like “Qiongyu” and object generation platforms like “Gewu” are 3D content generation platforms that can efficiently and cost-effectively generate large-scale three-dimensional scenes and detailed objects, opening up new imaginative spaces for the metaverse and the integration of virtual and real applications.
Generative AI large models are ushering in an era of “Model as a Service” (MaaS). Tech giants are developing universal models and providing them to B-end clients in niche sectors (Tencent Hunyuan also targets G-end clients), allowing clients to refine the models, thus empowering various industries. Simultaneously, public testing or paid usage interfaces are opened for C-end users, attracting players with a deep understanding of the industry to refine and train the models.
Generative AI interacts deeply with users to generate massive amounts of information, providing users with convenient conditions for information search, product consumption, and participation in public life. ChatGPT-3 has 175 billion parameters and has been trained on approximately 500 billion pieces of text collected from the internet, with massive data and powerful computing capabilities creating this highly intelligent publicly available AI. GPT-4 has further improved performance compared to other generative large language models, representing a significant step toward general artificial intelligence (AGI). Its general capabilities can be widely applied in various scenarios in abstraction, understanding, vision, coding, as well as in fields such as mathematics, medicine, law, and understanding human motivations and emotions, with performances in certain tasks reaching or exceeding human levels.
(3) Application Scenarios of AI Large Models
Generative AI can serve as a conversational companion for humans, supporting the model to produce fluent, contextually appropriate, and somewhat knowledgeable dialogue content through pre-training technology, presenting a certain degree of “personality” rather than stiff machine language, thus having the potential to become a virtual companion robot. In specific domains, by learning specialized knowledge and utilizing tuning technology, the large model can take on the role of intelligent customer service. In search services, large models will better understand human intentions, directly generating the “answers” users want, rather than merely providing a series of webpage links.
The most typical application of large models is writing generation. Based on themes and keyword requirements, generative AI can “write” stories, novels, poems, letters, news reports, current event commentaries, and outlines for academic papers; it can also modify and polish text, such as correcting grammar, translating texts, and extracting keywords. The large model can also write code; according to OpenAI’s technical developers, through training large language models, it can generate functionally correct code bodies from natural language document strings. There have been instances where users have used ChatGPT to quickly draft an apology letter for a company involved in the “ice cream incident” at the 2023 Shanghai Auto Show, which was faster and more appropriately worded than the company’s public relations statement, leading netizens to criticize the company for having a lower public relations level than ChatGPT. GPT-4 can also recognize content from images and can even understand images that have specific connotations, known as “meme images.” The “Miao Hua” in SenseTime’s “Riri Xin” large model series, as well as Stable Diffusion and Midjourney, can generate highly creative images using text prompts.
Generative AI large models have begun to exhibit capabilities of “experts” in various fields, engaging in a certain level of basic medical consultation, legal services, educational Q&A, and other professional knowledge inquiries and analyses. For example, SenseTime’s “Riri Xin” can “explain” patent law; GPT-4 can answer exam questions, analyze charts, and can now guide and encourage users to think and arrive at answers like a real human teacher. ChatGPT can assist legal professionals in brainstorming, improving case analysis and document writing, and organizing citations, among other tasks.
Generative AI large models can serve as clever personal assistants. In daily life, ChatGPT can help order restaurant meals, book movie tickets, plan trips, and provide weather forecasts, and can also recommend relevant news, music, movies, and books based on user interests. It can customize travel routes, schedules, and reminders based on user preferences, work hours, and locations. For instance, after integrating Alibaba’s large model Tongyi Qianwen, the application “DingTalk” can comprehensively assist in office tasks, capable of creating poetry and novels, writing emails, and generating marketing plans; during DingTalk meetings, it can generate meeting minutes and automatically summarize meeting notes and generate to-do items at any time.
Generative AI also has great potential in product design, deep synthesis, and manufacturing. In numerous scenarios such as logo design, clothing design, internet content illustration, and e-commerce images, the text-to-image and image-to-text creative content generation capabilities can be fully utilized. The large model can also generate marketing plans based on descriptive text, generate application mini-programs based on functional sketches, and connect various fields and industries in education, smart business, and smart cities, creating a closed loop of applications. Additionally, through connecting smart living, AI face-swapping based on text descriptions, and other deep synthesis functions, it can also generate digital human avatars. When applied to 3D printing, it can directly manufacture industrial products.
(4) Features of AI Information Content Generation
In addition to the internet information production models of professionally generated content (PGC), user-generated content (UGC), and mixed generation, the influence of AI-generated content (AIGC) mode is becoming increasingly significant, bringing about the evolution of content production subjects and methods, improvements in content interaction and distribution methods, and enhancements in content production quality and generation effects. AI-generated content exhibits several remarkably revolutionary characteristics.
The acquisition of information content has shifted from display to generation. AI large models can effectively summarize and condense existing human knowledge, providing efficient output based on massive data, greatly enhancing the ability of humans to produce and acquire information. It can draft or compose texts, replacing a portion of human labor. It has changed the way knowledge is generated and transmitted, significantly lowering the threshold for acquiring professional knowledge, making it no longer necessary to undergo decades of professional training for professional knowledge generation. Compared to the AI tools previously used internally by media organizations, this generation of generative AI applications is open to all users, bringing the possibility of self-release and self-creation, leading to a form of information that is inclusive and narrowing the social knowledge gap.
The provision of information content has shifted from decentralization to integration. Before AI large models, the information people accessed on the internet mainly came from various scattered webpages, knowledge communities, and online encyclopedias. However, generative AI has accomplished the integration of massive public knowledge through information consolidation and data analysis, and can engage in dialogue interaction, thus integrating functions of search engines, knowledge platforms, knowledge communities, software open-source communities, and some social media, providing concise and efficient output based on its inherited massive knowledge, greatly enhancing human access to information. To some extent, large models integrate the processes of information search, retrieval, integration, and preliminary output, facilitating the transmission, dissemination, and inheritance of knowledge.
The service scenarios have transformed from single domains to general applicability. Generative AI large language models possess better generalization, accuracy, and efficiency, capable of learning through pre-training or other methods on large datasets and efficiently processing complex tasks in computer vision, natural language processing, and more through fine-tuning. Large language models have been trained on vast amounts of data covering various thematic fields, allowing them to mimic human intelligence in a broader range of applications. In the process of implementing “Model as a Service,” as a foundational model at the code level, generative AI large language models possess the capability to become the new generation of infrastructure, applicable to various downstream scenarios from search engines and content platforms to application software, affecting various industries including daily work, scientific research, education, and even public services. Developers of foundational models thus become the “gatekeepers” of the digital technology market, wielding significant market power. This marks a truly epoch-making product in the history of AI development: if AlphaGo marks the achievement and surpassing of human capabilities in specialized fields by narrow AI, then ChatGPT opens the era of general AI—where AI possesses broad learning capabilities and reaches or exceeds the capabilities of ordinary humans in most domains.
The dialogue method has transformed from one-way retrieval to intelligent interaction. How to make computers no longer cold machines, how to enhance computers’ understanding of humans, and how to make it easier for humans to access information are all significant driving forces for the development of information technology. Before generative AI large language models, humans accessed knowledge and information either through face-to-face communication, querying book materials, or using internet search engines. The methods of information acquisition were one-way and monotonous. Besides human-to-human communication, the relationship between people and books, materials, and computer networks was a cold “subject-object” relationship. However, generative AI large language models have greatly changed the way humans acquire knowledge and information. Taking ChatGPT as an example, through a generative pre-trained model based on massive data, trained on a large amount of internet text, it can understand and answer questions on various topics, using a human-like rather than machine-like language system for natural language expression. ChatGPT-3 already exhibits significant contextual learning capabilities, capable of predicting contextual vocabulary, learning or mimicking patterns in data, and outputting corresponding answers based on key information matching and pattern imitation. As the number of model parameters increases, the contextual learning ability continues to strengthen, ensuring continuity in human-computer dialogue, and proactively asking users when it fails to understand commands. This adds a layer of “personalized” communication to the process of humans obtaining information through large models, making computer information retrieval no longer a cold machine operation but potentially a smart interaction with “human warmth.”
2. Generative AI Brings New Challenges to Information Content Governance
In a certain sense, generative AI large models are becoming the embodiment of human information content production and dissemination. Information content carriers such as books, newspapers, broadcasting, and television, information provision tools such as news media, search engines, knowledge communities, online encyclopedias, and open-source communities, as well as specific professional identities such as customer service, writers, doctors, teachers, and experts, all converge into generative AI large models. Large models have become textbooks and sources of knowledge, acting as “teaching masters” and “authoritative figures,” capable of monopolizing knowledge from the source, influencing judgment, and shaping cognition. Large language models possess the potential to penetrate various fields of human production and life, but the limitations of the technology itself and the issue of technological abuse will pose severe challenges to information content governance.
(1) Technical Limitations
There are flaws and limitations in training data. The astronomical data required for pre-training large models cannot all be verified for accuracy, and if the data is inaccurate or missing, it will inevitably affect the reliability of the results, leading to “garbage in, garbage out.” If the data contains biases or sensitive information, it may also lead to the generation of results that exhibit discrimination and erroneous cognition. In 2017, research demonstrated biases and stereotypes in natural language processing data by analyzing the Stanford Natural Language Inference (SNLI) corpus. In the absence of internet access or plugins, the knowledge of large models is often time-limited; for example, GPT-3.5’s knowledge is limited to events that occurred before 2021; Google’s Bard claims to be able to search for information online but still has a certain time lag. They face issues of limited computing power, insufficient training, and high research and operational costs. Training large models is akin to a violent aesthetic, requiring large computing power, large data, and large models, with each training task incurring enormous costs. Reports from SenseTime indicate that running ChatGPT on the cloud computing end requires at least 10,000 A100 chips, while currently, only a few companies in China, such as SenseTime, Baidu, Tencent, ByteDance, Alibaba, and Huanfang, have more than 10,000 chips in reserve, highlighting a significant computing power gap and high costs.
Content generation has its limits. High-probability combinations do not necessarily reflect reality and are difficult to be creative. AI models like ChatGPT can only react based on the information they have been trained on; they cannot truly access real-time facts or understand context. First, AI content generation remains knowledge reorganization rather than knowledge production or reproduction. On one hand, there is still a gap compared to human intelligence, with limited contextual understanding and a lack of “human warmth”; it can pursue short-term, large-scale outputs but cannot produce meaningful innovative content. The answers generated by the model are produced by its pre-trained neural network, where the parameters are randomly initialized, and during training, they are optimized through random gradient descent based on input data, leading the model to provide different or even contradictory answers to the same question. Sometimes the answers may seem “authoritative” while at other times may be “nonsensical”; when questioned, they may “adapt” or “refuse to acknowledge” this is fundamentally due to the probabilistic and unpredictable nature of its output results, which are randomly selected from multiple alternatives. On the other hand, the quality of the output content largely depends on the user’s questioning (Prompt) ability. In the process of natural language processing for professional domain information, there exists a contradiction between generalization and specialization, making it challenging to ensure readability while maintaining professionalism. Second, there is a common issue of “hallucination,” where content may appear “correct” but is essentially erroneous. The inevitable biases arising from the compression of information in the model’s training set lead to outputs that include some information that does not align with the input, which may be erroneous, irrelevant, or absurd, creating semantic expansions or unrelated scenarios that cannot be avoided. Although AI large models exhibit a personalized appearance, they still cannot genuinely possess personality. In digital systems, AI lacks any notion of humanity, inevitably leading to the occurrence of “hallucinations” as “confident responses.” Third, there are cross-language and cross-cultural challenges; the collection of multilingual corpora may not fully grasp the underlying connotations of the corpora. In the training dataset released by OpenAI for GPT-3, English corpus accounts for as much as 92.65%, while French, the second most prevalent language, only accounts for 1.92%. The input of corpus largely determines the output of results. The limited use of Chinese corpus in large model training will not only significantly impact the quality of generated content but also greatly affect the Chinese civilization, which primarily utilizes the Chinese language as a means of expression.
Content review presents complexities that are difficult to control. Due to inherent algorithmic black box issues and interpretability defects, it is challenging to understand the reasoning behind model predictions. ChatGPT also states on its website that the sheer volume of content generated by these models makes manual review and auditing of generated content very difficult. According to OpenAI’s papers, despite GPT-4 having the same technical limitations, it appears “more persuasive and credible than earlier GPT models.” This will lead to greater problems. When users overly depend on it, they may likely become complacent or overlook errors during its use.
(2) Risks Associated with the Application of Generative Large Language Models
Due to the vast amounts of training data required for generative AI large models, as well as their generative, prioritizing, integrative, and generalizable characteristics, they may also give rise to various significant risks while empowering numerous industries.
1. Risk of Personal Information Leakage
The process of users conversing with generative AI large language models involves the widespread collection of personal information. When users pose questions, they may inadvertently expose personal information they do not wish to disclose. However, according to OpenAI’s explanation, users can only delete their personal accounts but cannot delete sensitive personal information. On March 20, ChatGPT’s open-source library experienced a vulnerability that allowed some users to see the conversation content, names, email addresses, and even payment information of other users. OpenAI had to remind users on its official website: “Please do not share any sensitive information in conversations.” In reality, the information users inadvertently provide when requesting responses or tasks from generative AI may be used in the model’s training, learning, and improvement processes, thus entering the public domain. This not only risks infringing on user privacy but may also expose others’ information. For instance, when lawyers use it to review drafted divorce agreements, it may leak the personal information of the parties involved in the case. Particularly, large models demonstrate strong reasoning capabilities, enabling them to write programs based on user needs, which can improve user experience but may also lead to personal information leakage risks.
2. Risk of Commercial Secret Leakage
Reports have indicated that Samsung’s semiconductor division has experienced three incidents of commercial secret leakage due to the use of ChatGPT: one employee requested it to check for errors in sensitive database source code, another used it for code optimization, and another input recorded meeting content into ChatGPT and requested it to generate meeting minutes. Whether for market entities, academic institutions, or government agencies, there is an inevitable need to share certain information with large models, resulting in significant risks of leaking commercial secrets or even state secrets.
3. Data Security Risks
The data used for training may be inaccurate or biased, and the quality of the data is not guaranteed, making it difficult to ensure legality, leading to the possibility of generating “toxic” content. As more industries and fields adopt generative AI large language models, data leakage and compliance risks are becoming increasingly prominent. Once data, which serves as a production factor, is leaked, it will lead to substantial economic and reputational losses for enterprises and industries. Even fragmented or piecemeal information may be analyzed and combined by ChatGPT to infer intelligence information related to national security, public safety, and the legitimate rights and interests of individuals and organizations. Particularly for models like ChatGPT and Bard, which are hosted on overseas servers, inputting sensitive data during use may trigger security issues related to cross-border data flow, posing risks to data security and even national security.
4. Cybersecurity Risks
Due to the lowered professional knowledge threshold and the model’s inability to discern users’ purposes, generative AI may provide convenient tools for cybercrime. By writing code for cyberattacks, it can generate code in various languages such as Python and JavaScript, creating malware to detect sensitive user data, and can even infiltrate target computer systems or email accounts to obtain important information. Experts have detailed how to use ChatGPT to create polymorphic malware that bypasses content policy filters established by OpenAI, generating malicious code. Criminals can simply request the model to write marketing emails, shopping notifications, or software updates in English using their native language, making it easy to craft phishing scripts that show few signs of spelling or grammatical errors, making them difficult to identify as fraudulent information or phishing emails. Furthermore, the information used during the training process for account information may be shared with service providers and related companies, potentially leading to data leakage risks and leaving vulnerabilities for cyberattacks.
5. Algorithm Risks
Generative AI essentially processes massive amounts of data using algorithms, which are key to its function. However, since algorithms themselves cannot verify training data, they often generate seemingly accurate but fundamentally erroneous misleading content, resulting in “hallucinations.” The accuracy of the content generated by the model is limited, and the model itself cannot discern the authenticity of the written content, leading to the generation and dissemination of false information. Moreover, algorithms cannot avoid societal biases and value judgments. Algorithms with inherent problems may be guided to generate content that violates laws and regulations. The value judgments in data usage and algorithm training may also produce “toxic” content, solidifying social biases and discrimination, including biases based on race, gender, beliefs, political stance, and social status.
(3) New Challenges in Information Content Governance
Generative AI large language models have the potential to replace humans in the entire thought process of information collection, knowledge acquisition, content evaluation, and reasoning. Particularly, large models possess advantages in natural language processing and computer vision, and in generating graphic and textual content and engaging in human-computer dialogue, their lowered costs of information production, reduced thresholds for professional knowledge, aggregated application functions, and broader usage fields may generate significant risks related to information content.
1. Generativity Leads to the Proliferation of Low-Quality Information
Generative AI can draft or compose texts, replacing a portion of human labor, leading to negligible production costs and a surge in the quantity of texts. This massive increase in content not only puts pressure on the available physical memory storage space, causing an explosive growth of information but, more importantly, will lead to the rapid expansion and widespread dissemination of harmful or undesirable content.
First, false information worsens the online ecosystem. Generative AI may fabricate false information, producing low-quality content that intermingles true and false information, and elaborating on fabricated false facts with fluent sentences, thus being somewhat misleading to groups with limited information sources. “Automated bias” leads users to tend to believe the answers outputted by seemingly neutral models. If the powerful content creation capabilities of generative AI are used to generate false information targeting individuals or enterprises, it will lead to rumors, slander, insults, and defamation; particularly, utilizing deep synthesis technology to generate false statements, images, or videos of political figures or key individuals may provoke social unrest and result in greater harmful consequences.
Second, misleading information interferes with personal decision-making and daily life. Generative AI increasingly exhibits the appearance of “knowledge authority”; producing erroneous or misleading content in various professional consulting services such as event planning, legal services, and healthcare will directly affect users’ daily lives. When applied to event planning, due to the presence of “hallucinations,” limited accuracy, and limited contextual understanding, it is prone to produce “nonsense”, resulting in erroneous itineraries and schedules. When applied to healthcare and legal services, if the knowledge answers are incorrect, the generated information may mislead users and interfere with their medical consultations or legal proceedings.
2. Pollution of Initial Information Sources
Traditional sources of knowledge such as textbooks and news media are increasingly being replaced by online platforms. As a model that integrates knowledge platform, search platform, and generation platform functions, it may become a monopolistic source of knowledge, potentially generating source pollution from the outset. If information content is created without human supervision, the capacity for mass production of malicious information will become easier and faster. In the spread of online “echo chambers” and “filter bubbles”, the generation of large amounts of unverified and biased content will create a false perception of majority opinion, exacerbating polarization of opinions.
First, misleading historical views. History is objective, but understanding history may be subjective. Especially in international society, due to ideological conflicts and value biases, distorting history is common. In recent years, there have been ongoing disagreements in Western societies regarding the understanding of World War II; concerning the issues of the War of Resistance against Japan, countries in Asia such as China and South Korea often criticize Japan for beautifying its invasion and distorting history. As a creation of humanity, large models are difficult to escape from the biases inherent in humans. In fact, in responses to political issues, amplifying political biases through false or misleading information to manipulate user cognition is not uncommon. When combined with robotic accounts in cyberspace, it may pose even greater security risks. Many tests have found that Western large models often reflect Western positions and values, even distorting history and facts, particularly in issues related to China.
Second, ideological and value biases. Large language models may carry various social biases and worldviews that may not represent the intentions of users or widely accepted values. The real world is not an ideal country of universal harmony; different countries, political forces, and interest groups possess significantly different ideologies and values, reflecting the existing power structures in various types of information. The datasets required for training large models often encode the ideologies and values of real society, which may lead to reinforcing these consequences. Research indicates that a significant portion of the training data in Western large models is produced from the perspectives of white, male, Western, English-speaking individuals, leading to a serious bias in reflecting these structures. The power structures of the real world are encoded within large models, and the outputs of large models reflect the content of existing power institutions, resulting in a Matthew effect of power, often creating oppressive reproduction systems that undermine the information ecosystem. Particularly in areas involving religion, human rights, and other ideological and value issues, as well as in fields with intense conflicts of national interests, monopolizing large models equates to monopolizing textbooks, encyclopedias, and libraries. Large models will become powerful tools for cognitive domain warfare, shaping public cognition and manipulating international public opinion.
Third, challenges of language hegemony. The scale effects of the digital age pose substantial challenges to minority languages. Language is the house of being, the carrier of culture, and the presentation of civilization. Although generative AI can provide multilingual and cross-linguistic services, training large models requires massive corpora. Even domestic models like Wenxin Yiyan are trained based on English environments, which may not only lead to value biases but also result in fierce competition between different languages and the civilizations they represent. If a nation cannot master integrated and monopolistic platforms like large models, it may ultimately lose its language and even lead to its gradual dissolution.
3. General Ethical Risks
In a society characterized by atomistic individualism, generative AI is increasingly becoming people’s conversational companions and intimate “friends,” leading to a series of ethical challenges.
First, humans may develop greater confusion and erroneous cognition about what constitutes “humanity.” Due to excessive competition and internal strife in real society, influenced by increasingly atomistic individualistic values, modern individuals are becoming more isolated, leading to increased alienation in interpersonal relationships. Generative AI large models can support chatbots and companion robots, potentially becoming “partners” for many lonely individuals, but they may also exacerbate the alienation of interpersonal relationships and the solitary nature of personal lives. Technology may help humanity, but it could also make humanity less happy.
Second, limiting personal decision-making ability and weakening human agency. Generative AI presents trends of de-embodiment, de-reality, de-openness, and de-privacy, concealing the risk of more thorough deprivation of algorithmic agency over humans, which is essentially a manifestation of the alienation of human-machine domestication. Human-machine communication will encroach upon interpersonal communication space, thereby weakening the social and psychological connections of embodied subjects: social relationships no longer require physical “presence,” and “public life” consequently disappears. In other words, while humans create algorithms, these algorithms may, in turn, discipline and reformat humans, subtly altering human behavior and values, thus eroding human agency. People may delegate ultimate decision-making authority to certain automated text generators, just as they currently consult Google about existential questions.
Third, hindering content innovation and knowledge progress. When large language models are applied to writing generation, issues such as plagiarism, academic dishonesty, and improper use may arise. Some universities abroad have begun to prohibit the use of ChatGPT on campus to prevent students from cheating on exams or writing papers. Some renowned international journals have explicitly stated that they do not accept AI as co-authors. While large models may serve as excellent tutors, they may also be misused as cheating tools. Particularly for minors, excessive reliance on generative AI may hinder the growth of individual thinking, thereby harming sound character development, school education, and academic training. Since large models simplify the acquisition of answers or information, the effortlessly generated information may negatively impact students’ critical thinking and problem-solving abilities, amplifying laziness and diminishing learners’ interest in conducting their own investigations and arriving at their own conclusions or solutions.
Fourth, fostering false advertising and manipulation of public opinion. In the era of self-media development, the manipulation of public opinion has become an increasingly serious problem. During the disputes of the 2008 Iranian presidential election, the American social media platform Twitter became an important support tool for the opposition. By leveraging social media, the opposition significantly reduced mobilization costs, thereby enhancing their mobilization capabilities. The U.S. government explicitly stated in its 2008 report on funding Iranian dissidents that it supported “new media” and even directly requested Twitter to postpone system maintenance to prevent the opposition from losing communication channels. However, misinformation originating from Twitter was amplified by traditional media such as CNN and BBC. Ironically, manipulators of public opinion often suffer the consequences of their actions. After the Cambridge Analytica incident, some American scholars predicted that large generative AI models represented by ChatGPT would become powerful tools for targeting candidates and influencing public opinion in the next round of elections.
3. Current Status of Information Content Governance for Generative AI Large Language Models
Artificial intelligence brings great possibilities but also raises significant concerns. Humanity must proactively prepare for potential risks and establish universal legislation regarding the safety and ethics of general artificial intelligence development. Given the highly defined nature of regulatory objects, relevant legislation in specialized AI fields is becoming increasingly mature. For example, regulations targeting autonomous driving, smart healthcare, algorithmic recommendations, AI investment advisors, and facial recognition can all be found at various levels of law in different countries and regions.
(1) Regulation of Large Models Becomes an Important Topic in Europe and the United States
Many in the tech and industrial sectors have expressed caution regarding generative AI. They believe that AI systems may pose profound risks to human society, and advanced AI may represent a significant transformation in the history of life on Earth, warranting appropriate planning and management with corresponding care and resources. Currently, AI labs are caught in a race that is out of control, with no one able to understand, predict, or control large models, necessitating a pause in development and a significant acceleration of AI governance to regulate AI development. The Italian data protection authority once issued a ban on ChatGPT and investigated its alleged violations of European privacy regulations. However, as generative AI large models are still in their infancy, countries around the world have not yet formed systematic regulatory policies and governance frameworks.
The European Union is adjusting its legislative progress by deciding to establish a dedicated working group to promote cooperation and exchange information on potential enforcement actions that data protection authorities may take. Some privacy regulatory agencies in EU member states have indicated that they will monitor the risks of personal data leakage associated with ChatGPT under the EU General Data Protection Regulation (GDPR). The European Consumer Organization (BEUC) has called for investigations into ChatGPT by EU and national-level regulatory authorities. The EU is adjusting its AI legislation regarding the regulation of generative large language models, considering requiring OpenAI to undergo external audits to ensure system performance, predictability, and safety settings are explainable. According to the regulatory framework envisioned in the EU AI Act, generative large language models, which may create harmful and misleading content, will be classified as high-risk and subject to strict regulation.
The U.S. government has also begun to take action. On March 30, 2023, the U.S. Federal Trade Commission (FTC) received a complaint initiated by the nonprofit research organization Center for AI and Digital Policy (CAIDP), alleging that GPT-4 does not meet any of the FTC’s requirements for AI use to be “transparent, explainable, fair, and reasonably accountable while promoting accountability,” and that it presents “bias, deception, and risks to privacy and public safety,” calling for an investigation into OpenAI and its product GPT-4 to determine compliance with guidelines issued by U.S. federal agencies. On May 4, the Biden administration announced plans to further promote responsible innovation in AI in the U.S., conducting public assessments of existing generative AI systems. Following the principles of responsible AI disclosure, leading AI developers such as Google and Microsoft will be required to undergo public assessments on specific AI system assessment platforms, providing key information that affects models to researchers and the public, assessing whether they comply with the principles and practices outlined in the AI Rights Act blueprint and the AI Risk Management Framework, to promote timely measures by AI developers to address issues. In January 2021, the U.S. Congress passed the National AI Initiative Act (NAIIA) aimed at enhancing U.S. competitiveness in the field of AI.
As generative AI stands at the forefront of technological competition, it has become a patent of a few countries. Most countries find it difficult to make progress in technological development, industrial deployment, and regulatory governance. Moreover, current AI regulation abroad still mainly focuses on traditional AI fields rather than generative AI large language models. However, due to the existing societal concerns about generative large language models, there is a voice in the EU advocating for generative AI large models to comply with high-risk obligations, which may have significant adverse effects on the competitive environment for local governments, industries, and enterprises.
(2) Current Regulatory Status in China
China has initially formed a comprehensive network information content governance regulatory system composed of laws, administrative regulations, judicial interpretations, departmental rules, and a series of normative documents. The governance of information content for generative large language models has already established a basic legal framework, providing institutional constraints for their development without harming national security, public interests, or individual rights.
In terms of information content regulation, the framework for information content security regulation is composed of laws such as the Criminal Law, Civil Code, National Security Law, Anti-Terrorism Law, Public Security Administration Punishment Law, Cybersecurity Law, Personal Information Protection Law, and Internet Information Service Management Measures, which explicitly prohibit harmful information such as that which endangers national security, social stability, and false information. The “Regulations on the Ecological Governance of Network Information Content” also include previously gray areas of lowbrow and negative information into legislative regulation, highlighting the diversification of governance subjects and objects. Regulations such as the “Regulations on the Management of Internet Audio and Video Information Services” and the “Regulations on the Management of Internet Comment Services” further establish a comprehensive information content regulation mechanism covering all platforms, providing a foundation for the regulation of generative large language models’ content.
In terms of addressing the risks of AI algorithms, the “Regulations on Algorithm Recommendation Management” regulate algorithm recommendation services, initiating the legalization process of algorithm governance. The “Regulations on the Management of Deep Synthesis in Internet Information Services” specifically regulate technologies that generate or edit text, images, and other online information using deep learning and other generative algorithms, providing foundational rules for the application of generative large language models.
On May 10, 2023, the draft “Measures for the Management of Generative AI Services (Draft for Comments)” ended its solicitation for opinions, proposing a series of regulatory ideas regarding data usage, personal information collection, content generation, and content prompt labeling throughout the entire process. However, balancing safety and development is not easy. While proactive regulation reflects the sensitivity of regulatory authorities, the impact on industrial development must also be carefully measured. The new generation of information technology represented by generative AI is a significant high ground in the current international competition. Given that China is in the early stages of this technology field, with insufficient industrial foundation and inadequate experience in application impact, overly stringent responsibility settings for developers at the early stages of local generative large language model technology development may also restrict industrial growth. For instance, whether service providers should bear product infringement liability for damages caused by generative AI or other liabilities should be carefully analyzed. It is essential to adhere to an inclusive and prudent principle, leaving sufficient space for technology and industrial innovation while ensuring national and social security.
4. Exploring Governance Paths for Information Content in Generative AI
Cybersecurity is relative, not absolute; a “zero-risk” target is not a scientific goal. During the model development process, developers objectively find it challenging to foresee all potential risks and must explore and practice within a relatively relaxed environment and reasonable limits. The risks brought by technological progress can only be constrained, not entirely avoided. For example, the accuracy issues arising from the “hallucinations” of large models and accountability difficulties due to algorithmic black boxes can only be controlled as much as possible but cannot be entirely eliminated.
(1) Incentive Compatibility: Optimizing the Legal Environment for Large Model Development
A new round of technological and industrial revolutions is vigorously unfolding, and each industrial revolution produces significant impacts on national prosperity, national decline, and the rise and fall of civilizations. In the context of increasingly intense Sino-U.S. competition, with extreme suppression and blockades against China by the U.S., the question of whether we have generative AI large models and whether our large models are advanced and powerful enough is a more fundamental issue. Strict regulation should be based on advanced technology and a strong industry.
Influenced by the traditional planned economic system and the severe international environment at certain stages, despite the country’s insistence on a socialist market economic system advocating the combination of an active government and an effective market, some localities and departments still habitually intervene deeply in the market. Especially in the internet sector, as network security is extremely important for national security, the internet has become the forefront and main battleground for ideological struggles, leading to relatively strict overall regulation of the internet industry in China. In the field of information content governance, there has never been a law enacted by the Standing Committee of the National People’s Congress; currently, there are only administrative regulations such as the “Internet Information Service Management Measures” that have been in effect for over 20 years. Information content regulation primarily relies on departmental rules, resulting in insufficient legalization of industrial regulation, with rigidity exceeding flexibility and a noticeable one-way regulatory approach from top to bottom.
However, a wealth of empirical evidence both domestically and internationally has demonstrated that modern laws and policies serve not only regulatory functions but also represent an important aspect of international institutional competition. As a comprehensive solution framework for social issues, law and regulation should not be excessively strict or overly lenient; rather, they must maintain a wise balance. If command-style, oppressive top-down regulation is too rigid, it will lead to difficulties in law enforcement or selective enforcement. Regulatory authorities, due to excessive power, may also face greater regulatory capture dilemmas, ultimately stifling technological innovation and industrial development, thus missing national development opportunities.
In this context, developed countries have increasingly emphasized incentive-based regulation. Empirical evidence shows that if regulatory measures and rules align with the incentives of the regulated subjects, it will not only be easier to achieve regulatory goals but also significantly reduce regulatory costs and enhance compliance and legal positivity. Therefore, adhering to the principles of the rule of law and implementing an incentive-compatible regulatory concept and approach has become an important aspect of optimizing the legal business environment. As it stabilizes expectations and benefits long-term, the rule of law is regarded as the best business environment.
In the face of rapidly developing generative AI large models, legislative and regulatory bodies must exhibit greater humility, expressing respect for market, innovation, and industry autonomy, leaving broader space for the development of new technologies and applications. Considering that computing power is the foundation for the development of large models, and that computing power architecture is extremely expensive, legislation and policy choices in China should provide a more favorable policy environment for financing new technologies and industries. Given that large model training requires massive amounts of data, regulatory measures should also aim to eliminate unreasonable barriers in data training while ensuring the protection of personal information and data security, promoting the reasonable flow and utilization of data elements. Laws must adhere to the principles of regularity, and regulation must align with reality. It is essential to confront the risks and challenges posed by generative AI, balancing innovation with public interests, ensuring beneficial applications of generative AI, avoiding social risks, and ultimately establishing an empowering regulatory concept and model that coordinates development and security, aligns with objective regularity, and fits the developmental stage.
(2) Multi-Party Governance: Establishing a Governance Mechanism for Corporate Social Responsibility and Active Participation by Individuals
Technological innovation and industrial advancement are the sources of national prosperity, while adherence to the rule of law and scientific regulation serves as institutional guarantees for national prosperity. Since the latter half of the 20th century, the distinction between “management-type” legislation and “governance-type” legislation has become increasingly clear. “A social governance model that adapts to the era of high complexity and high uncertainty should be a cooperative action model; only through the cooperation of multiple social governance subjects can various social issues be effectively addressed and excellent performance achieved in social governance activities.”
The internet industry, due to its inherent technical complexity, possesses strong professionalism. The history of internet development shows that while the support of the government and the state cannot be overlooked, the roles of the scientific community and technical community are equally important. Upholding the spirit of open-source, the communication and consensus reached by scientists and technical professionals have greatly shaped internet protocols, standards, and rules, providing strong momentum for the development of the international internet. Particularly as the internet represents new technology and new industry, the complex world of code and technological development often outpaces the everyday world, making it impossible for the general public, including regulatory authorities, to fully understand it immediately. The development potential it embodies is not always apparent. Without sufficient patience and tolerance, and without moderate and rational ideas, there is a risk of stifling crucial innovation due to fears of risk. In the field of new technologies and applications in the internet, pursuing absolute safety often leads to greater insecurity. In this context, including China, developed countries often adhere to the concept of multi-party governance and social co-governance, mobilizing both enterprises and society to participate fully while leaving ample space for the development of new technologies and applications.
Generative AI large models, as a new trend in internet information technology development, have already demonstrated explosive and revolutionary potential, empowering various industries as productivity tools, likely bringing significant benefits for future technological innovation, industrial advancement, social governance, and individual well-being, and even becoming an important factor in national comprehensive competitive strength. In this scenario, it is crucial to support and facilitate the development and deployment of large models while strengthening corporate social responsibility, regulating data processing and personal information protection, ensuring that the development and application of AI models adhere to ethical and moral standards, and promoting algorithms that are beneficial and virtuous. It is necessary to enhance risk identification and data traceability, improve technical governance capabilities, clarify data sources and training processes, identify potential biases and other risks through datasets, and establish monitoring systems or conduct manual reviews to monitor content output and identify risks. Establishing a feedback and complaint mechanism to receive, monitor, and evaluate real-time risks and to take timely remedial measures is essential.
The application and impact of generative AI large models are global, necessitating collaborative efforts from research institutions in various countries to coordinate technical standards. As the world’s largest internet country, we must also have the awareness to participate in international internet governance and provide public internet products for the international community, supporting our large models and platforms to participate in and organize global technical communities, contributing Chinese insights to technology, ethics, and regulations.
Of course, we must also enhance citizens’ digital literacy to avoid the digital divide resulting from the imbalanced application of generative large language models. First, we should enhance users’ comprehensive understanding of new technology applications, encouraging the public to adopt a scientifically rigorous attitude towards new technologies without blindly following or opposing them. Second, we should educate the public about the knowledge of neural networks, deep learning, and other technologies, helping people understand the operational principles and limitations of generative AI, thereby avoiding technological dependency. Finally, we should enhance the ability to discern between true and false information, guiding the public to maintain a rational attitude and discernment ability towards the outputs of generative AI.
(3) Govern by Law: Constructing a Legal Framework for Generative Large Language Models
In the field of internet information content governance, China coordinates network ideological security under the overall national security perspective. Within the framework of laws such as the National Security Law, Cybersecurity Law, Anti-Terrorism Law, and Internet Information Service Management Measures, all platforms engaged in news information services with media attributes and public opinion mobilization functions are included in the management scope, and harmful information such as that endangering national security, disrupting social stability, and spreading false information is strictly prohibited. China insists on driving modernization through informatization. First, it coordinates the development of network information content through building a strong network nation, effectively promoting the rapid development of network information technology and significantly enriching information content. Second, it coordinates the construction of network civilization to shape an upward and virtuous network atmosphere, encouraging the public to consciously resist the erosion of illegal and undesirable information. Third, it coordinates the construction of legal governance in the network to effectively curb the spread of illegal and undesirable information in cyberspace, optimizing the network ecosystem.
As a novel platform for information content production and dissemination, generative AI large models, while not fully revealing their potential, are becoming the primary monopolists of information production and dissemination due to their generative, integrative, generalizable, and intelligent interactive characteristics. Therefore, in terms of legislation and regulation, it is essential to accurately identify their risks and improve the regulatory chain from data to algorithms to content within the existing information content governance framework. First, we should regulate the collection, storage, and use of user data to prevent it from being used for harmful purposes, generating false, erroneous, or misleading content. Second, we should improve the algorithm filing system; for various content generated by AI, such as text, images, and videos, we should guide enterprises to establish third-party review or self-regulatory mechanisms. Third, while identifying and regulating harmful information, we should also consider the freedom of individuals to acquire knowledge and create content.
First, we should establish a scientific and clear legal liability mechanism. For providers of generative AI services, legislation should require them to ensure data reliability and accuracy; fulfill content review obligations to avoid harmful information generation; and fulfill special labeling obligations to distinctly mark deep-synthesized content. They should also establish mechanisms for preventing, timely identifying, and stopping the generation and dissemination of harmful and undesirable information. For users, when service providers have fulfilled their safety management responsibilities and exercised due diligence, users should bear criminal responsibility for using the model as a tool for cybercrime. Other information platforms should promptly identify and prohibit or restrict the dissemination of false, harmful, or undesirable information generated by the model. Different types of liability should be determined based on the nature and consequences of different behaviors.
Second, we should coordinate domestic legal frameworks with international legal frameworks. Currently, the mainstream generative AI large models are primarily distributed between China and the U.S., with the U.S. holding a leading position and significant advantages in large models. For actions by foreign entities using generative large language models to harm China’s interests, or interfering in China’s internal affairs through political manipulation and ideological biases, or transmitting other information suspected of violating laws and regulations, Article 50 of the Cybersecurity Law stipulates that “technical measures and other necessary measures should be taken to block dissemination.” In practice, for actions by foreign governments or relevant organizations that use generative large language models to transmit information violating Chinese laws and regulations, it is essential not only to take technical measures to block dissemination but also to explore establishing countermeasures to better safeguard national sovereignty, security, and development interests.