Subscribe to “China Information Security”
Postal code 2-786
Subscription hotline: 010-82341063
Written by: Zhou Hui, Deputy Director and Researcher of the Network and Information Law Research Office, Institute of Law, Chinese Academy of Social SciencesWith the rapid development of artificial intelligence technology, large models have made significant technological advances in the field of artificial intelligence. However, while these technologies bring convenience to humanity, they also come with a series of security threats and challenges. Currently, the industry has conducted some practical explorations on how to enhance the security protection of large models, attempting to strengthen the safety of large models through improved internal control systems, establishing review mechanisms, timely evaluation audits, and applying technical tools. In the future, it is also necessary to further promote the legal construction of large model security from an institutional level, enhance the effectiveness of risk prevention, and ensure safety throughout the entire process of large model research and application.
1. Risks in the Development of Large Models
Large models refer to artificial intelligence models trained using massive amounts of data, consisting of complex computational structures and numerous parameters. Compared to smaller, single-purpose models, large models have a broader application scope, but they may also bring more unpredictable and uncontrollable risks. Firstly, the data used in the research and training of large models may have issues of legality and bias, which can lead to technical risks or trigger discrimination problems. Secondly, there is a risk of technical misuse of large models, and their reliability and stability may also be affected during widespread application. Finally, if large models themselves are attacked, the resulting impacts and damages will be even more severe.
Technical Risks. Among these, data privacy and security are core issues. Since training these models requires a large amount of data, which may contain sensitive information, the risk of privacy leakage also increases during the collection, storage, and processing of this data. Moreover, models may inadvertently memorize and leak personal information if not adequately anonymized. This requires strict privacy protection measures to be implemented during data collection and processing to ensure data security and users’ privacy rights. Digital Bias Risks. If the training data contains biases, the model may learn and amplify these biases, leading to unfair outcomes, such as unfair treatment of certain groups in facial recognition or language understanding tasks. To reduce the occurrence of such biases, diversified and balanced strategies need to be adopted during data collection and model training. Misuse Risks. Deep learning models are often regarded as “black boxes,” making their decision-making processes difficult to understand and explain, which not only increases the risk of misuse and abuse but also complicates regulation and auditing. Therefore, enhancing the model’s explainability and transparency, making its decision-making process clearer, is crucial for achieving the safety and reliability of artificial intelligence. Security Risks. Models may be vulnerable to adversarial attacks, where attackers deceive the model through carefully designed minor input changes, leading to erroneous outputs. This security vulnerability is particularly dangerous in sensitive applications such as autonomous vehicles. Therefore, improving the robustness and security of models to prevent adversarial attacks is key to ensuring the safety of artificial intelligence. Relevance Risks. Although the generality of large models is an advantage, it may also pose risks; for example, a model trained for a specific task may produce unforeseen and sometimes harmful results in other fields. Additionally, information leakage is also a concern, especially when handling texts or images containing personal data, as the model may inadvertently memorize and leak sensitive information. Overall, the security risks in the development process of large models are not isolated but will continue to propagate and accumulate throughout the entire lifecycle of the large model. The security risk challenges faced by large models not only raise higher requirements for regulatory work but also require close cooperation and collaboration among large model developers and other entities providing services based on large models.
2. Governance Practices for Large Model Security Protection
In the rapid development of artificial intelligence technology, the security governance mechanisms of large model platforms play a crucial role. These mechanisms can enhance the safety, reliability, and compliance of artificial intelligence technology applications, and also help increase public trust and promote healthy technological development.
Establish Governance Frameworks. Large model platforms manage the research, application, and deployment of artificial intelligence technology by establishing comprehensive governance frameworks. This framework includes both internal governance models and external expert participation in governance. The internal governance model involves management teams, legal teams, technical teams, etc., who are collectively responsible for formulating and executing governance strategies and rules, ensuring that platform operations comply with laws and regulations, while the technical team implements safety measures at the technical level. For example, Meta’s independent oversight board is a typical representative of internal governance. External expert participation provides independent voices and objective evaluations for the model’s assessment and governance processes through technical committees and ethics committees, such as Baidu’s Paddle platform and OpenAI’s collaboration model. Build Content Review and Filtering Mechanisms. To address misleading and false information generated by content, many large model platforms and service providers have established content review and filtering mechanisms. These mechanisms utilize manual reviews or automated algorithms to identify and remove false information, inappropriate content, and illegal information. These initiatives can help reduce the spread of harmful information to some extent, thereby enhancing the security of cyberspace. Implement Data Privacy Protection Measures. With the increasing awareness of personal privacy protection, organizations and enterprises engaged in large model research and development have adopted a series of measures to ensure user data security and privacy, including data encryption, access control, de-identification techniques, and the establishment of privacy policies and compliance processes. Moreover, existing laws and regulations in other areas also apply to large model research and development; for instance, data collected and used for training large models that contain personal information must comply with clear requirements for the collection, use, and protection of personal information as stipulated in the Personal Information Protection Law. Clarify Ethical Norms. Ethical norms provide moral guidelines and behavioral standards for the research and application of artificial intelligence technology, focusing on transparency, fairness, accountability, privacy protection, and safety. Large model platforms have constructed a responsible action guidance framework by mentioning or introducing specific ethical norms in user service agreements. Conduct Adversarial Testing. Adversarial testing, as a proactive safety audit and risk assessment method, simulates the perspectives and behaviors of attackers to discover and fix security vulnerabilities. Large model platforms have demonstrated their commitment to self-detection and improvement of model security by participating in public red team testing activities. To counter adversarial attacks and model deception, researchers and security experts have proposed a series of techniques and methods for adversarial attack and model security reinforcement. These techniques include adversarial sample training, robustness enhancement, and model robustness assessment. By improving the model’s resistance to adversarial attacks, its safety and stability can be enhanced. Conduct Evaluation Audits. Risk assessment and audits are important components of large model platform governance. Risk assessments are used to identify and analyze potential risks that may arise during the development, deployment, and use of large models, while audits evaluate whether the design, development, and operation of the model comply with established standards and best practices. The existing security protection measures have significantly improved the safety and reliability of large model platforms. Through internal governance and participation from external experts, platforms can more comprehensively identify and address potential risks. The clarification of ethical norms provides a responsible action guidance framework for all parties involved in technology development and operation, enhancing public trust in artificial intelligence. The implementation of adversarial testing and evaluation audits helps to timely discover and fix security vulnerabilities, ensuring the model’s transparency, explainability, and fairness. However, the effectiveness of these measures also faces challenges. Maintaining the independence and objectivity of audits, handling sensitive information, and meeting the constantly changing legal and regulatory requirements are all difficulties that need to be overcome. Furthermore, as artificial intelligence technology continues to advance, existing security protection measures also need to be continuously updated and improved to adapt to new technologies and application scenarios.
3. Systematically Promote the Legal Construction of Large Model Security
Various large models essentially belong to a branch of artificial intelligence technology, and improving the legal framework for large model security will inevitably require enhancing the overall governance effectiveness in the field of artificial intelligence. To achieve high-level security guarantees and high-quality development, we need to build on existing practices to conduct more effective comprehensive governance of artificial intelligence technologies, including large models, through improved institutional design, clarify security obligations and responsibilities during the research and application process of large models, and encourage the construction of an industrial ecosystem that emphasizes security protection. Additionally, attention should be paid to the flexibility and operability of relevant systems to enhance the capacity for security risk prevention.
Establish Specialized Authorities for Artificial Intelligence. The European Union’s Artificial Intelligence Act proposes the establishment or designation of specialized authorities for artificial intelligence in the EU and its member states, which helps strengthen coordination in the governance of artificial intelligence security risks across countries and across fields and industries. China can refer to related legislation to establish a national authority for artificial intelligence, responsible for the overall development and management of artificial intelligence, thus facilitating coordinated governance of large models applied across industries. The responsibilities of the national authority for artificial intelligence regarding security governance may include: conducting education and publicity on artificial intelligence security; organizing the formulation of artificial intelligence security standards and detailed regulatory rules; guiding and supporting specialized institutions to carry out artificial intelligence security monitoring, assessment, auditing, certification, etc.; establishing artificial intelligence security risk monitoring and early warning mechanisms and emergency response mechanisms for security incidents; and regulating and enforcing laws on artificial intelligence research and provision activities that pose security risks or may lead to security incidents. Implement List Management Based on the Severity of Security Risks for Large Model Research and Application. For large models and their derivative products and services that pose higher security risks and may cause serious impacts if security incidents occur, they should be included in a negative list. Research and provision activities for large models on the list must obtain legal permission in advance. For large models not on the list, a post-event regulatory approach based mainly on filing should be implemented to encourage innovation and avoid the notion that “not developing is the biggest insecurity.” Strengthen Security Obligations for Specific Entities, Focus on Preventing Large Model Security Risks. In general situations, the developers and providers of large models can be distinguished, and their obligations in security protection can be specified separately. Considering the widespread application of general large models and the ease of security risk transmission, specific security obligations can be stipulated for models exceeding a certain computational power scale, such as formulating model usage rules, timely cooperating with regulatory authorities and users to eliminate security hazards, etc. At the same time, if platform operators know or should know that product or service providers on the platform are engaged in large model applications and services, they also have the obligation to improve security specifications within the platform and ensure their effective implementation. Create an Artificial Intelligence Industry Ecosystem that Values Security to Support Large Model Security Assurance Work. Firstly, regular security assessments and audits should be established as a general obligation for artificial intelligence developers and providers, and the development of third-party institutions providing artificial intelligence security assessments, audits, and testing should be encouraged. Secondly, investments in specialized equipment for security governance should be eligible for tax deductions at a certain proportion to encourage enterprises to strengthen artificial intelligence security protection. Thirdly, support should be provided for enterprises, research institutions, etc., to research and develop technologies related to artificial intelligence monitoring and early warning, security assessment, emergency response, etc., encouraging the application of regulatory technology and compliance technology in the field of artificial intelligence. Clarify Legal Responsibility for Artificial Intelligence Security Incidents. On the one hand, violations of network security, data security, and personal information protection regulations during the research and application of artificial intelligence that lead to security incidents should be dealt with according to law; on the other hand, strict penalties should be set for violations of artificial intelligence security protection obligations, such as ordering the suspension or termination of related businesses, revoking licenses, etc. Additionally, considering the rapid iteration of artificial intelligence technology and the uncertainty of risks, a compliance exemption system for artificial intelligence developers and providers should also be clarified. If developers and providers have fulfilled existing security protection obligations without subjective malice, and promise to rectify compliance and strengthen security protection, their liability may be mitigated or exempted based on the circumstances to enhance proactive compliance with security measures.【This article is supported by the Chinese Academy of Social Sciences 2024 Laboratory Incubation Special Fund (Project Name: Research on Artificial Intelligence Security Governance; Project Number: 2024SYFH007)】
(This article is published in the June 2024 issue of “China Information Security” magazine)
Share Cybersecurity Knowledge and Strengthen Cybersecurity Awareness
Follow the official Douyin account of “China Information Security” magazine
“China Information Security” magazine strongly recommends
“Enterprise Growth Plan”
Click the image below for details
