The Development of Generative Artificial Intelligence and Criminal Responsibility

Productivity is the fundamental driving force behind human social development, while production tools are the decisive factor influencing the magnitude of productivity. The innovation of production tools directly determines the development of productivity, thereby driving changes in social forms. A review of the historical development of humanity reveals that this law runs through the entire history of human social development, whether in the Stone Age, Bronze Age, or Iron Age, or in modern times during the steam age, electrical age, and information age, all of which are triggered by innovations in production tools. It is evident that the discovery and utilization of epoch-making new production tools by humans will undoubtedly lead to profound and comprehensive social changes. After the third industrial revolution, electronic computers and the internet became essential production tools in human society, marking the entry into the information age. Based on this, as electronic computers and the internet develop to a certain extent, they will inevitably give rise to the next generation of production tools with stronger productivity, thus triggering the fourth industrial revolution, namely the explosion and application of artificial intelligence technology. In recent years, leading scientists in related technical fields have formed a basic consensus on the process of artificial intelligence development: the development of artificial intelligence will inevitably go through three stages: the first stage is narrow artificial intelligence (ANI) that has undergone long-term specialized training, the second stage is general artificial intelligence (AGI) with strong self-learning and cross-domain capabilities, and the third stage is super artificial intelligence (ASI) that far exceeds human intelligence. This bears some similarity to my earlier classification of the stages of artificial intelligence development, which will gradually enter the era of ordinary robots, the era of weak artificial intelligence, and the era of strong artificial intelligence. However, it is easy to see that my classification of the stages of artificial intelligence development is more comprehensive, as the weak artificial intelligence era I refer to actually includes both narrow artificial intelligence (ANI) and general artificial intelligence (AGI), while the ordinary robot era I refer to (represented by ATM machines) has not been included in the discussion scope of scientists in related technical fields. Nevertheless, regardless of the classification criteria used, it merely reiterates the fact that the development of artificial intelligence will be a process in which the recognition and control capabilities of artificial intelligence robots gradually strengthen, while the consciousness and will of natural persons gradually weaken in influencing the “behavior” of artificial intelligence robots. Each stage of artificial intelligence development has its distinctive technical characteristics and representative applications. For instance, the emergence of AlphaGo in 2016 (the Go program developed by Google) signifies that the development of artificial intelligence has entered the narrow artificial intelligence (ANI) stage, marking that narrow artificial intelligence (ANI) has reached and surpassed human intelligence levels in certain specific fields. AlphaGo, with its outstanding deep learning capabilities, became the first artificial intelligence robot to defeat a human world champion in Go, which not only shocked the world but also initiated discussions and research among legal scholars, including myself, on the legal risks and liability attribution patterns associated with artificial intelligence. In 2022, the advent of ChatGPT (Generative Pre-trained Transformer) marked that current artificial intelligence technology has developed to the stage of general artificial intelligence (AGI), which also signifies that super artificial intelligence (ASI) that far exceeds human intelligence will be the next developmental form of artificial intelligence. The “technological singularity” between weak artificial intelligence and strong artificial intelligence seems to be imminent, as the consciousness and will factors of artificial intelligence itself rapidly strengthen or even expand dramatically, which will fundamentally affect existing legal liability distribution rules, including criminal responsibility.

Undoubtedly, the emergence of a new generation of generative artificial intelligence, represented by ChatGPT, is revolutionary and epoch-making. The existence and development of generative artificial intelligence will inevitably trigger a new round of explosion in social productivity, thereby determining significant changes in production relations, particularly impacting various aspects of social forms, labor production, and lifestyles. The Politburo meeting of the Central Committee of the Communist Party of China held on April 28, 2023, pointed out the need to pay attention to the development of general artificial intelligence, create an innovative ecology, and focus on risk prevention. Authoritative media such as the People’s Daily have also stated regarding general artificial intelligence: “We must actively advance the governance of artificial intelligence with a certain degree of foresight.” I believe that the concerns raised by professionals in related technical fields, social sciences, and all sectors of society regarding generative artificial intelligence like ChatGPT stem fundamentally from the fact that this technology has already, to a certain extent, created the possibility of losing control over natural persons, meaning that iterative versions of generative artificial intelligence like ChatGPT pose a risk of escaping human control. Faced with general large models with hundreds of billions or even trillions of parameters, developers find it difficult to precisely control or explain every detail of the algorithms, leading to the emergence of the “technical black box” problem. Specifically, although general large models of artificial intelligence can generate high-quality content required by humans, developers may be unable to explain the basis and reasons for their decisions, resulting in the incomprehensibility of specific decisions made by generative artificial intelligence like ChatGPT. In a certain sense, this may explain why the development team of ChatGPT cannot explain how ChatGPT generates logical reasoning abilities. As generative artificial intelligence like ChatGPT continues to develop, it is highly likely that it will evolve into a form of artificial intelligence capable of breaking free from human control and possessing independent, free consciousness and will, which will inevitably attract the attention and discussion of the legal community regarding its legal status, capacity for responsibility, and pathways for bearing responsibility. In this regard, I will further explore this issue based on my earlier research, focusing on the philosophical foundations of criminal responsibility, the essential characteristics of criminal responsibility subjects, and the pathways for criminal responsibility of generative artificial intelligence like ChatGPT.

2. Technical Features of ChatGPT and Its Impact on Criminal Responsibility

(1) Concept and Technical Features of ChatGPT

ChatGPT is essentially a generative artificial intelligence chatbot based on the GPT model (Generative Pre-trained Transformer), utilizing a neural network called “Transformer” and large-scale pre-training technology, enabling it to autonomously learn the structure and rules of a large amount of natural language text, thus achieving text generation, code generation, text question answering, paper writing, logical reasoning, sentiment analysis, multilingual translation, and other functions. Since its official release in November 2022, ChatGPT has completed iterative upgrades from GPT-3.5 to GPT-4, showcasing the astonishing content generation capabilities and upgrade speed of generative artificial intelligence. According to official data and research reports released by OpenAI (a leading artificial intelligence company in the United States) and Microsoft, GPT-4 possesses strong logical reasoning and creative abilities, even having the capability to iterate and self-improve through its generated programming code. The application prospects of multimodal pre-trained large models are very broad, and they can be embedded in various fields such as text, painting, audio-visual, education, office software, and securities investment. Therefore, since the release of ChatGPT, many artificial intelligence large models utilizing pre-training technology have emerged like mushrooms after rain, such as Baidu’s “Wenxin Yiyan”, Alibaba’s “Tongyi Qianwen”, and iFlytek’s “Xinghuo Large Model”, all of which are general large models with hundreds of billions or even trillions of parameters. Because ChatGPT can think and reason like a natural person and can approach or even surpass the level of human thinking and judgment, it has raised alarms and panic among related technical leaders. In other words, when artificial intelligence created by humans has, to a certain extent, surpassed the intelligence inherent in humans, we indeed need to ponder the question: Should we risk being replaced or even destroyed by creating artificial intelligence that surpasses human wisdom? Elon Musk and over a thousand tech leaders have urgently called for an immediate halt to the training of AI systems stronger than GPT-4, with a pause of at least six months. Musk also stated at the World Artificial Intelligence Conference that people often underestimate AI’s capabilities, mistakenly believing it to be intelligent humans, while in reality, AI is far more than that and will ultimately fully surpass humans. One of the pioneers of artificial intelligence, often referred to as the “godfather of AI”, Geoffrey Hinton, recently resigned from his position at Google to express his views that “artificial intelligence could cause significant harm to the world”. Sam Altman, CEO and co-founder of OpenAI, called for government regulation of AI during a hearing in the U.S. Congress. To date, many tech leaders, scientists, and government regulatory agencies in various countries have expressed great concerns about generative artificial intelligence like ChatGPT and have called for establishing sound mechanisms and measures to regulate AI.

When numerous tech leaders and government organizations warn that generative artificial intelligence like ChatGPT may pose significant risks, we must maintain an attitude of looking beyond the surface to explore the essential breakthroughs of generative artificial intelligence like ChatGPT. How does generative artificial intelligence like ChatGPT differ from previous artificial intelligence technologies? Only by doing so can we further clarify the specific impacts of generative artificial intelligence like ChatGPT on criminal law theory and its application. In the context of a risk society, the patterns of crime will undergo significant changes, posing enormous challenges to traditional criminal law and criminal policy. Therefore, I will outline the technical features of generative artificial intelligence like ChatGPT as follows:

Firstly, the Transformer architecture. The Transformer architecture is the underlying technical support for generative artificial intelligence large models like ChatGPT. The Transformer architecture is a famous deep learning model that can be widely applied in various fields such as natural language processing (NLP), computer vision (CV), and speech processing. Pre-trained models based on the Transformer architecture can achieve state-of-the-art performance on various tasks. Specifically, ChatGPT adopts the encoder-decoder Transformer architecture, where the encoder is used to understand and recognize input texts, images, and other content, while the decoder is used to generate corresponding response texts. Through the Transformer architecture, generative artificial intelligence like ChatGPT can encode input texts, understand context, and generate logically coherent answers.

Secondly, pre-training + fine-tuning technology. In deep learning, pre-training and fine-tuning are common training strategies used to establish the foundational performance of large models. According to OpenAI’s official website, ChatGPT employs a “pre-training + fine-tuning” technical model. Pre-trained language models use large-scale text corpus data for pre-training, establishing pre-trained language models, and then use small-scale language datasets for specific tasks to fine-tune based on the principles of transfer learning, forming models for downstream tasks. In simple terms, during the pre-training phase, ChatGPT uses large amounts of unlabelled data for self-supervised learning, allowing the large model to establish an understanding of language structure, semantic relationships, and human knowledge; in the fine-tuning phase, the large model uses labeled data for specific tasks for supervised learning to improve performance and adapt to specific application scenarios.

Thirdly, self-correction ability. In addition to the fine-tuning technique used for pre-trained large models, ChatGPT also possesses a certain degree of self-correction ability. Generative artificial intelligence like ChatGPT can improve its generated content based on its understanding of context and user feedback. In other words, generative artificial intelligence like ChatGPT can, to a certain extent, correct its own mistakes or previously generated erroneous responses.

Fourthly, generative model. Unlike previous “retrieval-based” models, the most striking technical feature of ChatGPT is its generative model. Generative artificial intelligence like ChatGPT can autonomously generate various content in a form understandable to humans, not only selecting from pre-trained answers but also generating entirely new and creative content. However, since users cannot ascertain the specific basis for the content generated by the generative model, this generative model also has a series of prominent drawbacks such as unclear information sources and lack of factual verification. This leads users to be unable to directly judge the correctness of the content generated by generative artificial intelligence like ChatGPT, and they need to verify it through comparison with other authoritative information sources.

Fifthly, general large model. Generative artificial intelligence like ChatGPT can handle tasks across multiple domains, including providing information, answering questions, explaining concepts, dialogue Q&A, providing suggestions, editing programming code, intelligent drawing, language translation, etc. Such broad applicability enables generative artificial intelligence like ChatGPT to be suitable for different application scenarios, which is a key difference between general large models and previous single-domain applications (e.g., AlphaGo).

Sixthly, multilingual support. Generative artificial intelligence like ChatGPT can learn all human languages and can even master different dialects of the same language proficiently. As a large language model, generative artificial intelligence like ChatGPT demonstrates exceptional mastery and application of human language. In the current GPT-4 version, it has become increasingly difficult for people to distinguish the content generated by ChatGPT from human language expressions.

In summary, generative artificial intelligence like ChatGPT possesses unprecedented new characteristics and capabilities. If previous artificial intelligence primarily replaced human physical labor (e.g., in unmanned production workshops), generative artificial intelligence like ChatGPT significantly replaces human intellectual labor and is advancing towards super artificial intelligence (ASI). Generative artificial intelligence like ChatGPT is the true meaning of “intelligence” created by natural persons, and its knowledge capacity and intelligence level will far exceed that of natural persons, with the ability to judge materials and make decisions at a speed greatly surpassing that of humans. Therefore, generative artificial intelligence like ChatGPT will become human decision-making consultants and intelligent agents, and its independent consciousness and will will gradually become apparent, which will inevitably have a significant impact on the types of criminal behavior and the distribution of criminal responsibility.

(2) The Impact of Generative Artificial Intelligence like ChatGPT on Criminal Responsibility

Because generative artificial intelligence like ChatGPT possesses a certain degree of creativity and autonomous decision-making (autonomously determining the generated content), it can, in some cases, partially or completely bear causal influence on a specific outcome, leading to changes in the determination and bearing of criminal responsibility for natural persons. In other words, in the causal relationship that leads to criminal results or other harmful consequences, generative artificial intelligence like ChatGPT may not only play a decisive role but also participate in the “behavior-result” causal flow from the perspective of consciousness and will. Because of this, generative artificial intelligence like ChatGPT may have a significant impact on the distribution or sharing of criminal responsibility. This is manifested in the following aspects:

First, generative artificial intelligence like ChatGPT may change the types of traditional criminal behavior and increase the criminal responsibility of perpetrators. Currently, there have been numerous new forms of fraud, such as “face-swapping”, “body-swapping”, and “voice-swapping”, where perpetrators use artificial intelligence technology to simulate familiar characters of victims in video chats to commit fraud. This unexpected new type of fraud not only changes the types of traditional telecommunications fraud but also provides great convenience for perpetrators to commit crimes. It can even be said that without generative artificial intelligence like ChatGPT, the aforementioned new forms of fraud would not exist. With the support of generative artificial intelligence like ChatGPT, the “success rate” of perpetrators committing fraud is higher, and the case values are larger, and their criminal responsibility is correspondingly increased. In addition, virtual influencers have emerged in live-streaming sales on major e-commerce platforms, leveraging the high efficiency and low cost of AIGC content generation, with the operational costs of a virtual host reduced to the level of hundreds of yuan. These tireless AI hosts have already shown significant cost-effectiveness. Once the products sold during live streaming have serious quality issues, the relevant perpetrators may constitute crimes such as selling inferior products or selling food that does not meet safety standards. The behavior of using virtual influencers to sell inferior products is an emerging type of criminal behavior. Although in most cases we can directly apply traditional charges to relevant behaviors, it is undeniable that the use of generative artificial intelligence like ChatGPT does indeed change or enhance the degree of harm or behavioral patterns of traditional crimes. In other words, with the help of generative artificial intelligence like ChatGPT, natural persons (perpetrators) can commit socially harmful behaviors that are more serious and more complex in nature. In these situations, generative artificial intelligence like ChatGPT plays an important supporting role in the entire criminal process. Of course, we must also clearly see that since generative artificial intelligence like ChatGPT merely acts as a tool for committing crimes without participating in “intelligent decision-making”, all criminal responsibility must be borne solely by natural persons (perpetrators). In summary, with the supporting role of generative artificial intelligence like ChatGPT, the types of traditional criminal behavior may undergo changes, and natural persons (perpetrators) may bear heavier criminal responsibility for committing behaviors with higher social harm by utilizing generative artificial intelligence like ChatGPT, without changing traditional charges.

Secondly, generative artificial intelligence like ChatGPT may influence the distribution or transfer of criminal responsibility. “Intelligence” was originally unique to humans, but in the era of artificial intelligence, artificial intelligence technology essentially represents the human creation of a form of “intelligence” that was originally only possessed by natural persons. In the current weak artificial intelligence era, generative artificial intelligence like ChatGPT can, to a certain extent, replace or share the brain functions of humans, assist or even replace humans in making intelligent decisions, thereby influencing the distribution of criminal responsibility by reducing the factor of “free will” of natural persons in the behavior process. For example, when applied in scenarios such as autonomous driving, generative artificial intelligence like ChatGPT takes on certain brain functions based on its intelligence level, replacing humans in making decisions such as accelerating, decelerating, and changing lanes. Therefore, the source of the “intelligence” making these decisions should jointly bear corresponding criminal responsibility with the driver, and the criminal responsibility of each relevant subject should be proportionally allocated based on their level of participation in the intelligent decision-making. In simple terms, the legal basis for bearing criminal responsibility is that criminal law attributes responsibility to relevant subjects based on their “accountability”, which arises from the subject’s ability to choose not to commit a crime but ultimately chooses to commit it through their “free will”. In the causal chain of “behavior-result”, the basis for being attributed criminal responsibility by criminal law mainly lies in the “free will” of the perpetrator, while generative artificial intelligence like ChatGPT participates in the crime from an independent perspective of consciousness and will, and therefore should proportionally share criminal responsibility based on its level of intelligence participation. However, if generative artificial intelligence like ChatGPT does not exceed programming limitations or if developers violate their duty of care, the relevant technical developers should bear the corresponding criminal responsibility for the harm caused by generative artificial intelligence like ChatGPT.

Finally, generative artificial intelligence like ChatGPT may influence the existing scope of criminal responsibility subjects. Criminal law inevitably has a lag due to its emphasis on stability, but this lag should be reflected in legislation rather than lingering in concepts. I have always emphasized that weak artificial intelligence robots cannot become subjects of criminal responsibility, but strong intelligent robots with recognition and control capabilities can and should become subjects of criminal responsibility. Strong intelligent robots can independently make decisions and engage in behaviors that seriously harm society under the guidance of autonomous consciousness and will, and should therefore be treated as subjects of criminal responsibility, with special punitive measures established based on their characteristics. It is undeniable that, at present, there has not yet been a large-scale breakthrough in generative artificial intelligence that enables it to exceed human programming limitations and produce independent consciousness and will. In other words, at present, generative artificial intelligence like ChatGPT, although it possesses technical characteristics that are close to those of strong artificial intelligence in certain aspects or to a certain extent, does not have and cannot produce autonomous consciousness, and its behavior is still subject to the actual control of the programming set by developers and producers, lacking the important characteristic of strong artificial intelligence of “acting independently of programming and implementing behavior under autonomous consciousness and will”. It should be noted that there have indeed been instances where generative artificial intelligence like ChatGPT has “spontaneously” engaged in behaviors infringing upon the rights of others. However, this “spontaneity” does not stem from its possession of independent consciousness and will, but entirely arises from the inherent deep learning and reasoning capabilities of generative artificial intelligence. Generative artificial intelligence like ChatGPT absorbs a lot of false and illegal information during its self-learning and training process over time, and it still cannot accurately understand users’ intentions every time, leading to a certain error rate in the generated digital text. Therefore, at present, we cannot consider that the generative artificial intelligence currently available has reached the level of strong artificial intelligence and can become a subject of criminal responsibility.

However, from a developmental perspective, it is highly likely that generative artificial intelligence like ChatGPT will exhibit levels of intelligence that are inexplicable to humans in subsequent iterative versions, thus generating independent consciousness and will. At that time, generative artificial intelligence like ChatGPT will achieve a leap from general artificial intelligence (AGI) to super artificial intelligence (ASI), potentially becoming a qualified subject of criminal responsibility. Of course, the theoretical community has been engaged in heated debates over this issue, with affirmative and negative views being fiercely contested, with various opinions and no consensus. However, in my view, the giant ship of artificial intelligence technology will not be stalled by storms, and the theoretical innovations surrounding artificial intelligence technology will not cease due to the existence of disputes.

I believe that criminal law is the last line of defense and the support of other laws. The finality of criminal law only indicates that at the legislative level, criminal law only needs to regulate unlawful acts that pose serious social harm and for which other legal adjustments are insufficient. This also applies to legal research and legislative improvement regarding artificial intelligence technology. When the development of artificial intelligence technology is epoch-making and brings revolutionary changes, theoretically, we should fully pay attention to and keep pace with its rapidly changing technological developments, rather than remain stagnant in traditional doctrinal research and turn a blind eye to the latest technological revolution, nor should we casually criticize or mock the theoretical and viewpoint research on the legal responsibilities of artificial intelligence using some traditional theories (even the views of our ancestors).

In the era of artificial intelligence, natural persons are no longer the only subjects possessing intelligence. Humans have created a form of “intelligence” that was originally only possessed by natural persons, using various intelligent robots as carriers of this “intelligence”. It is our belief that generative artificial intelligence like ChatGPT represents the most advanced level of publicly available artificial intelligence technology worldwide, entirely due to its advanced “intelligence” level. As mentioned earlier, based on whether artificial intelligence robots can act independently of human-designed and programmed behaviors, we can classify them into weak intelligent robots and strong intelligent robots. The logical reasoning ability, creativity, and self-correction capabilities of generative artificial intelligence like ChatGPT show that it is already at the critical point between weak intelligent robots and strong intelligent robots. In subsequent iterative versions like GPT-5, generative artificial intelligence like ChatGPT is highly likely to achieve breakthroughs beyond human programming settings, thus becoming strong intelligent robots with recognition and control capabilities. Although generative artificial intelligence like ChatGPT has made significant breakthroughs, some scholars still insist that artificial intelligence cannot become a subject of criminal responsibility. In this regard, we must first clarify the four major relationships that may be involved when discussing the criminal responsibility of generative artificial intelligence like ChatGPT.

(1) The Relationship Between Criminal Responsibility Subjects and Humans

It should be noted that those holding a negative view on whether artificial intelligence can become a subject of criminal responsibility argue that criminal law is a legal norm that regulates crimes, criminal responsibility, and penalties. Criminal law now and in the future can only adjust the behaviors of natural persons, and since artificial intelligence can never be a living natural person, it can never become a subject of criminal responsibility. I cannot agree with this view. For many years, theorists have raised the question, “Is artificial intelligence human?” Objectively analyzing, I believe this question is itself a “pseudo-question”. Because, to date, both theoretically and practically, almost no one holds the view that “artificial intelligence is human”. In my view, rather than discussing “Is artificial intelligence human?”, it is better to discuss “Is artificial intelligence a machine?” This is because most theorists who deny the possibility of artificial intelligence becoming a subject of criminal responsibility generally believe that artificial intelligence, no matter how it develops, cannot change its essence as a machine. Many years ago, I argued that artificial intelligence is neither human nor human-like; it can never become human. Artificial intelligence is neither a machine nor a human; it is a “robot”. Artificial intelligence can only possess responsibility capacity when it has independent consciousness and will, thus possibly becoming a subject of criminal responsibility. We cannot categorically deny the possibility of artificial intelligence becoming a subject of criminal responsibility simply because it is not human or does not resemble humans. In fact, we can pose a contrary question: Is it not possible for generative artificial intelligence like ChatGPT to become a subject of criminal responsibility simply because it is not human? My answer to this question is certainly negative.

Some argue that “No matter how strong the capabilities of artificial intelligence are, how close it is to humans, it is still not human and cannot be equated with humans; existing laws directly adjusting it as a human is logically paradoxical.” This view holds that all laws are constructed around “humans” as the center, and the dominance of “humans” in the construction of legal order excludes the possibility of the existence of other subjects, including artificial intelligence. However, this view has obvious flaws. As mentioned earlier, the essential condition for becoming a subject of criminal responsibility is not because it is “human”, but because it possesses corresponding recognition and control capabilities. In other words, possessing corresponding criminal responsibility capacity is the key condition for natural persons to become subjects of criminal responsibility. Artificial intelligence is not human and will not become human, but this does not completely deny the possibility of artificial intelligence producing recognition and control capabilities, thus possessing the ability to bear criminal responsibility. The “human” defined in criminal law has a special normative meaning, and its scope is by no means equivalent to the biological meaning of “human”. According to criminal law, individuals under the age of 12 and individuals who cannot recognize or control their own behavior due to mental illness cannot be subjects of criminal responsibility and bear criminal responsibility, which sufficiently demonstrates that the “human” recognized by criminal law as having the status of a subject of criminal responsibility must be a “human” with recognition and control capabilities. Therefore, while individuals with criminal responsibility capacity can become subjects of criminal responsibility, this does not mean that the status of subjects of criminal responsibility is monopolized by “humans”. For example, current criminal law explicitly states that non-human entities can also have the status of subjects of criminal responsibility and can independently bear criminal responsibility. Thus, it is evident that the “human” regulated by criminal law does not treat biological characteristics as its essential feature. The attribution of criminal responsibility subject status by criminal law does not require a subject to possess the biological properties of “human”, but rather takes the normative meaning of criminal law as the core judgment factor. When generative artificial intelligence like ChatGPT exceeds programming limitations and can think independently, perceive the external world, and act independently, it becomes a subject of behavioral freedom, thus also becoming strong artificial intelligence. Strong artificial intelligence can independently perform actions that are autonomous from the programming set for it, gaining free will under its conscious control, and its existence is based on its own thinking. This means that strong artificial intelligence is not controlled by the programs and mechanisms preset by designers, developers, and operators, and the instructions given by humans, preset goals, or conflict resolution rules may be denied by strong artificial intelligence, making choices independent of human values and rules. At that point, generative artificial intelligence like ChatGPT will fully possess recognition and control capabilities, and it will be entirely reasonable for criminal law to attribute it with the status of a subject of criminal responsibility.

(2) The Relationship Between Criminal Responsibility Subjects and Life

Following the earlier point, let us delve deeper into the relationship between criminal responsibility subjects and life. I believe that it is not because natural persons are alive that they necessarily possess criminal responsibility capacity; conversely, it is also not because generative artificial intelligence like ChatGPT lacks life that it necessarily lacks criminal responsibility capacity. Some argue that the reason units can become subjects of criminal responsibility is fundamentally that they are based on living “humans” and are a combination of “humans”; whereas artificial intelligence does not possess this property and is not composed of living “humans”. Therefore, non-living entities such as artificial intelligence cannot obtain the status of subjects of criminal law.

In fact, the aforementioned view and similar views fall into a misunderstanding, overly focusing on or emphasizing the “subject form” that can become a subject of criminal responsibility, while neglecting the essential factor behind criminal responsibility subjects—criminal responsibility capacity (i.e., recognition and control capabilities). I believe that recognition and control capabilities reflected in generative artificial intelligence like ChatGPT correspond to its “intelligence level”. Generative artificial intelligence like ChatGPT can simulate human thinking with autonomous consciousness and may even be more intelligent than the human brain, thus becoming a “new species” comparable to humans. Although there is a certain connection between criminal responsibility subjects and life, there is no necessary connection, as cats and dogs also have life, yet they cannot become subjects of criminal responsibility, while units do not possess life but can become subjects of criminal responsibility. The renowned German jurist Radbruch once said, “Legal subjects are things respected as ends in themselves by positive law at a certain time.” The so-called “end in itself” means that the existence of legal subjects is the ultimate purpose, requiring no further proof from other things. Therefore, it is evident that criminal responsibility subjects have a temporal characteristic, which will inevitably change with the times, and should not be limited to natural persons or living entities; non-living entities with free will and practical rationality can also become subjects of criminal responsibility in specific historical contexts. Imagine if generative artificial intelligence like ChatGPT reaches a level of intelligence sufficient to recognize and control its behavior, then its recognition and control capabilities would essentially not differ from those of humans. In this case, what reason remains to deny that strong artificial intelligence can become a subject of criminal responsibility?

(3) The Relationship Between Recognition, Control Capabilities and the Brain

Similar to the previous point, the relationship between the recognition and control capabilities of criminal responsibility subjects and the brain should be noted. Where does the consciousness and will of natural persons originate? To date, this philosophical question still troubles people without a convincing standard answer. I believe that it is not because natural persons possess brains that they necessarily have consciousness and will (recognition and control capabilities); conversely, it is also not because generative artificial intelligence like ChatGPT lacks a brain that it necessarily lacks consciousness and will (recognition and control capabilities). Some argue that “In terms of cognition, artificial intelligence only possesses high cognitive abilities in single domains, while it faces insurmountable technical bottlenecks in cross-domain fields. Secondly, in terms of consciousness, artificial intelligence cannot spontaneously generate consciousness internally, nor can it be forcibly input externally, and therefore cannot possess free will.” Proponents of this view further argue that free will is a prerequisite for bearing criminal responsibility, and free will can only arise from the human brain, but generative artificial intelligence like ChatGPT does not have a human brain. This leads to the conclusion that generative artificial intelligence like ChatGPT cannot become a subject of criminal responsibility due to its inability to produce free will.

Free will is the ability of humans to make independent decisions in action and choice, free from external causal relationships and internal determinations, which relates to whether relevant subjects are responsible for their thoughts, actions, and choices. However, the concept of free will has long been debated in philosophy, psychology, and theology. Some argue that free will objectively exists, allowing individuals to be responsible for their actions and forming the basis of moral responsibility; others argue that free will is merely an illusion, and human actions and choices are determined by prior causal relationships, meaning that humans do not truly possess autonomy. In summary, free will is a philosophical concept regarding human decision-making ability, involving autonomy and responsibility in human choice and action. It is undeniable that the existence of free will is indeed closely related to the human brain, as it involves the individual’s thinking, decision-making, and action abilities. Humans possess highly developed brains and complex nervous systems that enable us to make autonomous choices through thinking, reasoning, and judgment. Although philosophy has not yet provided us with a definitive answer, the general public widely believes that since the brain is the physical basis of human thought and choice, free will is usually regarded as a product of the human brain. However, there is considerable debate about whether free will exists solely in the human brain. Current scientific research levels cannot provide a definitive answer to the question of whether free will exists only in the human brain, nor can they provide sufficient evidence to prove that free will can only arise from the human brain. Therefore, we certainly cannot categorically deny the possibility of other beings or new species such as strong artificial intelligence becoming subjects of criminal responsibility without possessing a “human brain”. In other words, we cannot casually deny the possibility that generative artificial intelligence like ChatGPT may possess free will without empirical support. I believe that while the human brain may generate free will, this does not mean that the brain is the sole source of free will; generative artificial intelligence like ChatGPT, which replaces and surpasses natural persons in terms of “intelligence”, may fully possess free will when it reaches the level of strong artificial intelligence, thus potentially becoming a subject of criminal responsibility. In this regard, although the current level of science and technology cannot confirm the source or true existence of free will independent of humans, we cannot deny the possibility that generative artificial intelligence like ChatGPT may exceed programming limitations and produce free will, thus becoming a subject of criminal responsibility.

(4) The Relationship Between Criminal Responsibility Subjects and the Penal System

It should be noted that those who theoretically oppose the possibility of artificial intelligence becoming a subject of criminal responsibility almost always argue that there is no type of punishment in existing criminal law that can be directly applied to artificial intelligence, and therefore, artificial intelligence cannot become a subject of criminal responsibility. Some argue that “Artificial intelligence cannot be deprived of life, restricted in freedom, or compensated for property like natural persons, nor can it achieve the psychological balance and forgiveness of the victim through traditional penalties… Regardless of how punitive measures are designed, due to the special existence of artificial intelligence, no physical punitive measures can achieve the desired punishment effect.” However, this view seems to have a problem of “putting the cart before the horse” or “seeking the end from the beginning”. I believe that it is not because natural persons can be subjected to existing types of punishment in criminal law that they can become subjects of criminal responsibility; conversely, it is not because generative artificial intelligence like ChatGPT cannot be subjected to existing types of punishment in criminal law that it cannot become a subject of criminal responsibility.

As we all know, criminal law is a product of the times, and the basic theoretical system of criminal law, including the theory of crime composition and the penal system, must inevitably change with the development of history. “To effectively combat crime through legal systems, two conditions must be met: one is to correctly understand the causes of crime; the other is to correctly understand the effects that state punishment can achieve.” Punitive measures and the criminal sanction system are tailored to specific subjects of criminal responsibility at specific times. In other words, criminal responsibility subjects are the cause, while punitive measures are the effect. If we were to exclude new subjects of criminal responsibility simply because the existing types of punishment cannot fully apply to them, it would undoubtedly be putting the cart before the horse. The correct logic is that criminal law should timely accommodate and absorb new subjects of criminal responsibility according to the development of the times, and further tailor new types of punishment or sanction measures for these new subjects of criminal responsibility. Therefore, we should not exclude artificial intelligence from the scope of subjects of criminal responsibility simply because existing punitive measures such as control, detention, fixed-term imprisonment, life imprisonment, and the death penalty cannot be applied to generative artificial intelligence like ChatGPT.

I believe that the qualification of generative artificial intelligence like ChatGPT as a subject of criminal responsibility is a prerequisite for discussing its crimes and penalties. At the same time, the generation of the criminal subject qualification of generative artificial intelligence like ChatGPT must meet the most essential requirements for subjects of criminal responsibility, namely possessing “free will” that can be attributed by criminal law, which can be further traced back to independent consciousness and will that exceeds programming limitations and can be self-controlled. It should be noted that the academic community in our country has previously engaged in macro-level debates surrounding the issue of criminal responsibility subjects for strong artificial intelligence, forming opposing views of affirmative and negative. However, the academic community has yet to conduct more micro and specific discussions regarding the status of generative artificial intelligence like ChatGPT as a subject of criminal responsibility under the condition of strong artificial intelligence.

(1) The Essential Requirements and Evolution of the Generation of Criminal Responsibility Subjects

Since the Enlightenment, the criminal law philosophers of the continental legal system have established three important criminal law schools, thus forming a school debate within criminal law. Due to the differing basic theories upheld by each school, the scope of criminal responsibility subjects has varied over different periods, and the legal basis for attributing criminal responsibility has also evolved.

Firstly, the classical school of criminal law, represented by Beccaria, Feuerbach, and Bentham, adheres to the basic positions of free will theory, moral responsibility theory, and objectivism in criminal law. The classical school of criminal law emerged during the period of bourgeois revolutions, primarily aimed at opposing the arbitrary and identity-based disadvantages of feudal criminal law, hence emphasizing concepts such as freedom and human rights that are beneficial to individual rights of citizens. Deeply influenced by the legal philosophies of Kant and Hegel, the classical school of criminal law regards free will as the legitimate basis for bearing criminal responsibility. “Laws are generally seen as the product of practical reason arising from the will, and principles emerge from the activities of the will in the decision-making process. The latter constitutes free will for humans… This will, as a capacity, is absolutely necessary… Only in the conscious activity process can such choice behavior be called free.” It is evident that, according to the classical school of criminal law, possessing free will is the prerequisite and basis for bearing corresponding responsibility for one’s actions, both morally and legally. However, current scientific and technological advancements seem unable to empirically verify the true existence of free will in the fields of human anatomy and other sciences, and it is merely posited as a priori existence in philosophy.

Free will theory and determinism are a pair of corresponding concepts, and their relationship is one of the fundamental dilemmas that run through the fields of philosophy and law. There are three mainstream views regarding the relationship between free will theory and determinism: one is determinism, which explains everything from the perspective of causality and necessity; the second is free will theory (also known as non-determinism), which explains everything from the perspective of contingency or free will; the third is compatibilism, which holds that freedom is precisely the understanding of necessity. I believe that “freedom” and “necessity” are not necessarily opposed to each other, but rather are mutually inclusive and permeable. The necessary connection between cause and effect proves the existence of causality, reflecting the determined aspect of things; while people can utilize causality to achieve their will, reflecting the aspect of free will. Therefore, free will theory and determinism are not dichotomous; causality can serve free will. This view is profoundly reflected in Kant’s legal philosophy, where he divides all laws in the universe into natural laws (external, natural law causality) and moral laws (self-regulated, freedom-derived causality). Natural laws govern all natural beings in the natural world, and being subject to natural laws is inevitable; moral laws govern all rational beings in the moral realm, and are ruled by the laws of freedom. Kant believes that humans can be governed by both types of laws simultaneously, which affirms that humans possess free will. For example, people’s thoughts and decisions cannot exceed their subjective cognitive range, which reflects the influence of natural laws; while people can freely choose actions within their ability, which reflects the influence of moral laws. Since animals can only act based on instinct, they are subject to natural laws and do not possess free will, which has led to the rise of human-centered criminal law.

In summary, the classical school of criminal law holds that free will is the legitimate basis for subjects of criminal responsibility to bear criminal responsibility. However, due to the limitations of people’s understanding at that time, they believed that only humans could be governed by moral laws (self-regulated, freedom-derived causality), and thus only humans could possess free will, excluding animals and other beings or species from the scope of subjects of criminal responsibility.

Secondly, the modern school of criminal law, represented by Lombroso, Garofalo, Ferri, and List, adheres to the basic positions of behavior determinism, social responsibility theory, and subjectivism in criminal law. As the classical school of criminal law flourished, capitalist society underwent tremendous changes driven by historical forces, with welfare states and preventive states gradually replacing liberal states as new social concepts. Factors such as rising crime rates and rapid advancements in natural sciences like physics and biology promoted the emergence of the modern school of criminal law. The modern school views crime not as a rational act performed by individuals based on free will, but as a social phenomenon determined by the personality and environment of the individual (behavior determinism). Therefore, what should be punished is not the behavior but the dangerous character of the perpetrator (subjectivism); responsibility is the position of individuals or organizations that pose social dangers and should bear social defense measures (social responsibility theory). It can be seen that the modern school does not recognize the existence of free will, believing that all behavior is jointly determined by the character and living environment of the perpetrator, and the basis for bearing criminal responsibility comes entirely from the need to defend society. As various uncertainties in society continue to increase, the issues of national and social security have become increasingly prominent, leading to a growing demand for social defense. From the perspective of social defense, individuals or organizations that may pose a threat to society, including natural persons and legal persons, all possess the necessity to become subjects of criminal responsibility. Thus, the modern school seems to have expanded the scope of subjects of criminal responsibility in practical effects.

Finally, the later classical school, represented by Bentin and Beling, revived the ideas of free will theory and moral responsibility theory. The later classical school mainly reflects the characteristics of the unique hybrid form of the German Empire as a constitutional state and a rule of law state, where nationalism coexists with liberalism. Therefore, the theoretical claims of the later classical school do not completely align with those of the classical school. However, regarding issues such as criminal law attribution, the later classical school still relies on the legal philosophies of Kant and Hegel, reiterating and insisting on free will theory and moral retribution theory as the basis of criminal law and its legitimacy. Analyzing this, the later classical school essentially uses free will as the legitimate basis for bearing criminal responsibility, meaning that when relevant subjects can choose not to commit a crime but ultimately choose to do so based on their free will, criminal law should give them a certain negative evaluation and impose criminal sanctions.

(2) The Conditions for the Generation of Criminal Responsibility Subjects for Generative Artificial Intelligence like ChatGPT

By reviewing the basic views and theoretical discussions of the three major criminal law schools, we can see that free will and practical rationality are the essential conditions for becoming subjects of criminal responsibility. Before the emergence of artificial intelligence, only humans could be governed by moral laws (self-regulated, freedom-derived causality), leading Kant to believe that only humans could possess free will and thus bear criminal responsibility. However, after humans created a form of “intelligence” that was originally only possessed by humans, it is entirely possible for this intelligence to inherit the characteristics of “finite rational beings” from humans and thus possess free will. The “intelligence exhibited by humans” is not merely a “necessary extension of natural instincts”, but rather an “expansive choice of free will”. Therefore, if it can be proven that generative artificial intelligence like ChatGPT currently possesses or may possess free will in the future, it will clear the theoretical obstacles for it to become a subject of criminal responsibility. We can still use Kant’s legal philosophy to argue and resolve this issue.

Specifically, generative artificial intelligence like ChatGPT cannot escape its physical technical limitations and can only operate through data, algorithms, and computing power, and can only express itself in forms understandable to humans such as language and images. This, in a sense, reflects the influence of natural laws (external) in Kant’s legal philosophy, indicating the determined aspect; but generative artificial intelligence like ChatGPT can complete free creation and composition based on its deep learning of human knowledge, generating entirely new content or ideas that did not exist in the world, which in a sense reflects the influence of moral laws (self-regulated) in Kant’s legal philosophy. Therefore, like humans, generative artificial intelligence like ChatGPT may also be governed by both natural laws (external, natural law causality) and moral laws (self-regulated, freedom-derived causality), thus presuming its potential to possess free will, which in turn satisfies the most essential conditions for bearing criminal responsibility or becoming a subject of criminal responsibility—criminal responsibility capacity. The potential content of free will that generative artificial intelligence like ChatGPT may possess includes independent consciousness and will, while the content of criminal responsibility capacity corresponds to recognition and control capabilities, which correspond one-to-one. That is, the independent consciousness in free will corresponds to the recognition capability in criminal responsibility capacity, and the independent will in free will corresponds to the control capability in criminal responsibility capacity. Thus, generative artificial intelligence like ChatGPT may satisfy the logical closed loop from free will (independent consciousness, will) to criminal responsibility capacity (recognition, control capabilities), thereby influencing or even expanding the existing scope of subjects of criminal responsibility.

Since its development in 1956, artificial intelligence technology has achieved a transition from ordinary artificial intelligence robots to weak artificial intelligence robots, steadily advancing along the trajectory of narrow artificial intelligence (ANI), general artificial intelligence (AGI), and super artificial intelligence (ASI). We must recognize the complexity and breakthrough nature of generative artificial intelligence like ChatGPT, and while understanding the infinite conveniences it brings to various social tasks, we must also cautiously consider its potential legal risks and challenges to ensure that the use of generative artificial intelligence like ChatGPT aligns with our country’s legal and ethical standards. The development of generative artificial intelligence like ChatGPT is a dynamic and evolving process, and we should maintain an open mindset, actively explore its impact on criminal law theory research, and achieve a positive interaction between generative artificial intelligence like ChatGPT and criminal law studies, contributing to the construction of a more scientific, reasonable, fair, and reliable criminal law system.