Author: Cao Jianfeng, senior researcher at Tencent Research Institute

Responsibility for artificial intelligence accidents is the core issue of the AI ​​era

After the "golden decade" of deep learning, the AI ​​era has Since then, AI systems have become one of the most significant and important technological objects in the 21st century, constantly spawning a variety of new smart products, services and applications, such as Robotaxi, AI companion applications, humanoid robots, etc. Moreover, under the guidance of the scaling law, AI technology represented by large models is still accelerating its development, even triggering differences in development concepts of "AI accelerationism vs. AI value alignment." Well-known experts and scholars in the field of AI have predicted the arrival time of AGI and envisioned the changes and impacts in the next ten years. People may not be able to fully predict what changes and impacts this AI revolution will bring, but we should at least not underestimate the long-term impact of AI technology.

At present, the development and application of AI technology are not only making various products increasingly have independent intelligence, but also accelerating people's transition from the Internet era to using algorithms, robots and AI agents (AI agents). ) is an AI society or algorithmic society with the core feature of making economic and social decisions. Algorithms have thus become the core technical factor supporting the development of networking, digitization, and intelligence. This can lead to significant improvements in safety and efficiency, but there is no guarantee that all accidents and risks will be eliminated. There is no absolutely safe technology. While technology reduces and eliminates risks in some aspects, it may create new risks in other aspects. In recent years, AI accidents that have harmed the rights and interests of others have increased rapidly, from safety accidents of self-driving cars and tangible robots to incorrect diagnosis of AI diagnostic and treatment software to algorithmic discrimination and unfair decision-making in various automated decision-making systems. It can be said that AI Accidents and AI infringement are increasingly becoming the “new normal” in the AI ​​society. According to OCED's monitoring of AI accidents worldwide, global AI accidents have increased rapidly since January 2014, and the total number has reached 7,195 as of December 2023. For example, since October 2024, the AI ​​chatbot platform Character AI has faced controversies such as causing the suicide of American teenagers, and has become a defendant in at least two lawsuits. The plaintiff claimed that Character AI has design defects and is a defective product, requiring its developers to bear product liability. .

Today, when the application of artificial intelligence is everywhere, people must face up to the legal liability issues of AI accidents and AI infringement. When AI systems cause accidents and damage, the law must provide fair and effective relief to the victims. But the question is, who should be responsible for AI accidents and AI infringements? Highly autonomous AI systems may act or make decisions independently without direct human control, intervention, or supervision. This means that, in the context of artificial intelligence, accidents and damage occur as the associated behaviors and decisions shift from humans to intelligent systems.Harms have also begun to shift from being caused by humans and human behaviors to being caused by AI systems and their behaviors. This shift poses challenges to the allocation and assumption of legal responsibilities. The relevant challenge lies not only in the difficulty of identifying the responsible subject, but also in the difficulties brought by the autonomy, unexplainability, unpredictability and other characteristics of the AI ​​system to prove fault/defect, causality and other liability components, and even more in the difficulty of AI infringement. Difficulties in how to assume responsibility (for example, how to take punitive measures such as behavioral bans and shutdowns against AI systems).

Are the three new options for AI tort liability really feasible?

To this end, some people have proposed establishing a new liability system for AI infringement, which roughly includes three options.

The first is the personality plan, which simply means giving the AI ​​system legal subject status, so that the AI ​​system can directly bear legal responsibility for its own actions. It is a very tempting idea to regard autonomous and complex AI systems as independent legal subjects, thereby transferring tort liability from humans to artificial intelligence. EU lawmakers had proposed creating a special legal status for "cyborgs" for autonomous robots, but ultimately rejected the idea. Some scholars have proposed that AI systems can be given a legal person status similar to a limited liability company (LLC) to solve the problem of liability. People imagine that just as large models may make "1-person companies" a reality, future development of artificial intelligence may also make "0-person companies" a reality, that is, AI systems (agentic AI) with the ability to act autonomously can operate independently. A company without any human employees.

The second is new liability schemes such as vicarious liability and high-risk no-fault liability. According to one theory, especially in the case of alternative artificial intelligence, if companies use AI systems to replace human employees, We should be vicariously liable for the actions of so-called "AI employees" because this is consistent with the principle of functional equivalence. As the capabilities of large models continue to increase, we can envision a future in which people may not only have personal AI assistants that can truly act on their behalf, but may also work and collaborate with so-called “AI colleagues.” Therefore, it seems reasonable to hold operators vicariously liable for the actions of “AI employees”. Another idea is to base on the risk-based AI supervision path, allowing providers, owners, users and other entities to bear no-fault liability for the damage caused by high-risk AI systems. For example, the core idea of ​​the EU Artificial Intelligence Act is to focus on safety supervision of high-risk AI systems based on the typing of AI risks, and to prohibit AI systems with unacceptable risks.

The third is the insurance solution. For damage caused by a completely autonomous AI system, you can consider completely replacing the existing tort liability system with insurance-based no-fault compensation mechanisms such as social insurance and compensation funds, because it bypasses Tort law would avoid many of the difficulties faced in applying existing liability rules to artificial intelligence. In the past, no-fault compensation mechanisms completely replaced tort damages.It is not uncommon, and similar practices exist in the fields of work injuries, traffic accidents, medical injuries, vaccine injuries, etc.

The construction of the AI ​​tort liability system needs to get rid of several misunderstandings

However, these new plans for AI tort liability are too radical and difficult to ensure the balance between safety and freedom. Not only are they inconsistent with the fact that we are still in the early stage of the development of the AI ​​revolution and the social reality in the era of weak artificial intelligence, and is based on several misunderstandings of blame that need to be avoided.

Myth 1: Attributing blame to artificial intelligence itself.

Attributing liability to the AI ​​system itself means treating the AI ​​system as a legal subject. However, at this stage, legal personality for artificial intelligence is morally unnecessary and legally asking for trouble. Most of the arguments in favor of AI legal personality are both too simple and too complex. Too simple because AI exists in a vaguely bounded sphere and there is no meaningful category that can be recognized as a legal subject; too complex because Many arguments are variations on the “robot fallacy” (such as believing that robots will be just like humans) and are based on specious assumptions about the future development of artificial intelligence. At present, granting legal personality to AI systems is not a "panacea" to solve their "behavior" responsibilities. Instead, it may open a "Pandora's box" and trigger a series of new legal and ethical issues. In particular, AI legal personality will easily lead to Abuse becomes a mechanism to avoid and shift legal responsibilities and obligations. In other words, AI legal personality may be a kind of “legal black hole,” an entity that sucks away the legal responsibilities of human actors without seeing any trace of accountability. In short, artificial intelligence, as an activity of human beings, no matter how complex, intelligent, or advanced it is, is only a tool to serve human beings and achieve human purposes, fully demonstrating the need to serve as a legal object and promote human well-being. Fundamentally, we need to develop tool AI, rather than the so-called subjective AI that is fully close to humans.

Misunderstanding 2: Connect the concept of AI risk typing in public law with AI tort liability rules.

One of the main ideas for global artificial intelligence regulation is to adopt "risk-based regulation" and adopt differentiated supervision for AI systems with different levels of risk. The EU Artificial Intelligence Act is a typical representative of this idea. According to the level of risk, AI systems are divided into four categories: unacceptable risk AI, high risk AI, limited risk AI and minimal risk AI, and focus on high-risk AI. requirements and obligations of relevant operators (providers, deployers, etc.). Among them, the criterion for judging high-risk AI is that the AI ​​system poses a significant risk of harm to the health, safety, and basic rights of natural persons. Under this regulatory approach, people tend to link the risk level of AI systems to liability principles, such as linking high-risk AI to no-fault liability, while linking low-risk AI or non-high-risk AI to fault liability or fault liability. Presumptive liability is linked. The “Draft Regulation on Responsibility for Artificial Intelligence Operations” previously proposed by EU legislators is this idea of ​​responsibility.A typical representative of the road. However, it is unreasonable to uniformly connect and match risk-based AI typing to different liability rules under the public law regulatory framework. In fact, it is a misplacement. The main reason is that high-risk AI cannot be simply equated with unusually dangerous objects or activities targeted by traditional no-fault liability theories; on the contrary, the introduction of artificial intelligence may change people's perception of dangerous areas. High-risk AI may actually be safer than similar objects or activities controlled and operated by humans. In other words, so-called high-risk AI is actually designed to reduce risk, increase safety, and is actually safer than the human activities it replaces.

Misunderstanding 3: Negligently evaluating the “behavior” of the AI ​​system.

When an AI system causes an accident or causes damage, how to evaluate the "behavior" or performance of the AI ​​system is a key issue. Some people have proposed that the principle of negligence be applied to the AI ​​system itself. Specifically, by analogy with the "rational person" standard used to judge whether a human actor is negligent, the "rational robot" standard can be used to judge whether the AI ​​system has so-called "negligence". ”, in order to limit the liability of relevant entities for AI systems. For example, in the previous case in the United States, in the case of Nilsson v. Gen. Motors LLC, the plaintiff sued the American self-driving car company Cruise, but did not file a product liability claim, but chose a theory based on negligence: The motorcycle owner claimed that Cruise’s automatic The car was driven in such a negligent manner that it entered an adjacent lane and knocked him down without regard for passing vehicles. This may be the first time in history that a robot has been formally accused of negligent operation - a tort charge that was once reserved for human actors. However, this idea of ​​​​imputation should be rejected at any time. Even if the identification of negligence in modern tort law has become more objective, the concept of negligence always points to the behavior of human actors and is linked to human subjectivity. It is unrealistic to apply a negligence standard to the “conduct” or performance of an AI system. It is foreseeable that as the autonomy of AI systems increases, in many AI accidents in the future, courts will need to shift from evaluating the behavior of users (such as drivers) to evaluating the behavior of AI systems (such as autonomous driving systems). The "behavior" or performance of the AI ​​system should be evaluated from the perspective of product defects rather than from the perspective of faults. This requires us to promptly update the product liability system for traditional products in the industrial era.

Myth 4: Substitute accountability for entities that deploy and operate AI systems based on the principle of functional equivalence.

The principle of functional equivalence holds that if the use of autonomous technologies such as AI systems is functionally equivalent to hiring human auxiliary personnel and causes harm, then the operators deploy, Responsibility for using the technology lies with the principal and its human support personnelIt corresponds to the existing vicarious liability mechanism of Human Auxiliary, that is, the operator of the AI ​​system bears vicarious liability for the damage caused by the AI ​​system. However, this line of thinking is asking for trouble. The analogy of responsibilities based on functional equivalence may seem reasonable at first glance, but in fact it is not feasible. Moreover, the functional equivalence theory only superficially focuses on the substitution effect of technology, but fails to gain insight into the real risk creation and control behind this technological phenomenon. For example, in the era before artificial intelligence, factories used automated devices to replace workers. If the automated device malfunctioned and caused damage, the victim would consider pursuing product liability against the manufacturer of the automated device, rather than letting the factory bear vicarious liability for the automated device. While the risk profiles of AI systems may vary, they are simply more advanced and smarter tools than traditional automation devices, which means one needs to cut through the fog of functional equivalence and examine which subjects (generally i.e. providers and users of tools) create or control risks. Because ultimately people just want someone to be held accountable for the damage caused by AI systems, rather than holding AI systems accountable in the same way that human actors are held accountable.

Where does the tort liability system go in the AI ​​era?

Although artificial intelligence poses challenges to the effective application of the current tort liability system, this does not mean that we need to start from scratch and adopt a new liability solution. On the contrary, at this stage, by making necessary adjustments to existing tort liability rules such as fault liability and product liability, we can adapt the tort liability system to the development needs of the AI ​​era and achieve a balance between safety and innovation.

First, adhere to the legal object status of artificial intelligence and implement human responsibility in AI accidents and AI infringements. Starting from technical reality, no matter how advanced and intelligent the current AI system is, someone will always need to develop it and put it into use. Specifically, although the AI ​​value chain is complex, we can relatively clearly distinguish two groups: the provider camp and the user camp. This distinction makes legal sense because within each group (for example, between producers and suppliers, owners and users), liability can be relatively easily assigned to one of its members through contractual instruments Or shared among several members. For example, the EU Artificial Intelligence Act distinguishes between AI providers and AI users (deployer of AI systems), and focuses on imposing relevant obligations and responsibilities on these two types of subjects. Therefore, for the purpose of tort liability, it is necessary and important to establish standards for the identification and determination of AI providers and AI users.

Second, innovate the product liability system for the AI ​​era. Although in many specific usage scenarios of artificial intelligence applications, users still need to fulfill certain obligations of care (such as using according to the intended purpose, ensuring data quality, monitoring, maintenance, etc.), they have control over those aspects of the use of artificial intelligence.However, in the long run, the user's duty of care will be reduced, which means that the user's responsibility may also be reduced accordingly. As the roles and control of AI owners and users continue to weaken, the liability of AI providers may enter the center stage of tort liability law in the future. As a new type of "smart" product, AI systems call for necessary innovations in the existing product liability system, including product concept, producer definition, defects, compensable damage, causality, burden of proof, etc. For example, in terms of artificial intelligence regulation, EU legislators, while formulating the world's first comprehensive artificial intelligence bill, comprehensively revised the EU Product Liability Directive introduced in 1985, aiming to establish a new regulatory framework for the digital and AI eras. Product liability system. At the same time, EU lawmakers are still preparing the "AI Liability Directive" (AI Liability Directive), which aims to establish clearer and more operable rules for the responsibilities of AI users.

Third, insurance should be used as a useful supplementary mechanism to the AI ​​liability framework, rather than a substitute mechanism. As a risk management tool, insurance plays an important role that cannot be ignored in promoting the safe integration of new technologies into society, such as by providing financial protection to stimulate innovation and ensure the safe implementation of new technologies. With appropriate adjustments and regulatory intervention, insurance can continue to support technological innovation and provide necessary protection to society. Existing insurance systems can be used to regulate AI systems, but there is no need to develop dedicated or comprehensive AI insurance. At the same time, we should be cautious about introducing mandatory insurance for artificial intelligence applications, so as not to backfire and hinder the promotion and popularization of AI technology that can bring significant economic and social benefits.

Fourth, in addition to the AI ​​tort liability system, we need to pay attention to and actively respond to the security risks of cutting-edge AI. In terms of artificial intelligence governance, AI tort liability rules are necessary, but their role is limited. Although they can effectively deal with the risk of damage to people’s personal and property rights and interests that AI systems may cause, they are still not applicable to cutting-edge AI (frontier) such as super intelligence. AI) may bring extreme risks or catastrophic risks, and it is difficult for AI tort liability rules to play a substantial role. Under the accelerating development trend of AI, superintelligence (superintelligence) is already on the horizon, and its potential security risks are increasingly receiving active attention and attention from various countries, research communities, and industries. Foreign AI experts pointed out that in the long run, most people underestimate how serious the security risks of superintelligence may be. Therefore, actively advocate, develop, and build welfare AI, use artificial intelligence to maximize personal, social, and environmental well-being, and integrate the concept of human-machine alignment, including AI value alignment, into the development of super intelligence. becomes particularly important.

The content of the article is only for academic discussion and does not represent the views of the employer

[1] Zheng Zhifeng: "Legislative Update on Artificial Intelligence Product Liability", in "Law" Science (Journal of Northwest University of Political Science and Law)" Issue 4, 2024

[2] Cao Jianfeng: "Human-computer alignment in the context of large models", in "Social Sciences" Newspaper》

https://www.cssn.cn/skgz/bwyc/202410/t20241029_5797216.shtml

[3] https://www.youtube .com/watch?v=559VdVIz5V4

[4]https://eur-lex.europa.eu/eli/dir/2024/2853/oj

[5] https: //mp.weixin.qq.com/s/HKbVSvQzULG13BSLCfVpBQ

[6] https://darioamodei.com/machines-of-loving-grace

[7] https://ia.samaltman.com/

< p style="text-align: left;">[8]https://www.washingtonpost.com/documents/028582a9-7e6d-4e60-8692-a061f4f4e745.pdf