EU: Report on Liability for Artificial Intelligence and Other Emerging Digital Technologies – Part 2

By Zayed Al Jamil, Counsel, and Holly Tunnah, Knowledge Paralegal and AI Specialist for LIBOR

The New Technologies Formation (NTF) of the Expert Group on Liability and New Technologies, set up by the European Commission, was convened in June 2018. After analysing the relevant national laws and looking at specific use cases, its 2019 report compared various aspects of existing liability regimes and discussed how emerging digital technologies could fit into the pre-existing framework. The NTF decided that certain adjustments need to be made to EU and national liability regimes to allow for just and equitable results for those affected.

The NTF considered both artificial intelligence (AI), and more broadly, “emerging digital technologies”, including the internet of things (IoT), distributed ledger technology (DLT), advanced robotics and autonomous systems. This means that the conclusions reached in the report could have implications for a broad range of technologies, sectors and individual businesses, even where, on the face of it, a particular technology (to pick an example from the report, an IoT-enabled kettle) does not immediately appear to be on the technological cutting edge.

In this series of three Blog Posts, we discuss the key points addressed in the report, and in particular, the potential implications for businesses that are procuring, developing and/or deploying emerging digital technologies.

The current legal framework – issues and potential solutions

Below (and continued in Blog Post 3) are the potential changes to the existing legal framework recommended by the NTF  that we believe would have the most significant implications for businesses that are procuring, developing and/or deploying emerging digital technologies.

Wrongfulness and fault

 
Identification of duties

The NTF observes that emerging digital technologies have not yet developed well-established models of proper functioning, which is what duties in fault-based liability rules are created to uphold. Without these standards, it becomes difficult to create or apply these rules.

Additionally, the duties of care that are currently in place are designed for human conduct. As a result, the NTF notes that it will be very difficult to apply these to emerging digital technologies – in particular, AI – as the process of running these systems varies massively.

For example, if the operation of some technology that includes AI is legally permissible, and the developer made use of state-of-the-art knowledge at the time the system was launched, then any subsequent choices made by the AI technology independently may not necessarily be attributable to some flaw in its original design. The question therefore arises as to whether the choice to put it on the market, or implement the AI system in an environment where harm was subsequently caused, in itself could be a breach of the duties of care applicable to such choices.

 
The NTF comments that operators are already subject to a number of duties of care relating to: the choice of technology (in particular in light of the tasks to be performed and the operator’s own skills and abilities); the organisational framework provided (in particular with regard to proper monitoring); and maintenance (including any safety checks and repair). Failure to comply with such duties may trigger fault liability regardless of whether the operator may also be strictly liable for the risk created by the technology.

In our view, this highlights the importance for businesses to have in place a robust framework for due diligence and assessment of new technology before it is deployed and to maintain adequate ongoing oversight thereafter.

The NTF goes on to say that the more advanced technologies become, the more difficult it is for operators to develop the right skills and discharge all duties. While the risk of insufficient skills should still be borne by the operators, it would be unfair to leave producers entirely out of the scope of liability. The NTF therefore considers that producers should have to design, describe and market products in a way which will effectively enable operators to discharge their duties.

 
In our view, this principle could have a significant impact on the contractual scheme for emerging digital technologies. For example:

• suppliers may find it more difficult to support traditional exclusions: (1) of warranties as to fitness for purpose; or (2)that the operation of a particular system will be error free. Suppliers may react by explicitly stating narrow purposes for which a particular technology may be used in order to restrict the scope of their duty, and either prohibit or exclude liability for use by a customer outside those narrow purposes.
• it is common where a product is provided on a software-as-a-service (SaaS) basis for the contract not to exhaustively set out product features, as these are subject to regular change. Suppliers may seek to be more prescriptive in their product descriptions in order to discharge this duty.
• suppliers may require their customers to be more proactive in reporting faults and errors in order to adequately discharge their duty to monitor the product once in circulation.   

Strict liability

 
Strict liability typically only applies in cases of physical harm to persons or property, but not for pure economic loss.

The NTF observes that existing rules on strict liability for motor vehicles or aircraft could potentially be applied to autonomous vehicles or drones, but there are many potential liability gaps. On the other hand, strict liability for the operation of computers, software or similar is (so far) widely unknown in Europe, even though there are some limited examples where countries provide for the liability of the operator of some specific computer system (for example, databases operated by the state).

The NTF considers that strict liability is a difficult area in relation to emerging digital technologies, as there is a danger that the risk of liability may be perceived as a deterrent to technological research. The legislator must take this into account when considering the introduction of strict liability.

Proposed solution: operator’s strict liability

The NTF proposes that strict liability should not merely be introduced to new technology in general. Rather, it should be introduced to those emerging digital technologies that may typically cause harm comparable to the risks which are already subject to strict liability.

According to the NTF, strict liability should lie with the person who is in control of the risk connected with the operation of emerging digital technologies and who benefits from their operation (operator). Strict liability is an appropriate response to emerging digital technologies that are operated in non-private environments and that may typically cause significant harm.

 
In the NTF’s proposed solution:

• operator is described as “the person who is in control of the risk connected with the operation of emerging digital technologies and who benefits from such operation”.
• control is a variable concept which will range from activating the technology to determining output or result, and may include further steps in between. However, the more sophisticated and more autonomous a system, the less someone exercises actual control over the details of the operation. At this point, defining and influencing the algorithms may have a greater impact than just operating the system.
• more than one operator - with emerging digital technologies, there is often more than just one person who may, in a meaningful way, be considered to be “operating” the technology.
• frontend operator may be described as the owner/user/keeper.
• backend operator is the provider who, on a continuous basis, defines features of the technology and provides essential backend support services.

Where there is more than one operator (such as a frontend and a backend operator), the NTF takes the view that strict liability should be on the one who has more control over the risks posed by the operation. Relying on benefit, which is difficult to quantify, would lead to uncertainty. Initially, the frontend operator will have more control; but as emerging digital technologies become increasingly reliant upon backend support, it will make more sense to hold the backend operator as the person primarily in a position to control, reduce and insure the risks associated with the use of the technology. 

The NTF emphasises that, ideally, in order to avoid uncertainty, the legislator should define which operator is liable under which circumstances.

In our view:

• if there is a risk that a supplier is deemed to be a backend operator, then one would expect this to be reflected in the contractual scheme when procuring emerging digital technologies, especially on a SaaS basis (as noted above when considering fault-based liability regimes).
• the NTF’s proposed solution further highlights the need for robust due diligence and ongoing oversight in the deployment of these technologies, as businesses will need to understand the degree of control that they exercise with respect to a given solution. In particular, this will need to be considered in the early stages of systems design and architecture, as design decisions made at an early stage of a project could have ongoing liability implications.

The NTF observes that, as these schemes stand today in many Member States, they include a range of defences, exceptions and exclusions that may not be appropriate for emerging digital technologies, because they reflect a focus on continuous control by humans. According to the NTF, therefore, existing defences and statutory exceptions from strict liability may need to be reconsidered in the light of emerging digital technologies, in particular if these defences and exceptions are tailored primarily to traditional notions of control by humans.

Product liability

 
The NTF notes that the PLD was drawn up on the basis of the technological neutrality principle (broadly, that the law’s application and effect should not depend on the form of technology used). The NTF’s view is that some key concepts underpinning the existing EU regime, as adopted in 1985, are now an inadequate match for the potential risks of emerging digital technologies. This is because the regime was designed with traditional products and business models in mind – contemplating material objects placed on the market by a one-time action of the producer, after which the producer does not maintain control over the product. The NTF considers that emerging digital technologies put the existing product liability regime to the test in several respects – in particular in relation to what constitutes a “product”, a “defect” and a “producer”.

Products

 
The NTF considers that emerging digital technologies challenge the clear definition set out above.  For example:

• in AI systems, products and services permanently interact and a clear segregation of the two is not tenable. It is also questionable as to whether software would fall under this definition.
• although the PLD covers consumer property, it is not clear whether it covers damage to data, as data may not be an “item of property” within the meaning of Article 5, PLD.

Defectiveness

 
In the NTF’s view, emerging digital technologies raise a number of issues in relation to defectiveness:

• the interconnectivity of emerging digital technologies makes it hard to identify defectiveness.
• sophisticated AI autonomous systems with self-learning capabilities raise the question of whether unpredictable deviations in the decision-making path can be treated as defects; and, even if they do constitute a defect, whether the state-of-the-art defence may apply.
• the complexity and the opacity of emerging digital technologies complicate chances for the victim to discover and prove the defect and causation.

As the PLD focuses on the moment when the product was put into circulation as the key turning point for the producer’s liability, the NTF observes that this cuts off claims for anything the producer may subsequently add via an update or upgrade. Highly sophisticated AI systems may not be finished products that are put on the market in a traditional way (as the producer may retain some degree of control over the product’s further development in the form of additions or updates after circulation).

The NTF is also of the view that the producer’s control may be limited and non-exclusive if the product’s operation requires data provided by third parties or collected from the environment, and depends on self-learning processes and personalising settings chosen by the user. When a multitude of actors contribute to the design, functioning and use of the AI product/system, the traditional role of the producer is diluted.

Development risk defence

 
The NTF considers that this defence may become much more important practically with regard to sophisticated AI-based products.

Producer

 
It is the NTF’s view that fault liability, as well as strict liability for risks and for defective products, should continue to coexist.

Logging by design  

The NTF notes that emerging digital technologies may allow for the reliable and detailed documentation of events that may enable the identification of what has caused an accident. As a result, it would be desirable to impose a duty (where appropriate) to provide for appropriate logging and to disclose the data to the victim in readable format.

 
The NTF emphasises that any requirements should be suitable and proportionate for the goals to be achieved, and should take into account the technical feasibility and costs of logging, the values at stake, the magnitude of the risk, and any adverse implications for the rights of others.

Safety rules

 
The reverse of the burden of proof in relation to emerging digital technologies accounts for the information asymmetry typically found between those developing and producing emerging digital technologies, on the one hand, and third-party victims, on the other hand.

We continue our examination of some of the potential changes to the existing legal framework recommended by the NTF  that we believe would have the most significant implications for businesses that are procuring, developing and/or deploying emerging digital technologies in Blog Post 3.

For more information

For more information on emerging digital technologies, see the following Norton Rose Fulbright resources:

• MicroCertificate in Disruptive Technologies
• FinTech hub
• Inside Tech Law