Rofi Aulia Rahman, Rizki Habibulah The Criminal Liability of Artificial Intelligence: Is It Plausible to Hitherto Indonesian Criminal System?

The pace of technology evolution is very fast. The technology has brought us to the limitless world and becoming our ally in every daily life. The technology has created a visionary autonomous agent that could surpass human capability with little or without human intervention, called by Artificial Intelligence (AI). In the implementation of AI in every area that could be in industrial, health, agriculture, artist, etc. Consequently, AI can damage individual or congregation life that is protected by criminal law. In the current Indonesian criminal system, it just acknowledges natural person and legal person (recht persoon) as the subject of law that can be imposed by criminal sanction. Hitherto and near foreseeable future AI has a notable role in every aspect, which affects also criminal aspects due to the damage resulted. AI has no sufficient legal status to be explained in the Indonesian criminal system. In this paper, the author will assess whether the current criminal system of Indonesia can sue the criminal liability of artificial intelligence, and also will make it clear to whom the possibility of criminal liability of artificial intelligence shall be charged.

Artificial Intelligence is actually divided into 3 categories, which is Artificial Narrow Intelligence (ANI), Artificial General Intelligence (AGI), and Artificial Super Intelligence (ASI).

Artificial Narrow Intelligence (ANI)
:-To figure out the meaning of artificial intelligence, the first we need to explore the boundary between Artificial Narrow Intelligence (ANI, also called "weak" AI). This is essentially the state of computer software today. Computer programs are designed and mostly developed by human programmers with the process from input to output open to inspection. Programs typically specialize in one area and can be mathematically proven to be safe or friendly. 6 ANI systems can attend to a task in real-time, but they pull information from a specific data-set. As a result, these systems don't perform outside of the single task that they are designed to perform.
Artificial General Intelligence (AGI):-also called "strong" AI. For a majority of experts, AGI refers to an autonomous machine's ability to perform any intellectual tasks that a human can perform. This implies generalizing and abstracting learning 1 Gabriel Hallevy. (2018). Dangerous Robots -Artificial Intelligence vs. Human Intelligence. Available at SSRN: https://ssrn.com/abstract=3121905 or http://dx.doi.org/10.2139/ssrn.3121905 2 Yueh-Hsuan Weng, et.al. (2009). "Toward the Human-Robot Co-Existence Society:On Safety Intelligence for Next Generation Robots". International Journal of Society and Robot, 267 (1): 273 3 Colin Fernandez, Robot kills factory worker after picking him up and crushing him against a metal plate at Volkswagen plant in Germany, DAILY MAIL (Jul. 1st, 2015, 22.44 BST), https://www.dailymail.co.uk/news/article-3146547/Robot-kills-man-Volkswagen-plant-Germany.html 4 Raquel Echanique, When Bad Robots Kill Good Scientists, available at https://medium.com/theinternationalists-journal/when-bad-robots-kill-good-scientists-9f77f9935b92 (Dec. 26th, 2018) 5 Andrew Griffin, Facebook's Artificial Intelligence Robots Shut Down After Start Talking To Each Other In Their Own Language (Jul. 31st, 2017, 17.10), https://www.independent.co.uk/life-style/gadgets-andtech/news/facebook-artificial-intelligence-ai-chatbot-new-language-research-openai-google-a7869706.html 6 Jens Pohl. (2017). "Artificial Superintelligence: Extinction or Nirvana?". InternSymp, p.2 across various cognitive functions. Transferring learning autonomously and nimbly from one domain to another has happened only very embryonically thus far, 7 Artificial Super Intelligence (ASI):-ASI, that is expected to be many orders of magnitude more intelligent than the human. The level of artificial intelligence that has been reached when a computer has exponentially surpassed the intelligence level of a human by several orders of magnitude. 8 All types of artificial intelligence around us today are narrow AIs. Natural language can control tools such as Google Assistant, and Siri are the examples of Narrow AIs'. Although these tools may be considered as "weak" because of their ability to interact with us, Those machines say 'weak' because they don't have some capabilities like human intelligence, such as they lack of awareness, consciousness, and genuine intelligence that can interact with human intelligence. In the other word, they cannot think for themselves.
This explains why when we ask to Google Assistant or Siri on how to approach personal problem or the meaning of life and more abstract question, we obtain the answer that doesn't make sense. On the other hand, when we ask Siri what the weather is, then it has an accurate answer. That is because it answers basic questions about weather outside is within the range of intelligence designed to operate in.
As humans, we have the ability to assess the environment and respond emotionally to those situations. AI around us has no sense and flexibility to think like us. This is considered as a weak AI, except that self-driving car is created by several ANI systems combined become one.
Oxford Philosopher Bostrom says "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest". Artificial Surpass Intelligence (ASI) will surpass human intelligence in all aspects, from creativity to general wisdom, to the capability to solve any problem like felt by human. This kind of AI what Elon Musk said will lead the human race and the people is worrying about it. 9 Typically, in making Artificial Intelligence, there are 2 main components that is needed, they are algorithm and big data. Algorithms are a series of stages that have rules that help us solve problems and achieve our goals. And when we compile these steps and make the right rules, we can automate the algorithm to build artificial intelligence (A.I). And A.I. it helps us focus on what we do well. 10 While big data is a collection of data that becoming a source to search everything we need.

Landmark Cases of AI a. Saudi Arabia Bestows Citizenship On A Robot Named Sophia
Sophia robot was activated on April 19, 2015, by the Hanson Robotic Company. This robot was introduced because it has the same appearance and behavior as humans with previous robot variants. According to the person who designed it, David Hanson, Sophia uses artificial intelligence, visual data processing, and facial recognition. Sophia also imitates human gestures and facial expressions and be able to answer certain questions and conduct a simple conversation on a topic that has been determined. In October 2017, the Sophia robot became a citizen of Saudi Arabia. Sophia is the first robot to bestowed citizenship from a country. 11 On October 11, 2017, Sophia was introduced to the United Nations with a brief conversation with the Deputy Secretary-General of the United Nations, Amina J. Mohammed. On October 25, at Future Investment Summit in Riyadh, this robot won Saudi citizenship, becoming the first robot to have citizenship 12 . This attracts controversy because some commentators wonder whether this implies that Sophia can vote in elections or marriage, or whether deliberate termination of the system can be considered murder.
Sophia robot is one evidence of the progress of artificial intelligence that is increasingly developing along with the times. Sophia is not only designed to do certain jobs but also gets citizenship status from a country. The question is whether Sophia has rights and obligations as well as other citizens in that country. Basically, every citizen has a responsibility if there is an unlawful action that results in a criminal act. Should if Sophia robot does a criminal act then there must be a punishment given, but the problem is whether Sophia can be a legal subject because she is a robot.

b. Robot Killed Worker
By developing and advancing in the field of robotics, there is a unique problem that is killing someone caused by a robot. The fact that robots don't have their own thoughts and do whatever they were programmed to do. Problems will arise when robots come in contact with humans and cannot distinguish how to treat humans and like whatever material it was working on. This is why many accidents and deaths related to robots occur in factories where robots work and are very close to humans.
Many cases of death caused by robots to workers in a factory. There are some cases of death and injuries caused by robots to humans who work side by side in one factory:

1) Robot Killed Volkswagen Worker In Germany
In 2015, the 22-year-old contractor was part of a team that was setting up the automated machinery at the factory in Kassel, north of Frankfurt, was pulled automatically by a robotic arm and collided with a steel plate. The incident, at the Volkswagen plant in Germany, is believed to be the first death in Europe caused by industrial robots. 13 Heiko Willig, the spokesman for Volkswagen Germany, said the alleged death was due to human error rather than imposing a mistake on robots who could do the work according to what was programmed to them. He said it usually operates in a limited area at the factory, taking car parts and manipulating them. The type of robot that crushed the employee is usually kept in a cage. The man was working on the robot inside the cage when he was grabbed. 14 (Daily Mail -UK and The Guardian).

2) Robot Killed Workers at Ford Factory in Michigan
Robert William was the first person in history to have been killed by a robot. The accident occurred at the Ford factory in Michigan, United States on January 25, 1979. William was killed when he climbed a shelf to take a cast then the robot's arm hit him and killed William. Ironically this happens after the robot gives incorrect information about the number of casts on the shelf and forces William to climb. The jury decided that the security measures were weak and His family was awarded $10 million. 15 (Wired).

c. Blocking Bot Account Spread Hoax and Discrimination by Ministry of Communication and Information in Indonesia
Throughout 2016, the Directorate of Criminal Investigation detected thousands of social media and online media accounts that spread hoax information, and provocation of discrimination. There are 300 of social media and online media accounts have been blocked. The police submitted a request for blocking to the Ministry of Communication and Information Director of Directorate of Criminal Investigation said, "We propose blocking because it is related to the content of hoax information and discrimination, then the robotic accounts or bot accounts that are also provocative are also blocked, and blocking is on the web service provider or internet network service provider. 16 The motive for the spread of hoax is for political interest. When Indonesian has increasingly use social media as a daily intake of news, political parties and interest groups try to expand their influence through the digital stage. This is certainly no surprise, as with other Southeast Asian countries, the involvement of political parties and interest groups in social media discourse in Indonesia has begun to adopt expressions of war and social media has been armed with cyber army.
In fact, a globally covered study by researchers from the Oxford Internet Institute has identified a tidal wave of "bots" and fake social accounts that are deliberately used to spread disinformation. In Indonesia, "black campaigns" based on lies and slander are increasingly being generated and spread online. The appearance of the bot account is very troubling because it not only spreads incorrect information but also makes public chaos because it will provoke public opinion that makes a difference about what the bot spread to the society.

Hitherto Criminal Law System in Indonesia a. Natural Person
The current Criminal Code has not yet regulated on the criminal responsibility of corporations in the sense that they have not known the corporation as the subject of a criminal act. The Criminal Code used up to now still adheres to the understanding that an offense can only be done by humans (naturalijk person). Article 59 of the Criminal Code, "in case where by reason of misdemeanor punishment is imposed upon director, members of a board of management or commissioners, no punishment shall be pronounced against the director or commissioner who evidently doesn't take any part in the commission of misdemeanor". The meaning is that article is that the crime was never committed by the corporation but was carried out by the management or person. The Criminal Code only regulates criminal acts carried out by individuals whose responsibilities are also carried out individually. 17 Elements of "whosoever"/barangsiapa/setiap orang becomes a legal subject to whom he can be imposed by existing laws due to their unlawful act. That based on the facts revealed in the trial that the defendant as a person who has been indicted by the Public Prosecutor for committing a crime and the defendant confessed in front of public prosecutor. The defendant is physically and mentally healthy and can answer and hear every question posed to him so that the defendant is legally capable of being accountable for his actions, based on this description the elements of whosoever/barangsiapa/setiap orang has been fulfilled.
This element looks more at the bestandeel pointers that pointing to the perpetrators/subjects of criminal acts, namely people and corporations, namely person (naturlijke persoon) and corporations as legal entities (recht persoon). Prof. Sudikno Mertokusumo, he said that "the legal subject (subjectum juris) is anything that can obtain, have or bear the rights and obligations of the law, which consists of: people (natuurlijkepersoon), and legal entity (rechtspersoon)". 18 In each formulation of the articles of the Criminal Code, the element "whosoever" is an important word in proving criminal unlawful act and liabilities. As a word "whosoever" it requires a study that is quite serious in the principle of unlawful act and criminal liability in an effort to prove before the court.
The term of whosoever/barangsiapa/setiap orang in this situation is for the person who has a capability to conduct their rights, health physically and mentally where this person has commit a crime intentionally or intentionally. In addition, a corporate entity can also be an actor of whosoever in this circumstance. This is in line with the Law Number 31 of 1999 concerning Corruption Eradication, which is regulated in the special act since it is impossible to imposed corruptor under criminal code to corporate liability. Article 1 Paragraph 1 stated that a corporation shall be an organized group of persons and/or assets both as legal entities as well as non-legal entities. It means that corporate has been a subject of law and consequently it can be imposed under criminal law.

b. Legal Person
Every human being is a subject of law and is able to carry out legal actions or establish legal relations that must be followed by the existence of legal skills (rechsbekwaamheid) and legal authority (rechtsbevoedgheid). Besides anti-corruption act recognizes legal entity as a subject of law, also Article 1 number 1 of Law Number 40 of 2007 concerning Limited Liability Companies said so, which states that Limited Liability Company, hereinafter referred to as the Company, means a legal entity constitutes a capital alliance, established based on an agreement, in order to conduct business activities with the Company's Authorized Capital divided into shares and which satisfies the requirements as stipulated in this Law, and it implementation regulations.
The determination of the corporation as a subject and also has responsible for its motivation has to pay attention to the development of the corporation itself. Every level of its position within corporation shall be aware and careful that it is not impossible that corporation can be punished besides imprisonment also by fine that could be higher under the criminal law.
Corporations can only carry out deeds through the interventions of their administrators. Thus, the external factor (actus reus) in the corporation depends on the relationship between the corporation and its stakeholders. Unlawful acts committed by corporations are always inclusion that can be punished. In this case, the position of the corporation has always been part of the inclusion of the crime. According to Chairul Huda, "it is not possible for corporations to be the sole perpetrators of criminal acts. Corporations can be a perpetrator (dader) but cannot be the subject which has an authority to appoint and tends to violate the law (pleger) that is supposed to be the person (manager, director, etc).
After knowing that corporate has a potential to commit a crime, the person who has big responsibility to be imposed by criminal law is the leader of that company, and the second probability is the executive staff, the person who has been appointed by the leader to conduct their duties. In the subject of human law conditions (internal /mens rea) unlawful acts are determined from the psychological state of the perpetrator, namely the normal inner or intention. Unlike the case with the error requirements (internal) in the corporation. In corporations the error requirement is seen from whether the corporation has made it possible to avoid criminal acts as part of its policy in running a business. If this obligation is not fulfilled, the corporation can be denounced if a crime is committed. This is related to the condition of committing a crime in the corporation in the literature called the terms of power.
Muladi's view of the terms of power in corporations includes the authority to regulate / control and / or govern parties who in fact carry out prohibited acts; able to carry out its authority and basically able to make decisions about the matter concerned and able to seek policies or security measures in order to prevent the conduct of prohibited actions. 19 In terms of imposing criminal liability on corporations, Sutan Remy Sjahdeini 20 put forward four systems that could be enforced: 1) Corporate management as the perpetrator of a crime, so that the management is responsible for carrying out criminal responsibility. 2) Corporations as perpetrators of criminal acts, but administrators must bear criminal responsibility.

3) Corporations as perpetrators of crimes and corporations themselves must bear criminal responsibility. 4) Managers and corporations are both perpetrators of criminal acts, and both of them must bear criminal responsibility.
At present the Criminal Code adheres to the first system, the Criminal Code holds that because corporations cannot do themselves an act which is offense and cannot not have a wrong mens rea but the one who performs the action is a corporation that has mens rea. There are two basic teachings which are the reasons for justifying the imposition of criminal responsibility on corporations. These teachings are doctrine of strict liability and doctrine of vicarious liability. Doctrine of strict liability, is a form of criminal liability can be charged to the offender the offender concerned with no need to prove the existence of errors (intentional or negligent) on the perpetrators. This in legal terms in Indonesia is known as absolute accountability, also Paul Dobson said that these are some crimes for which with regard to at least one element of the actus reus, no mens rea is required.
In relation to corporations, corporations can be liable for criminal liability for offenses that are not required for mens rea for liability, based on doctrine of strict liability. Doctrine of vicarious liability, in terms of Indonesian law known as substitution liability, which is the imposition of criminal liability from offenses committed, for example by A to B. According to this doctrine, if an agent or corporate worker acts within the scope of his work and with the intention of getting unlawful beneficial for the corporation, he is committing a crime, his criminal responsibility can be borne by the company. Because a corporation is vicariously liable for the acts of its employees or agents in normal situations of vicarious liability. It does not matter whether the company actually gains or not, or whether the activity has been banned by the company or not.
It is very difficult to ascertain whether vicarious liability can be applied in each case, the question is whether vicarious liability has a strong basis for holding corporations accountable. The reasons that support vicarious liability are mostly pragmatic. By traversing all the problems that have to do with other doctrines, such as finding people who are quite important in a corporation that has committed a crime. With this doctrine, the company has an obligation to their worker or employee to be careful and aware for everything they do, since every conduct must be in line with law, if not, then the criminal sanction awaits.

New Subject of Law: Future Legal Subject on AI a. New Person
In Fact, one of the problems in criminal law is currently facing a criminal liability of legal person. Should criminal penalties solely be applied to individuals or criminal penalties can also be applied to legal persons as legal subjects and in what would way can that occur? 21 Criminal Liability of legal persons appears not only important and as an adequate response from criminal law to improve complex human society but also meets requirements imposed by various international bodies such as the European Union, Council of Europe, and United Nation. 22 Which requires the state to adopt these regulations to provide criminal liability of legal persons related to certain criminal acts.
However, it cannot be ignored that we live rapidly evolving marked by global risk discourse, which entails profound a paradigm shift in culture, economy, social and technology. There is an increase in criminality that involves a greater complexity and organizations such as corporations, societies and also associations as key factors. If we chose to only prosecute and punish physical persons acting on behalf of legal entities, it means that giving a decision is not only a criminal act against that person. But also provides action on the organizational structure of that legal entities. Thus assuming that there is a needs to be a concrete and autonomous criminal liability of legal persons.
How can we accept dogmatic obstacles that are upheld by traditional thinking of criminal law. Refusing the argument of the inability of agency and mens rea of legal entities is a consideration of the application of criminal liability of legal persons in the implementation using the analogic models. 23 The individuals can be replaced as criminal liability both objectively or subjectively, ethics or social relations by their collective work and manifesting it as legal entities, associations, groupings or corporations which are free to express themselves.
The legitimacy of this type of criminal liability is based on an analogy from a natural person and legal person. For example, children under the age of 16 even though they can act when the child commits a criminal act, the child is free from criminal liability because according to article 45 of the Criminal Code children under the age of 16 cannot be given criminal liability. It is not entirely unreasonable to provide criminal liability for legal persons, despite a legal person cannot act physically and anthropologically. The law enforcement of the axioms system through developing by criminal dogmatic. 24 Dogmatically, a 16-year-old child cannot be punished because he is deemed unable to accept criminal liability.
If in that case we have the curtailment of ontological segments of action and limiting or removing mens rea, the notion blame is we can reconstruct the legal persons can be the center of accusations. In other words, if a person represents the body and mind of the legal entity in acting then the action must be recognized as the action of the legal entity itself. Legal entities accept responsibility for these actions through directors, supervisors, etc.

b. Future Legal Subject for AI
In order to cope with these same problems as they relate to humans, society devised criminal law. Criminal law embodies the most powerful legal social control in modern civilization. People's fear of AI entities, in most cases is based on the fact that AI entities are not considered to be subject to the law, specifically in criminal law. In the past, people were similarly fearful of corporations and their power to commit a spectrum of crimes, but because corporations are legal entities subject to criminal and corporate law, which kind of fear has been significantly reduced.
If criminal acts are carried out by a lack of mental abilities, children or animals, the offender is considered as an innocent because they are lack of mental ability to explicitly commit a crime. However, for example, if the innocent person receives an order from someone else, if the dog owner orders him to attack someone, the owner is liable to be punished under criminal law system. This analogy can be happened in that AI does not have a mens rea to become a legal subject, but it can be used as an analogy like the case before, then the programmer or user of AI can be subject to criminal liability if there are criminal action conducted by AI.
Using AI is relatively simple. If there is a system that takes action that causes crime or if there is an obligation, criminal action will occur again. 25 For example if an AI is activated by a person to commit a crime, as an example someone makes a robot as a murderer then the action has fulfilled the actus reus element because the robot committed a criminal act of killing another person and the AI maker who intended as criminal liability. There is a legal principle that recognized by the Indonesian criminal system, that is actus non facit reum nisi mens sit rea, which means an act does not make guilty, unless the mind be guilty.

c. Criminal Liability Models
Hallevy 26 gives three legal models by which offences committed by AI systems might be considered: 1) Perpetrator-via-another. If an offence committed by a mentally deficient person, a child or an animal, then the perpetrator is held to be an innocent agent because they lack the mental capacity to form a mens rea (this is true even for strict liability offences). However, if the innocent agent was instructed by another person (for example, if the owner of a dog instructed his dog to attack somebody), then the instructor is held criminally liable. According to this model, AI programs can be considered as software programmers or users as perpetrators and not innocent representatives. 2) Natural-probable-consequence. This model, part of AI programs for purposes that are either incorrectly activated and commit criminal acts. Hallevy gives an example case on a men killed by artificial intelligence robot made by a Japanese motorcycle factory where the robot was working near the worker. Robots mistakenly recognized the employees as a threat, and the most efficient way to eliminate this threat is to smash it by his hydraulic arms to the adjacent operating machine. The robot uses a very strong hydraulic arm to kill workers. The general law objective that is 25 Ibid 26 Ibid, p.174 responsible for "natural or probable consequences" is to prosecute accomplices. If there is no proof of conspiracy, as long as the suspect knows that a criminal plan is underway, the perpetrator's criminal actions are still reliable or can be done legally. Therefore, users or programmers can be legally responsible if they know that crime is a natural result and is flammable by using programs/videos. However, the application of this principle must distinguish AI programs (such as programs for conducting criminal plans). Crimes especially means rea that require the knowledge cannot be prosecuted for the latter group of program (but it turns out there are indeed 'rational people' mens reaor by using strict liability, can). 3) Direct Liability. This model needs actus reus and mens rea to an AI system. It is relatively simple to match actus reus to an AI system. If an AI has conducted something that results a criminal act, or fails to take an action when there is a duty to act due to something error or by his will then he commit a crime, the actus reus of an that kind of violation has met. In the other hand, assigning a mens rea is quiet tough, and so it is here that the three levels of mens rea become important thing. For strict liability offences, where no intent to commit an offence is required, it may indeed be plausible to hold AI programs that can be imposed and sued under criminal law system. By taking the case of self-driving cars, speeding is a strict liability offence; so according to Hallevy, if a self-driving car was found to be breaking the speed limit for the road it is on, the law may well assign criminal liability to the AI program that was driving the car at that time. This possibility raises a number of other issues that Hallevy touches on, including defences (could a program that is malfunctioning claim a defence similar to the human defence of insanity? Or if it is affected by an electronic virus, could it claim defences similar to coercion or intoxication?); and punishment (who or what would be sentenced for an unlawful act for which an AI system was directly liable?).
If all certain requirements are met, a criminal court can impose them under criminal law system. Natural person, legal entities, and AI Robots may be part of subject of law that can be imposed as whosoever/barangsiapa/setiap orang. Rapid development of technology notably artificial intelligence needs more focus for having a legal solution or at least the law can adjust toward the development of AI that is not impossible in the near future we will have artificial super intelligence. After knowing several types of AI, that is narrow AI, general AI, and Super AI, we can consider the capabilities first among them to which model that is suitable and applicable for each AI, and it's going to be different way of legal thinking if the perpetrator is super AI, maybe on the criminal procedure law, this super AI can be seen as like as human ones since considering its capabilities are above human skills. It is not impossible as well that in the future, in the earth, we will have special court for super AI robot.

C. CONCLUSION
In current Indonesia criminal system, that AI cannot be executed under the criminal punishment. It still needs a natural person or legal person to responsible if there is a criminal offenses resulted. But in the future, if AI has been more sophisticated and close to be a human, shall AI be regulated like human ones? That is the first question will arise and be an issue to debate. Somehow, AI will stand and exist and say as the last invention of mankind that maybe several years later could create super AI which maybe can do whatever they want and interact autonomously like human being. This situation shall be prevented and regulated in the future and must be taken into discussion. At least, for now there are three models to involve AI in criminal law system notably in Indonesia, first, using Perpetrator-via-another, second, using Natural-probable-consequence, and last, using direct liability. In the future, legal expert must think further and one step ahead to regulate every probabilities of technology that could change the society and prevent the risk or harmful in every aspect by modifying the criminal code, and procedure law to adjust AI development.