The killer robots? Stop, even the good robots are unpredictable to the horror

The Heads of more than hundred leading companies in the field of artificial intelligence is very concerned about the development of "killer robots". In an open letter to the UN these business leaders, including Elon musk of Tesla and the founders of Google DeepMind, has warned that the use of technology Autonomous weapons could be taken by terrorists and despots, or in varying degrees will get hacked.

But the real danger is much more serious – and it is not only human behaviour but also in the engine. The study of complex systems shows that they can behave much more unpredictable than it can be concluded from the sum of individual actions. On the one hand this means that human society can behave in a completely different way than you might expect, studying the behavior of individuals. On the other hand, this also applies to the technology. Even ecosystem simple artificial intelligence programs – which we call stupid, good bots can surprise us. Even the individual bots can behave horribly.

The Individual items that comprise complex systems such as economic markets or global weather, tend not to interact in a simple linear way. This makes these systems very difficult to model and understand. For example, even after years of research climate it is impossible to predict long-term behavior of weather. These systems are so sensitive to the smallest changes in how explosive they react. It is very difficult to know the exact state of such a system at a particular point. All this makes these systems internally unpredictable.

All of these principles apply to large groups of people acting in their own way, be it human society or group of bots with artificial intelligence. Recently, scientists studied one of the types of complex system in which good bots are used to automatically edit articles on Wikipedia. These different bots are designed, written and used by trusted Wikipedia editors, and their underlying software is open source and available for everyone. Individually, they have a common goal to improve the encyclopedia. However, their collective behaviour turned out to be surprisingly ineffective.

These bots with Wikipedia are well-established rules and conventions, but since the web site has no Central control system, there is no effective coordination between the people running different bots. The results showed a couple of bots, who for several years had canceled the edits to each other and no one noticed. And, of course, because these bots can not learn, they did not notice.

These bots are designed to speed up the editing process. But small differences in design between bots or people that use them, can lead to massive waste of resources in the ongoing "war for the edit", which would be solved much faster with the help of editors.

The Researchers also found that the bots behave differently in different language versions of Wikipedia. Rules, it would seem, is almost the same, the goals are identical, the technology is similar. But in the German Wikipedia bots cooperation was much more efficient and productive than, for example, in Portuguese. This can only be explained by differences between editors-people in charge of these bots in different environments.

Exponential confusion

Bots Wikipedia have broad autonomy and the system is working in accordance with the purposes of the individual bots. But the Wikimedia Foundation plans to use AI that will give more autonomy to these bots. And this will likely lead to even more unpredictable behavior.

A Good example of what can happen, showed bots, created to talk to people when they were forced to communicate with each other. We are no longer surprised by the answers of personal assistants like Siri. But make them communicate with each other and they quickly start to behave in unexpected ways, to argue and even insult each other.

The more the system becomes and the more Autonomous each bot becomes, the more complex and unpredictable will be the future behavior of the system. Wikipedia is an example of the work of a large number of relatively simple bots. Example of chat-bots is an example of a small number of relatively sophisticated and ingenious bots – in both cases, there are unforeseen conflicts. The complexity and hence unpredictability grow exponentially as you add to the system of individuality. So when in the future there will be systems with a large number of very complex robots, the unpredictability of them will go beyond our imagination.

Blazing madness

Self-driving cars, for example, promise to make a breakthrough in efficiency and road safety. But we still don't know what happens when we have more wild the system is fully Autonomous cars. They may behave differently even within a small fleet of individual vehicles in a controlled environment. And even more unpredictable behavior can occur when self-driving cars, "trained" by different people in different places will begin to interact with each other.

People can adapt to new rules and agreements relatively quickly, but difficult to switch between systems. Artificial agents can be even harder. For example, if the "trained in Germany" car, for instance, go to Italy, we don't know how he will learn unwritten cultural agreement, which is followed by many other "trained in Italy" cars. Something mundane like crossing the intersection can be mortally risky, because we just don't know whether the machines to interact properly, or begin to behave unpredictably.

Now think of the killer robots that are bothering the Mask and his colleagues. One killing machine can be very dangerous in the wrong hands. And the system unpredictable killer robots? Think for yourself.