How did the neural network?

Date:

2017-04-22 21:30:11

Views:

2969

Rating:

1Like 0Dislike

Share:

How did the neural network? Source:

Over the past 10 years, due to the method of so-called deep learning, we have the best artificial intelligence system — for example, speech recognizers on smartphones or the latest automatic translator Google. Deep learning, in fact, has become a new trend in the already known neural networks, which were in Vogue and went on for more than 70 years. For the first time neural network proposed Warren McCullough and Walter Pitts in 1994, two researchers from the University of Chicago. In 1952 they went to work at the Massachusetts Institute of technology, to lay the groundwork for the first Department of cognitive science.

Neural networks were one of the main directions of studies in neurobiology and computer science until 1969, when, according to legend, killed of mathematics Massachusetts Institute of technology Marvin Minsky and Seymour Papert who in a year became co-managers of the new artificial intelligence lab, MIT.

The Revival of this method has experienced in the 1980s, slightly faded in the first decade of the new century with a fanfare back in the second, on the crest of an incredible development of graphics chips and processing power.

"it Is believed that the idea of science — as the epidemic of the virus," said Tomaso Poggio, a Professor of cognitive science and brain science at MIT. "There is, apparently, five or six major strains of flu viruses, and one of them returns with surprising frequency in 25 years. People get infected, become immune and not get sick the next 25 years. Then there is a new generation, ready to be infected by the same strain of the virus. In science, people fall in love with the idea, it's driving everyone nuts, then beaten to death and become immune to it — I get tired of it. The ideas should be similar to the frequency".

the

Weighty issues

Neural networks are a method of machine learning where the computer learns to perform certain tasks, analyzing the training examples. As a rule, these examples are labeled manually in advance. The system is object recognition, for example, can absorb thousands of labeled images of cars, houses, coffee cups and so forth, and then will be able to find visual images in these images that consistently correlate with specific labels.

A Neural network is often compared to the human brain, which also has such networks, consisting of thousands or millions of simple processing nodes that are closely linked. Most modern neural networks are arranged in layers of nodes, and the data pass through them in only one direction. A separate node may be associated with multiple nodes in the layer beneath it, from which it obtains the data and multiple nodes in the layer above, in which it sends data.

Each of these incoming links node assigns a number — the "weight". When the network is active, the node receives different data sets — different numbers for each of these compounds and multiplies by the corresponding weight. It then sums the results to form a single number. If this number is below a threshold, the node does not transmit data to the next layer. If the number exceeds the threshold, the node is "activated" by sending the number — the sum of the weighted input data on all outgoing connections.

When the neural network trains, the weights and thresholds are initially set at random. Training data is provided in the lower layer — the input — and pass through the next layers, stacking and multiplying in a complex manner, until they finally arrive, already transformed, in the output layer. During training the weights and thresholds are constantly adjusted until the training data with the same label would not give similar conclusions.

the

Minds and machines

Neural networks are described by McCullough and Pitts in 1944, and had the thresholds and weights, but were not organized in layers, and scientists do not ask any specific learning mechanisms. But McCullough and Pitts showed that neural network could, in principle, calculate any function like any digital computer. The result was more of a neuroscience than computer science: it was necessary to assume that the human brain can be viewed as a computing device.

Neural networks continue to be a valuable tool for neurobiological studies. For example, the individual layers of the network rules or settings of the weights and thresholds reproduced the observed features of human neuroanatomy and cognitive functions, and thus touched on how the brain processes information.

The First trainable neural network, "Perceptron" (or "Perceptron"), has been shown by Cornell University psychologist Frank Rosenblatt in 1957. The design of the "Perceptron" was similar to the modern neural network, except that it had one layer with adjustable weights and thresholds between the input and output layers.

"Perceptrons" were actively studied in psychology and computer science until 1959, when Minsky and Papert published a book entitled "Perceptrons", which showed that the work of everyday computing with a perceptron was impractical from the point of view of time expenses.

"of Course, all restrictions would disappear if you make the machines a little more complicated", for example, in two layers," says Poggio. But while the book had an inhibiting effect on the study of neural networks.

"These things should be considered in a historical context," says Poggio. "The proof was built for programming languages such as Lisp. Shortly before this, people used analog computers. It was not clear at that time what will generally result in programming. I think they went a little overboard, but, as always, it is impossible to divide everything in black and white. If we consider it as a contest between analogue computation and digital computation, then they fought because it was necessary."

the

Frequency

1980 years, however, scientists have developed the algorithms to modify neural network weights and thresholds, which were quite effective for networks with more than one layer, eliminating many of the constraints defined Minsky and Papercom. This area experienced a Renaissance.

But with a reasonable point of view in neural networks was lacking something. Long enough exercise could lead to a revision of the network settings until it will start to classify data in a useful way, but what these settings mean? What are the features of the image looks a Discerner of the objects and how he collects parts to form a visual signature cars, houses and cups of coffee? Learning weights of individual compounds does not respond to this question.

In recent years, computer scientists began to think of ingenious methods to determine the analytical strategies adopted neural networks. But in 1980-ies the strategy of these networks was unclear. Therefore, at the turn of the century the neural network were driven vector machines, an alternative approach to machine learning based on clean and elegant mathematics.

The Recent surge of interest in neural networks — the deep learning revolution — bound gaming industry. Complex graphics and fast-paced modern video games requires hardware that can keep up with the trend, which introduces the GPU (graphics processor) with thousands of relatively simple processing cores on a single chip. Very soon scientists realized that the architecture of the GPU is perfectly suited for neural networks.

Modern GPUs has allowed him to build a network of 1960-ies and two - and three-layer networks of the 1980s in bunches of 10, 15 and even 50-layer networks today. Here is responsible for what the word "deep" in "deep learning". The depth of the network. Currently, deep learning is responsible for the most effective system in almost all areas of artificial intelligence research.

the

Under the hood

The opacity of the networks are still concerned about the academics but on this front there is progress. Poggio directs a research program on the topic of the theoretical foundations of intelligence. Not so long ago, Poggio and his colleagues have produced a theoretical study of neural networks in three parts.

The First part which was published last month in the International Journal of Automation and Computing, addressed to the range of computations that can conduct network, deep learning, and that, when the deep web have advantages over the shallow. Parts two and three, which were released in the form of reports addressed to the problems of global optimization, that is, guarantee that a network will find the settings that best fit its training data, as well as cases when the network is so well aware of the specifics of the training data that may not generalize to other manifestations of the same categories.

There is still a lot of theoretical questions, the answers to which will have to give. But there is hope that neural networks can finally break the cycle of generations, which plunge them in hot, then cold.

...

Recommended

Can genes create the perfect diet for you?

Can genes create the perfect diet for you?

Diet on genotype can be a way out for many, but it still has a lot of questions Don't know what to do to lose weight? DNA tests promise to help you with this. They will be able to develop the most individual diet, because for this they will use the m...

How many extraterrestrial civilizations can exist nearby?

How many extraterrestrial civilizations can exist nearby?

If aliens exist, why don't we "hear" them? In the 12th episode of Cosmos, which aired on December 14, 1980, co-author and host Carl Sagan introduced viewers to the same equation of astronomer Frank Drake. Using it, he calculated the potential number ...

Why does the most poisonous plant in the world cause severe pain?

Why does the most poisonous plant in the world cause severe pain?

The pain caused to humans by the Gimpi-gympie plant can drive him crazy Many people consider Australia a very dangerous place full of poisonous creatures. And this is a perfectly correct idea, because this continent literally wants to kill everyone w...

Comments (0)

This article has no comment, be the first!

Add comment

Related News

Scientists found out why the AI might be racist and sexist

Scientists found out why the AI might be racist and sexist

the Failed experiment of Microsoft, with its AI algorithm Tay (Tay), who within 24 hours after the beginning of interaction with people from Twitter has turned into a hardened racist, showed that the newly emerging AI systems can ...

SETI: aliens don't make contact, but we'll still find

SETI: aliens don't make contact, but we'll still find

Some researchers from the Institute for the search for extraterrestrial intelligence (SETI) believe that the best way of detecting aliens is scanning the cosmos and the search for laser beams. In one of the last major studies of t...

Astronomers have found another

Astronomers have found another "perfect" exoplanet

nowadays, astronomers are beginning to find potentially habitable exoplanets, but the latest find could be the most intriguing. The planet is called LHS 1140b, it is a bit more Land and is located approximately 40 light years from...

LHCb found new hints on possible deviations from the Standard model

LHCb found new hints on possible deviations from the Standard model

the LHCb Experiment, which operates in the framework of the Large hadron Collider at CERN, showed a curious anomaly in the decay of certain particles. If this information is confirmed we will receive new physical phenomena not pre...

Physicists have created a

Physicists have created a "negative mass"

Physics of the University of Washington have created a fluid with negative mass. Push it, and unlike all the physical objects in the world that we know, it will not accelerate in the direction of the push. It will accelerate in th...

Supermassive black holes in the Universe may be two times larger than previously thought

Supermassive black holes in the Universe may be two times larger than previously thought

a New discovery by astronomers has the potential to double the number of supermassive black holes in the Universe. Science for a long time it was thought that supermassive black holes usually exist only in the centers of large gal...

Scientists have discovered a terrifying new species of molluscs

Scientists have discovered a terrifying new species of molluscs

the Discovery of the animals is always very interesting and important scientific event. Scientists believe that in the future they will find many incredible surprises that nature at this day concealed from mankind. To take at leas...

Create a drug that causes dangerous bacteria to destroy their own DNA

Create a drug that causes dangerous bacteria to destroy their own DNA

the Sustainability of some species of pathogenic bacteria to existing antibiotics every year. Scientists from different countries trying to come up with universal medicine to which the bacteria would not get used so quickly, but s...

Artificial intelligence will be a threat only when stupid use

Artificial intelligence will be a threat only when stupid use

the technologies are based on continuous improvement over time. As you progress, technology entice and humanity. However, there is a fear that surrounds technologies such as artificial intelligence (AI) and robotics, partly becaus...

The next step in AI is to teach machines to think like us

The next step in AI is to teach machines to think like us

When you think about the "incredible" task, which can handle a computer, the first thing to come to mind of complex computations in a short time or analysis of massive volumes of data — something you will never be able to solve th...

Chinese surgeons grow girl new face on her breast

Chinese surgeons grow girl new face on her breast

We wrote about how Chinese doctors decided , and then sew it into place. People have waited, I understood, after which the doctors decided to move on to more complex operations. From sepsis 27-year-old girl in childhood is almost ...

NASA: Enceladus has all the conditions for the origin of life

NASA: Enceladus has all the conditions for the origin of life

Scientists at NASA reported the discovery of new evidence on the icy moon of Saturn, Enceladus conditions to sustain life. In escaping into space from the moon's surface geysers researchers have discovered a molecule of hydrogen. ...

Astronomers photographed a black hole. Possible

Astronomers photographed a black hole. Possible

This is not a photo of a black hole. This is artistic representation the Project to create a virtual planetary telescope Event Horizon is fully completed and the system has already started to work, providing astronomers with a gia...

Elon Musk is ready to present the plan for the merger of brain and computer

Elon Musk is ready to present the plan for the merger of brain and computer

it is known that Elon musk plans to create a company that will deal with the merger of brain and computer (Yes), will be announced on Monday, April 17, in a giant article on the website Wait But Why. Will the presentation Tim urba...

What are the similarities of artificial intelligence and electricity?

What are the similarities of artificial intelligence and electricity?

What comes to mind when you hear "artificial intelligence"? Educated several generations of Hollywood movies, we often think about "evil robots and intelligent computers that want to destroy humanity". But the AI already plays an ...

Can futurists predict the year of occurrence of the technological singularity?

Can futurists predict the year of occurrence of the technological singularity?

the end of the known world close. And this is good, as is believed by many futurists, predicting the inevitable onset of technological singularity. What is it? Technological singularity is the idea that technological progress, esp...

Old mice rejuvenated with a new drug

Old mice rejuvenated with a new drug

In the life of a cell is something horrible. Like the young heroes of the popular teen novels, the cells are born simply "companies" of the authority is to perform certain roles, predefined expression of their DNA. As the body in ...

Russian scientists were able to speed up and cheapen the process of obtaining gold

Russian scientists were able to speed up and cheapen the process of obtaining gold

Getting from the ore is time – consuming and lengthy process. Today one of the most popular ways of gold recovery is the direct cyanidation using the ability of gold to react with cyanide in the presence of oxygen. There is anothe...

The study of galactic wide web gives the first results

The study of galactic wide web gives the first results

the universe is not only vast expanses of darkness and trillions of galaxies, containing many billions of stars and billions of planets. In fact, it is much more difficult. Each individual galaxy as a single galactic cluster, are ...

#Video | SpaceX posted a panoramic video of the Hyperloop tube

#Video | SpaceX posted a panoramic video of the Hyperloop tube

Recently, the company SpaceX launched a competition in which 27 teams have presented their prototypes of capsules for the Hyperloop. The purpose of the event — to spur outside interest in the Hyperloop project, to stimulate ...