The rebirth of neural networks

After the hype of the 1990s, where companies like Intel or Philips built commercial hardware systems based on neural networks, the approach quickly lost ground for multiple reasons: hardware neural networks were no match for software neural networks run on rapidly progressing general-purpose processors, their application scope was considered too limited, and even progress in machine-learning theory overshadowed neural networks.

However, in the past few years, a remarkable convergence of trends and innovations is casting a new light on neural networks and could make them valuable components of future computing systems. Trends in technology call for architectures which can sustain a large number of defects, something neural networks are intrinsically capable of. Tends in applications, summarized in the recent RMS categorization, highlight a number of key algorithms which are eligible to neural networks implementations. At the same time, innovations in technology, such as the recent realization of a memristor, are creating the conditions for the efficient hardware implementation of neural networks. Innovations in machine learning, with the recent advent of Deep Networks, have revived interest in neural networks. Finally, recent findings in neurobiology carry even greater prospects, where detailed explanations of how complex functions, such as vision, can be implemented further open up the defect-tolerance and application potential of neural network architectures.