Enlarge (credit: Mooganic)
Neural networks were all the rage for a while, but progress eventually slowed and interest cooled.

Then, as computing power increased, the field experienced a renaissance, and deep learning was the new big thing.
Throughout this ebb and flow of interest, there has been an underlying, annoying fact: neural networks as currently implemented are not that great.

Especially when you compare them with the brain of… well, pretty much any creature. Researchers have been trying to make neural networks that have all the advantages of the brain (and none of the disadvantages) for as long as the field has existed.

And it may be that they’ve gone about it wrong. Now, some new work is suggesting that the only way to get the advantages of the brain is to accept the disadvantages as well.
Brains vs. memristors
The brain has two features that no inorganic computer has. One is that it is highly interconnected.

Each neuron may be connected to a vast number of other neurons—not just neighbors, but also neurons that are well separated spatially.

This natural interconnectedness is what makes the brain such a powerful computational tool.

The brain is also highly efficient.

A synapse—the connection between two neurons—consumes at most 100 femtoJoules per event. Once you realize that the entire human body is about the equivalent of an 120W light bulb, you can see that the efficiency of the brain is just astounding.
Read 17 remaining paragraphs

Leave a Reply