Neurons

How do brains work?

I mean, how do brains work at a level slightly higher than a synapse, but much lower than, e.g. psychology. “How is thought done?” etc.

Notes pertaining to large, artificial networks are filed under artificial neural networks. The messy, biological end of the stick is here. Since brains seem to be the seat of the most flashy and important bit of the computing taking place in our bodies, we understandably want to know how they work, in order to

• fix Alzheimer’s disease
• steal cool learning tricks
• endow the children of elites with superhuman mental prowess to cement their places as Übermenschen fit to rule the thousand-year Reich
• …or whatever.

Real brains are different from the “neuron-inspired” computation of the simulacrum in many ways, not just the usual difference between model and reality. The similitude between “neural networks” and neurons is intentionally weak for reasons of convenience.

For one example, most simulated neural networks are based on a continuous activation potential and discrete time, unlike spiking biological ones which are driven by discrete events in continuous time.

Also, real brains support heterogeneous types of neurons, have messier layer organisation, use less power, don’t have well-defined backpropagation (or not in the same way), and many other things that I as a non-specialist do not know.

To learn more about:

• Just saw a talk by Dan Cireșan in which he mentioned the importance of “Foveation” — blurring the edge of an image when training classifiers on it, thus encouraging us to ignore stuff in order to learn better. What is, rigorously speaking, happening there? Nice actual crossover between biological neural nets and fake ones.
• Algorithmic statistics of neurons sounds interesting.
• Modelling mind as machine learning.