That's too naive. There's next to nothing known about how activity in individual neurons and their synapses relates to mental contents, in particular "higher level" concepts and thought patterns that you're concerned with in adult learning. In other words, the network dynamics are complex and unknown, and so the suppression / deactivation of certain synapses may just as well be a normal and necessary part of learning as it may be a part of forgetting, or neither. There is currently no contender for a "neuroscience standard model" that would bridge between this kind of neural dynamics and cognitive functions. I hope to live to see one.
Amen to that, we are starting to get some of the groundwork but are far far away from a “standard model”, neuroscientists,psychiatrists and AI developers oversell their understanding in order to keep their jobs and funding.
Regardless of one’s general level of intelligence, there is that age-old adage of a trade-off between book smarts and street smarts, or any other niche skill. Forgetting, and the resulting ignorance, is part of learning. Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies.
Still, any society has certain forms of learning with outsized benefits that make them worthwhile.
I remember reading a paper a little while ago that stated that experts are better at their craft because they're better at ignoring unimportant and irrelevant input.
Isn't that mostly a supervised learning technique? Maybe the larger question is whether learning in the brain is better described as unsupervised or supervised.
I don't know how babies learn to initially use words but after that entails 99% supervised learning through school IMO nowadays. I would even argue that most new economic growth is the also the result of general supervised learning (aka, learning from others because the wealth of human knowledge is way more than anyone can learn in a million lifetimes and so the economic growth comes from continuously combining new ideas that combine other new ideas).
But we aren't beating kids when they get facts wrong and we aren't giving them candy when they are right. So studying how the reward signal is computed should be interesting.
Beating kids and giving them candy is way on the extreme side of reinforcement. Having good grades be praised by teachers/parents and bad grades be associated with disappointment is already pretty good since humans are highly receptive to other people's emotions.
Really we are being too static in the thinking about it. Supervised learning can essentially be done by one brain region supervising another that is being trained. It doesn't have to be outside the skull.
My naive take on this is that it makes biologically sense to keep net expected electrical impulses approximately the same before and after strengthening. At the end of the day the brain has energy constraints.
This can-be/is done functionally in ANNs but achieves a different end (avoids over-fitting) but doesn't reducing energy(compute) expenditure in dense ANNs since activation and non-activation is computed in expectation and take the same number of cycles in dense networks.
I'd love to see more work on massive sparse networks, where you actually get compute efficiency if you can reduce number of activation without reducing hurting your optimization target.
This looks like a lateral inhibition effect running on biochemical means that are persistent compared to the transient electrochemical phenomena of classic neuronal lateral inhibition. The latter does lots of signal processing in visual and other systems. [1]
I would think any kind of training would have this effect. In mechanism—not purpose or ethics—brainwashing doesn’t strike me as all that different from ordinary practice.
You are right of course. But because a trivial message has many "layers" of "messages", according to semantic theory, it's often a single lie in a whole message packet.
45 comments
[ 3.1 ms ] story [ 80.6 ms ] threadStill, any society has certain forms of learning with outsized benefits that make them worthwhile.
I have heard this before and would really question if it is true.
Has anyone studied this?
This seems pretty normal to me
Citations here:
https://scholar.google.com/scholar?cites=6395717759743355511
ConvNets use LRN where most active neurons inhibit other neurons at the same location in neighboring feature maps. http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
This can-be/is done functionally in ANNs but achieves a different end (avoids over-fitting) but doesn't reducing energy(compute) expenditure in dense ANNs since activation and non-activation is computed in expectation and take the same number of cycles in dense networks.
I'd love to see more work on massive sparse networks, where you actually get compute efficiency if you can reduce number of activation without reducing hurting your optimization target.
This looks like a lateral inhibition effect running on biochemical means that are persistent compared to the transient electrochemical phenomena of classic neuronal lateral inhibition. The latter does lots of signal processing in visual and other systems. [1]
[1] https://en.wikipedia.org/wiki/Lateral_inhibition