No expectation. Just took pork as a breakfast for few days of hackathon but have not consumed any of that. But pork is meal N1 for me when I'm doing hard work like digging a ground or travelling on a bicycle.
0.2 watts is 172 (k)calories per hour, which isn't all that small (in addition extra load on the rest of the body, such as the heart, lungs, liver, etc. to support that metabolism)
Forgive me if I'm being dense, but that number seems really implausible to me. 172 kCal an hour is (assuming just waking hours consume calories) about 2700 kCal just from the cortex over the course of a day, which is more than the average Total Daily Energy Expenditure.
0.2 watt is 172 small calories. A large "food Calorie" is a kilocalorie, the energy required to heat one kilogram of water by 1 degree C. The small gram calorie is the energy required to heat one gram (or one mililitre) by one degree C. These may be distinguished by a large or small 'c', where "Calorie" -> kilogram calorie and "calorie" -> gram calorie. I prefer spelling out "kilocalorie" where there's any possibility for confusion.
0.2 watt is 0.172 kilocalories per hour, or about 4 kcal/day.
I find it super weird that my food intake nor my sleep duration is correlated with my brain use.
There are days where I will hardcore hyperfocus learning technical scientific papers all day and also deeply think/debate about a complex topics for hours and other days where my brain will be complely un-stimulated and yet as I said, no variation. It's absurd.
A tangeant would be: do highly intellectual people have shorter lifespan because their body would trade-off energy for their brain thus limiting the supply on other organs and hence creating deficit in e.g. mitochondria bioenergetics?
This is the same base paper/research as in https://news.ycombinator.com/item?id=31386091, which has a lively discussion... (and, at least as I write this, is also on the first page) :)
I find it very relevant to this Brain FLOPs estimation as it reduce the upper limit by a factor of ten
https://news.ycombinator.com/item?id=31385913
It means the brain has at max 8.1 million firing events per seconds so only 10% of neurons (or axons?) would be active in a second timeframe?
BTW if so this has a huge implication on consciousness.
premise 1) we know that reduced consciousness is a continuu, indicating a percentage of inhibited (deactivated) neurons, which can be gradually increased via e.g. benzodiazepines up to a coma or cerbral death. Many depressants reports as a symptom the intermediate feeling of reduced consciousness.
Now if we assume only ~ten percent of neurons are active at a given time, and as shown in premise 1) consciousness is a result of neuron activation and is physically located as the acitvated neurons. Then time passe and for some reason/stimulus, the newly activated neurons do not overlap with the previous 10%. In this experience of thought, since there is no overlap, the previous consciousness location has totally shut down like a brain death, and the new consciousness reside in newly activated neurons. Are you the same person?
Since the physical location is disjoint, if you suppose you are, you have zero criterion for discriminating you from newly made isomorphic clones which is a paradox assuming there can't be multiple you.
I think 0.2 watt is sufficently humiliating to prove that deep learning research inefficiency of large models is indicative of missing a fundamental parcimonious paradigm shift. What is it that the brain do that transcend our algorithms so much? Well the C.elegans research underfunding will not help us answer that question. But hey deepming & ci have their priorities and only the incoming transformers winter will force them to evolve.
I’m confident that everyone in the deep learning industry knows our algorithms are horribly inefficient. The canonical example is training inefficiency. They are constantly noting that humans can do one shot learning and our algorithms generally can’t.
EDIT: That said there is valid criticism of their focus on very large models at the expense of more efficient algorithm research, so maybe that’s what you were getting at.
that is fallacious considering e.g. non-coding DNA
see e.g. https://www.quora.com/How-many-bytes-memory-size-is-a-humans...
the estimate is more like 50MB and the difference between humans of the dozen of MBs.
Considering that the brain is just one component of our very complex bodies, that's just a couple of MBs at max allocated to Brain encoding, including data that is needed for constructing the brain structurally, and not encoding a memory or running the runtime.
The brain runtime probably has a ~1MB max DNA budget, which is pathetically small.
Alhough that is an underesearched topic (there is not even a dedicated wikipedia page), genetic memory is a thing (how do spiders learn to draw the web, some animals know how to walk at birth, to breath, birds innately know how to nest (and some maybe how to fly). Genetic memory is a nice topic and I wonder if we have findings how it manifest (via IRMs, etc) but it's unclear how/where it is encoded given that the DNA budget is very very tight.
Human brains have hundreds of millions of years of evolution baked in. And each one still takes about 20 years of training and development to get to the point where it can apply logic rationally.
None of us can. At most we can do some very virtualized logic interpretation, but we are all faulty, emotional-driven “machines”, and even behind the most logical people’s actions the reason is simply they felt that way.
trained people that have no involvement/conflict of interest in something can perfectly analyse it in a cold and objective manner, with the given provided information.
People's actions are necessarilly irational, that's not the same thing as an external analysis/decision making output.
For one, unlike DL models, we approximate things, and I do not mean that in the universal approximator sense. What I mean is that most of the things we do are essentially pulling from memory similar situations and stitching everything together.
Unless you are John Von Neumann, you are either approximately correct or very slow in solving novel tasks. On the other hand, we have the capacity to generalize much easier than our DL models.
I think that that relates to our memory. That is, we compress information in a way that allows us to search for the structure, map to the target domain, and then stitch it all together.
Hofstadter argues about it a lot in his books, that is, that we use analogies to learn stuff. In the same vein and in a very handwave-y manner, we can think of analogies as functors across categories. That is, as mappings between one domain to another where the relationship (morphism) between the objects is preserved.
For whatever reason we are capable of identifying those mappings quickly and applying them everywhere. This relates to symmetries where we have little to no trouble identifying objects that underwent certain transformations but our DL have trouble with.
A user in HN suggested that with respect to the visual transformations, we can do that because our optic nerve travels a long distance, from our eyes to our visual cortex, and that allows our brains to perform all kinds of data cleaning and signal processing.
The ideas about compression that I mentioned here is essentially what we saw in deepmind's GATO. More precisely, creating multimodal embeddings in that manner seems to enable good multi-task performance.
Forgive my lack of knowledge of brain physiology, but is “cortex” a very small fraction of the total brain tissue?
The common story is that our brains consume disproportionate amounts of energy compared to the rest of our body. So if both of these stories are true, it means the cortex for whatever reason is a very tiny portion of brain energy use, and the rest of the brain uses a lot more?
* edit: Ah, I see the other story on the front page about brain comms using 35x more energy. So maybe the story with brains is the same as the story with computers - memory and transfers are the expensive part, and compute is surprisingly cheap.
The common story is that the brain consume 20% of the body energy. This study (if correct) refine the interpretation that the brain consume 20 watt by partitioning between computation and communication cost. Here there is no disproportionate amount of energy, it is just that the remainder is spent is mere communication.
Also yes they speak about the cortex, it is true that the majority (~70%) of the neurons are not in the cortex but in the forgotten cerebellum, however while mysterious, the cerebellum is not necessary for language processing, there are some humans on earth that don't have a cerebellum and while less intelligent, rumors are that they are still human. Although this should be double checked.
also while I think about it, there's probably a huge chunk (how much) of energy spent in building/catabolizing new axons vs executing the runtime.
There are cases of extreme hydrocephaly where 95% of brain matter was absent but the patient led a fairly normal life. I can't find the MRI image now but just imagine a thin, bright layer of brain tissue right against the skull, and almost total darkness (water) inside.
Fair warning, you will see some pretty disturbing image results if you look it up.
you can find many here
https://www.google.com/search?q=hydrocephalus+irm&sxsrf=ALiC...
Look I have a hard time believing it.
I mean yes those cases truly happened and the CBF fluid truly filled those zones.
However my question is, can an IRM see beyong the a CBF fluid overload? Maybe there was just a shallow CBF overload and deep inside the neurons/synapses where still working.
Or maybe not and those people really have lost 90% of their neurons, it's unfortunate no one care enough to fact check given this is essential towards understanding what is necessary in the brain.
34 comments
[ 0.24 ms ] story [ 82.1 ms ] threadHow much carbohydrates did you expect you should consume?
0.2W x 3600s -> 720J -> 172 cal
0.2 watt is 172 small calories. A large "food Calorie" is a kilocalorie, the energy required to heat one kilogram of water by 1 degree C. The small gram calorie is the energy required to heat one gram (or one mililitre) by one degree C. These may be distinguished by a large or small 'c', where "Calorie" -> kilogram calorie and "calorie" -> gram calorie. I prefer spelling out "kilocalorie" where there's any possibility for confusion.
0.2 watt is 0.172 kilocalories per hour, or about 4 kcal/day.
"The average human, at rest, produces around 100 watts of power"
http://large.stanford.edu/courses/2014/ph240/labonta1/
A tangeant would be: do highly intellectual people have shorter lifespan because their body would trade-off energy for their brain thus limiting the supply on other organs and hence creating deficit in e.g. mitochondria bioenergetics?
Poor availability of healthy + low calories + palatable + inexpensive food in developed countries.
The obesity crisis comes from that.
BTW if so this has a huge implication on consciousness.
premise 1) we know that reduced consciousness is a continuu, indicating a percentage of inhibited (deactivated) neurons, which can be gradually increased via e.g. benzodiazepines up to a coma or cerbral death. Many depressants reports as a symptom the intermediate feeling of reduced consciousness.
Now if we assume only ~ten percent of neurons are active at a given time, and as shown in premise 1) consciousness is a result of neuron activation and is physically located as the acitvated neurons. Then time passe and for some reason/stimulus, the newly activated neurons do not overlap with the previous 10%. In this experience of thought, since there is no overlap, the previous consciousness location has totally shut down like a brain death, and the new consciousness reside in newly activated neurons. Are you the same person? Since the physical location is disjoint, if you suppose you are, you have zero criterion for discriminating you from newly made isomorphic clones which is a paradox assuming there can't be multiple you.
EDIT: That said there is valid criticism of their focus on very large models at the expense of more efficient algorithm research, so maybe that’s what you were getting at.
Long answer: https://www.quora.com/How-many-bytes-memory-size-is-a-humans...
Unless you are John Von Neumann, you are either approximately correct or very slow in solving novel tasks. On the other hand, we have the capacity to generalize much easier than our DL models.
I think that that relates to our memory. That is, we compress information in a way that allows us to search for the structure, map to the target domain, and then stitch it all together.
Hofstadter argues about it a lot in his books, that is, that we use analogies to learn stuff. In the same vein and in a very handwave-y manner, we can think of analogies as functors across categories. That is, as mappings between one domain to another where the relationship (morphism) between the objects is preserved.
For whatever reason we are capable of identifying those mappings quickly and applying them everywhere. This relates to symmetries where we have little to no trouble identifying objects that underwent certain transformations but our DL have trouble with.
A user in HN suggested that with respect to the visual transformations, we can do that because our optic nerve travels a long distance, from our eyes to our visual cortex, and that allows our brains to perform all kinds of data cleaning and signal processing.
The ideas about compression that I mentioned here is essentially what we saw in deepmind's GATO. More precisely, creating multimodal embeddings in that manner seems to enable good multi-task performance.
Sorry if this comes across as rambling.
The common story is that our brains consume disproportionate amounts of energy compared to the rest of our body. So if both of these stories are true, it means the cortex for whatever reason is a very tiny portion of brain energy use, and the rest of the brain uses a lot more?
* edit: Ah, I see the other story on the front page about brain comms using 35x more energy. So maybe the story with brains is the same as the story with computers - memory and transfers are the expensive part, and compute is surprisingly cheap.
Also yes they speak about the cortex, it is true that the majority (~70%) of the neurons are not in the cortex but in the forgotten cerebellum, however while mysterious, the cerebellum is not necessary for language processing, there are some humans on earth that don't have a cerebellum and while less intelligent, rumors are that they are still human. Although this should be double checked.
also while I think about it, there's probably a huge chunk (how much) of energy spent in building/catabolizing new axons vs executing the runtime.
Fair warning, you will see some pretty disturbing image results if you look it up.
Are you sure 95% is by weight, not by volume?