That didn't work for me. The top result is still the WSJ URL; however, a few links lower, we can find another URL which does show the same full article:
Not necessarily, I got the link from Google and still saw the paywall when going through regular Chrome mode. However, once I opened the same referrer link in incognito mode, I was able to see the article.
Looks like WSJ is now using cookies to decide whether to show paywall or not.
Definitely cookies. If you click the link before doing the web search it will remember. I had to open an incognito tab and then search for the title. Definitely a pain in the ass.
>We need to update the New Deal for the 21st century and establish a trainee program for the new jobs artificial intelligence will create.
Maybe I'm being a bit pessimistic, but can AI really create jobs just by taking over old ones? Sure we could train some new "data analysts [and] trip optimizers", but in the end can we really mass replace low skilled blue collar jobs with higher skilled ones with the wave of a wand? In the period between when robots can automate the majority of low skill jobs and virtually all jobs, there is very likely to be some sort of significant turmoil as our economy undergoes a paradigm shift of sorts, and I don't think "just retrain workers" is that viable of a solution.
When I hear "retraining workers," I usually think of Charlie (of "And the Chocolate Factory" fame)'s dad losing his job screwing in toothpaste caps and coming back in the end as a technician for the machines that took his job.
That is the wrong way to approach this new shift.
There will simply be too many menial jobs made obsolete for blue collar workers to step up into supervision/technician roles.
Instead, they will have to find new kinds of jobs. The kinds of jobs that cannot be replaced by robots without getting stuck in the Uncanny Valley. Jobs like personal trainers, yoga instructors, physical/massage therapists, tattoo artists, hairdressers, and so on. Unless a robot can 100% mimic human form and action, these jobs aren't going anywhere - certainly not overseas.
I'm calling it here and now: as jobs are lost too quickly for people to retrain (or even want to adopt an entirely new skill set) we will see a new push for the legalization of prostitution. If nothing else, you can always sell your body by the half-hour, and no robot could ever truly replicate the closest of human touch.
There will simply be too many menial jobs made obsolete for blue collar workers to step up into supervision/technician roles
I think you're thinking about this backwards. AI will claim professional jobs like accountants and lawyers LONG before it affects hairdressers and plumbers.
This is partly because robotics and computation are orthogonal. We have fast progress in computation, but robotic progress is much slower at the scale of human hands.
Agreed. We used to wish for the day automation would take the burden of work from our shoulders. Now that's its upon us, all I hear is "how will we put everybody back to work?!" How about: we don't have to work like that any more. Ever again.
>Deep learning, modeled on the human brain, is infinitely more complex [than machine learning]. Unlike machine learning, deep learning can teach machines to ignore all but the important characteristics of a sound or image-a hierarchical view of the world that accounts for infinite variety. -Yann LeCun
I strongly disagree with a lot about this quote, even though it comes from a brilliant man I highly respect (my thesis research was inspired from some of his older work, and in my job we work with techniques he developed around deep learning). What I dislike is it utilizes hype to make deep learning seem mystical; in actuality it's a natural extension of old techniques that clearly fall under "machine learning".
Deep learning is a neural network with 3 or more layers instead of the 2 layer networks that were developed in the 1980s. People tried 3 layers back then, and they didn't work well. Yann LeCunn and other researchers found cool ways to get 3+ layers to work in the 90s and again in the mid 2000s. More recently, researchers have just thrown a ton of data and computational power at them to get them to work. But fundamentally this was an extension of established techniques. This article that recently hit the front page actually breaks it down really well: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-ar...
I think my main point here is that deep learning is quite accessible to people who are learning machine learning. It's great at solving some really complex problems (that can certainly resemble true intelligence), but is not the right tool for other problems.
I remember reading that the fastest human reflexes and computational outputs seem to happen around 100ms, and it takes roughly 10ms to transmit a signal from neuron to neuron, so that would imply 10 layers deep is enough for, for example, human level visual recognition. Of course, the human brain is highly recurrent, so it gets much more complicated for any higher level thinking. And it's also much, much wider than our neural networks, even if not spectacularly deep.
The human brain is what they call a recurrent neural network which means that it contains directed cycles. Classic neural networks are strictly hierarchical and one-way.
If you take into account that signals can keep resonating in the brain (firing in circles) then the amount of layers is infinite.
Recurrent neural networks are incredibly hard to train with the exception of liquid state machines (also known as echo state networks or reservoir computing).
Brain doesn't work like NNs in software. It's different beast you can't really compare layers in one and in second.
There are several reason why this model just does not apply to the brain[1]:
1. The brain is not organized in layers this way. Instead there are interconnected circuits, feedback, brain regions, bidirectional connections, and such.
2.The brain's connections are feedforward only without "error feedback".
3. The brain doesn't learn from training data with correct answers (or a "ground truth").
4.The neurons in the brain spike and use different learning algorithms.
Just wanted to add that there _are_ certain portions of the brain which do seem to be layered, namely fundamental image-recognizers in the visual cortex. These were a lot of the inspiration for layered neural nets, and may explain some of their success on similar tasks.
As I point out in another post (currently above) the layers of Neural nets have nothing in common with those in the brain. Only confusion will be met in heading down that path.
What is had in common, and what was inspirational were (harsh) experiments on kittens, done in the 60s which provided hints as to what concepts were important in the primary visual cortex. These experiments lead to ideas of pooling and averaging as ways to calculate invariance, which made their way into the neurocognitron and then the convolutional net. The concepts of thresholding units and summation of inputs also played a part in the designs of Artificial Nets more generally. Most recently, new artificial NN designs have been loosely inspired by the importance of attention in biological networks.
Another similarity, but this is a general computational similarity, is the importance of hierarchical organization in easing learning difficulty. But unlike in the mostly feedforward networks we're used to in ANNs, brain's networks are also feedback, or with backward projections, across areas and up and down hiearchies and so on.
But there is more to vision than V1 and V4. The visual cortex has more arbitrary connections across layers and is not restricted to the bipartite structure common in MLPs and is more fine grained than the hidden layer summation across timesteps of RNNs.
Other things the visual cortex computes but not well captured in models: speed and directional selectivity, nontrivial color transformations, temporal frequency, global v local + foreground v background, optical flow, motion tracking, inhibitory neurons, higher level scene understanding, feedback connections, perceptual color constancy and contributions to memory formation.
Yes, I was specifically referring to the findings from those kitten experiments.
Of course there are plenty of other things that the visual cortex is involved in, as far as we can understand it at least. I'm talking purely about the macrostructures that are involved in elementary vision functions like orientation.
I'm in the camp as well that conflating brains and ANNs is a dangerous and hype-prone line to walk, but I'm not sure you can categorically say that the layers in V1 and the layers in a CNN have nothing in common, with respect to the computational capabilities they model. Likewise with convolutions and columns.
I say that the computations in V1 and the computations in the CNN share a fair bit in common.
However, I doubt that the layers in V1 and CNN have much in common for a couple reasons. First is because the layers in V1 will differ depending on who you're reading from and when they wrote it. The layer divisions are more structural than functional, so far as we know. You can't trace ever more abstract features as you move across V1 layers, not least because V2-V4 are also involved in this.
Next is that V1 sends projections to "higher" and also "lower" areas while connecting to other areas than just the visual. Within V1, connectivity from lower to higher layers are not obvious. Some inputs from the LGN might feed into layer 4 and others to differing layers depending on whether the computation is motion or color etc specific. Layer 5 might also get connections from more distant areas. This level of interconnectivity suggests that very different computations are going on even at the level of just V1. The connections are finer and more intricate than simple bipartite divisions. Basically, they're working out more than just features that are robust to rotations and translations (e.g. speed, direction, dual stream signal integration, oscillations...).
The prevalence and importance of feedback and recurrent connections is not yet well understood but are believed to be important in the sort of high level scene understanding that are still open problems.
And related to that, the notion of inhibition is turning out to be more and more important in biological networks though they aren't really key to any neural nets. Some of their repeating patterns or motifs, have a place in directional and size selectivity but more intriguingly, are figured to be key in the top down flow of information that allows for our (mammalian) complex scene understanding.
The importance and whether there is any function of the column is not without controversy, it may well be an artifact of what tools were used in the early days. The brain is very flexible in its organization and attempts at organizational structures such as columns aren't very consistent even across individuals of the same species: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1569491/
That is a good explanation. I'll point out for anyone that clicks through, that unlike what he says, depth in ANNs do serve a purpose: to make learning more tractable by building representational abstractions.
There are feedback and recurrent connections in the brain but what (2) means to say that the specific mechanism is not biologically plausible. I know there are prominent Deep learning researchers who are trying to work around this but those explanations have never struck me as particularly parsimonious. It's better to start an explanation from experimental data on brains than from them most important aspect of neural nets.
Neural networks are based on a very, very outdated (and incorrect) model of how a neuron works, so your question is roughly akin to asking "how many vacuum tubes are in my GPU?"
Mammalian brains are so densely packed that we don't really know what their internal architecture actually looks like--that's why estimates of the number of neurons keeps increasing over time [1]. Most of our models are based on some autopsies, and rely on things like fMRI or electroencephalography (not often you get to spell that word!) to work out which portions of the brain are involved in certain tasks. We have no hope of being able to build a connectivity graph between neurons at our current state of technology.
The short answer is that the cerebral cortex is usually identified as having 6 layers. These can't be correlated with layers in a neural network, however. I do seem to recall a spitball number of 30-ish in relation to the visual cortex, but I've been unable to find any references as to what that number would actually represent, let alone if it's been superseded.
[1] The current estimate is around 100 billion. If you want to know the number involved in, say, the visual cortex, that appears to be on the order of a few billion.
> The short answer is that the cerebral cortex is usually identified as having 6 layers. These can't be correlated with layers in a neural network, however. I do seem to recall a spitball number of 30-ish in relation to the visual cortex, but I've been unable to find any references as to what that number would actually represent, let alone if it's been superseded.
That short answer is very misleading. Those 6 layers are histological layers of density and structure in the cerebral cortex (eg how the cross sections look a little different). You might have many (30+) layers of cells with high interconnectivity in each of those 6 layers. Current estimate is 100 billion for the number of neurons, but the estimate of inter-connectivity is on the order of 100 trillion synapses. The human brain complexity many orders of magnitude higher than anything we have ever "simulated" in silicon.
It's not misleading because they were careful to point out that these 6 'layers' have nothing in common with how layers are defined in Neural networks. In Neural nets, layers are effectively subgraphs that are bipartite. They can also be computational stages in some networks like convolutional.
The brain's layers are both much more loose in their allowed interconnections but also more modular in their functionality. There are different visual areas (usually 5) at different locations of the neocortex that connect to each other (feedforward and back) in complex ways. As residents of the neocortex, each visual area will thus have 6 layers which are in turn dividable into sublayers (often 1 - 4). These layers are structural but are also partially organizational tools for the scientists.
In terms of number of neurons, human brain is around 10^11. Current image classification deep networks are around 10^7. However, its not just the number of neurons, the connections per neurons also matter. The connections per neuron in a human are in thousands while in modern networks they are in hundreds.
The human brain is also very efficient so it also makes a very good use of those neurons, and modern networks aren't. However, a human brain trains over a very very long time and also prunes itself constantly, while such efficiency techniques at scale for very large networks are being explored in deep learning only recently.
If you mean the lack of theoretical guarantees due to a highly non-convex optimization, you could argue the same thing about 30 year old 2-layer networks. And we actually do have a decent idea why deep learning works well:
Btw, "why" questions are always a bit ill-defined, it's like asking for some kind of narrative to satisfy the questioner. "well in the beginning there was the big-bang..." So anyway I'm quite impressed at these papers.
I think the way forward for AI will be in simulating human conciousness processes, including whatever limitations computationally that come with that. In this sense, I feel that the kind of AI that will break barriers first is going to be in a game, and not on a factory floor.
Can you elaborate a bit more on this? I'm fascinated by consciousness, and I've often wondered about how it arises in biological systems and how it might do so in artificial ones.
My basic intuition is that consciousness arises as a byproduct of any sufficiently interconnected (i.e. Complex) network which processes input from the outside world. The problem with AI right now is bottlenecking of both kinds — we don't have computers powerful enough to support the level of complexity of the human brain (100bn neurons with an average of 7k synapses each), and we don't have good enough sensors for capturing the real world (hint: touch is a BIG one).
What sort of game do you have in mind? I definitely think a game in which humans interact with an AI in an artificially-supported information-rich environment (say, the VR of 5-10 years from now) could help provide a suitable environment for "growing" such an AI. Consider the insane amount of
Stimulation and calibration human babies require for development! It takes years for the visual system to stablilize, for example.
> My basic intuition is that consciousness arises as a byproduct of any sufficiently interconnected (i.e. Complex) network which processes input from the outside world.
I think you are underestimating the importance of the motor/action system for the development of consciousness. Many neuroscientists and robotic researchers think that the sensory-motor-environment-sensory loop plays an important role for consciousness. This loop allows you to unsupervised learn all sorts of laws of physics and allows you to think that your body actually is you and not just an extension of your mind and therefore also allows the development of the concept of self awareness. Some researchers might call this sensory-motor-contingencies or embodiment. I think when talking about consciousness there it is very important to take into account this loop and not only the sensory perception and building a neural representation of the world...
The only proof of consciousness case we have (i.e., consciousness arising from Big Bang gases and getting to this conversation) incorporated sense-motor-environment-sense loops. Don't know if it can happen without such interactions, but the one route we know worked, used them.
I definitely agree with you — I think interacting with the environment is an important part of "processing" it. The motor/action feedback loop (cf. artificial motor control in monkeys) is exactly what I had in mind when I said that "touch" is a big one.
I definitely should have clarified that point, though, as it's a huge one, I agree!
Everyone has piled on the learning bandwagon as the path to "intelligence" but honestly creativity is just as important, and it is hardly even being addressed. Even worse, while function approximation (i.e. learning) is a fairly well defined problem, creativity is nebulous and ill-defined.
That's not true, actually. There's a ton of work being done in computational creativity. AI is actually pretty good at sensing similarities between instances of data, and recombining various elements to create something new. There's been a ton of recent work using deep learning, including DeepDream.
New jobs created by AI: Oxford faculty member pontificating about values for AIs. Who says there isn't job creation.
My worry: AIs held only to the moral standards now expected of corporations. Optimize for shareholder value. We're close to this now with machine-learning assisted hedge funds.
Ah the upcoming "automation utopia" will "free man from the chains of labour" to pursue their passions -- at least that's what they will tell us while SF firms endlessly drive towards automating more people's jobs away while they rake in the cash. My worry is the lengthy, inevitable in-between time of human suffering and corporate greed until something bursts and we actually pass 21st century laws like Basic income, vote in technology-competent politicians, etc.
The problem with news articles like this is that they attempt to appeal to a general readership through grand promises and fear-mongering. The reporter's basic problem is: How can I make my audience care?
So you call up Nick Bostrom and he very reliably gives you a quote about the existential threat of a superintelligence, even though no one in the industry thinks we're anywhere close to that. (What's next for AI is not superintelligence...) And you force a great researcher like Andrew Ng to talk about job loss among truck drivers, because that's what will make it relevant to people outside AI.
We should be thinking about job loss and how the job market will change, but this type of article never gets past the "Oh No AI Will Destroy US" stage of thinking. But a lot of the questions raised by AI are actually relevant now, in an economy where AI hasn't even made a big impact. That is, they're not AI issues, but we're treating them as though they are. How should our societies treat and support the humans who have become unnecessary? They obviously will not all become data analysts and robot caretakers.
What's next for AI is better natural-language processing. Right now, chatbots are pretty dumb, but in the next few years, they'll get much better, and in more languages.
What's next for AI is the wider deployment of mature technology. Many problems such as image recognition have been solved, but developers and companies have not figured out how to deploy the solutions yet. We still have chokepoints in the number of data scientists who can tune and train models, and the number of engineers who can plug them into existing stacks. So AI will be felt directly, rather than just talked about.[0]
What's next for AI is an arms race. The major powers will be escalating the how they deploy AIs against AIs, embedded in drones or through the creation of adversarial data to slip through filters. Commercially, many smaller arms races will occur in different industries, as AI drives down the costs of interpreting data, and allows rival organizations to compete on price.
What's next for AI is the combination of the flavor of the month, deep learning, with other extremely powerful algorithms like reinforcement learning and Monte Carlo Tree Search to create goal-oriented strategic decision-making agents.
I don't have anything valid to back up my claims here, so take what i have with a grain of salt unless you find external validation for it. That being said i will venture a guess:
Yes and no. Neural networks have a lot of nice properties like being easier to grasp, easier to parrallelize(?) and have better tools which i guess is because neural networks are applicable to more problems than SVM's because they are extensible.( see https://en.wikipedia.org/wiki/Recurrent_neural_network for example).
Either way SVM's still do what they do very nicely and in the field of "Human Activity Recognition" where i did my thesis neural networks are practically never used but SVM's pop up from time to time.
Right, scalability is a good point to consider. Along the same vein, my last ml job we ended up using gradient boosting which worked really well, but definitely does not scale to big data (afik.) Robustness is another thing to consider, both svm's and neural networks need a reasonable amount of data-massaging before they behave themselves. Hence the success in image processing where every pixel can be treated equally.
There's smarter, but there's also faster. Historically, robots have been rather slow and clunky. That's over.
Industrial robots have become much faster. Here's Bosch's packaging robot from 2014, looking at small objects and putting them in order for packaging.[1] Then Fanuc built a faster one.[2] Faster CPUs allow using modern control theory that considers dynamics, and fast machine vision. The resulting machines are much faster than humans.
Those are production machines, able to run for long periods. Research robots are even faster.[3] That's from 2012. Progress continues.
Everyone seems to believe that achieving artificial general intelligence is inevitable. I'd argue that it's only inevitable if humanity survives long enough to make it happen. I'm not a pessimist, but the next 300-400 years will be the most difficult humanity has ever faced. In addition to climate change, population expansion, nuclear weapon proliferation, and naturally increasing inequality, humanity will face many more threats that haven't yet been perceived.
I believe building strong intelligence will optimistically take 3-4 centuries. To complete a "system that could successfully perform any intellectual task that a human being can," it is first necessary to understand what defines the human intellect and conscience. Although there is much ethical debate on what defines a conscience, scientifically it is a product of experience, and can be emulated if three requirements are met:
- The system can accept every input the human body can.
- It can process every combination of stimuli in the same way a human would.
- It can to the stimuli in every way a human would.
For these requirements to be met by software, we must either:
- Aquire a nearly full knowledge of the human brain (and probably body)'s information-processing mechanisms, and figure out how to implement this in software.
- Build enough processing power to completely simulate the human brain/body, atom by atom.
Both will not be feasible for an extraordinary amount of time, and it's probably better to spend our time worrying about the current existential threats to humanity.
A much more relevant ethical problem for humanity is that of eugenics. Unlike AI, recent advances like CRISPR/CAS-9 make it viable now, through modification of sperm-generating stem cells (not embryos), and like AI, it offers world-changing benefits (the eradication of diseases, lengthened life spans, increased knowledge and strength, etc), while also providing the keys for modern humanity's destruction (designer babies, lack of diversity, separation of humans into casts, etc).
Perhaps increases in intelligence driven by eugenics will even cause computer systems and AI to become obsolete.
The fact is, we don't know how deep Neural Networks work - we simply know how they form. They are adaptive systems that learn to perform complex tasks.
This is scary because it means we don't "design" an A.I., we design an adaptive system and allow it to emerge. Nothing could be more dangerous in the long term.
64 comments
[ 4.1 ms ] story [ 128 ms ] threadhttp://newspot.me/n/03LT3uuT
Looks like WSJ is now using cookies to decide whether to show paywall or not.
Maybe I'm being a bit pessimistic, but can AI really create jobs just by taking over old ones? Sure we could train some new "data analysts [and] trip optimizers", but in the end can we really mass replace low skilled blue collar jobs with higher skilled ones with the wave of a wand? In the period between when robots can automate the majority of low skill jobs and virtually all jobs, there is very likely to be some sort of significant turmoil as our economy undergoes a paradigm shift of sorts, and I don't think "just retrain workers" is that viable of a solution.
That is the wrong way to approach this new shift.
There will simply be too many menial jobs made obsolete for blue collar workers to step up into supervision/technician roles.
Instead, they will have to find new kinds of jobs. The kinds of jobs that cannot be replaced by robots without getting stuck in the Uncanny Valley. Jobs like personal trainers, yoga instructors, physical/massage therapists, tattoo artists, hairdressers, and so on. Unless a robot can 100% mimic human form and action, these jobs aren't going anywhere - certainly not overseas.
I'm calling it here and now: as jobs are lost too quickly for people to retrain (or even want to adopt an entirely new skill set) we will see a new push for the legalization of prostitution. If nothing else, you can always sell your body by the half-hour, and no robot could ever truly replicate the closest of human touch.
Joke aside, it's funny all the Uber drivers are collecting data for a company that is going to replace them with AI cars..
I think you're thinking about this backwards. AI will claim professional jobs like accountants and lawyers LONG before it affects hairdressers and plumbers.
I strongly disagree with a lot about this quote, even though it comes from a brilliant man I highly respect (my thesis research was inspired from some of his older work, and in my job we work with techniques he developed around deep learning). What I dislike is it utilizes hype to make deep learning seem mystical; in actuality it's a natural extension of old techniques that clearly fall under "machine learning".
Deep learning is a neural network with 3 or more layers instead of the 2 layer networks that were developed in the 1980s. People tried 3 layers back then, and they didn't work well. Yann LeCunn and other researchers found cool ways to get 3+ layers to work in the 90s and again in the mid 2000s. More recently, researchers have just thrown a ton of data and computational power at them to get them to work. But fundamentally this was an extension of established techniques. This article that recently hit the front page actually breaks it down really well: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-ar...
I think my main point here is that deep learning is quite accessible to people who are learning machine learning. It's great at solving some really complex problems (that can certainly resemble true intelligence), but is not the right tool for other problems.
https://en.wikipedia.org/wiki/Neocortex#Anatomy
Recurrent neural networks are incredibly hard to train with the exception of liquid state machines (also known as echo state networks or reservoir computing).
There are several reason why this model just does not apply to the brain[1]:
1. The brain is not organized in layers this way. Instead there are interconnected circuits, feedback, brain regions, bidirectional connections, and such.
2.The brain's connections are feedforward only without "error feedback".
3. The brain doesn't learn from training data with correct answers (or a "ground truth").
4.The neurons in the brain spike and use different learning algorithms.
[1]https://www.quora.com/How-many-hidden-layers-are-in-the-neur...
For example, we get some translation-invariance for free with the ability to move our eyes.
[1] http://ilp.mit.edu/event.jsp?id=6094 (I had the video URL bookmarked, but it's broken now)
What is had in common, and what was inspirational were (harsh) experiments on kittens, done in the 60s which provided hints as to what concepts were important in the primary visual cortex. These experiments lead to ideas of pooling and averaging as ways to calculate invariance, which made their way into the neurocognitron and then the convolutional net. The concepts of thresholding units and summation of inputs also played a part in the designs of Artificial Nets more generally. Most recently, new artificial NN designs have been loosely inspired by the importance of attention in biological networks.
Another similarity, but this is a general computational similarity, is the importance of hierarchical organization in easing learning difficulty. But unlike in the mostly feedforward networks we're used to in ANNs, brain's networks are also feedback, or with backward projections, across areas and up and down hiearchies and so on.
But there is more to vision than V1 and V4. The visual cortex has more arbitrary connections across layers and is not restricted to the bipartite structure common in MLPs and is more fine grained than the hidden layer summation across timesteps of RNNs.
Other things the visual cortex computes but not well captured in models: speed and directional selectivity, nontrivial color transformations, temporal frequency, global v local + foreground v background, optical flow, motion tracking, inhibitory neurons, higher level scene understanding, feedback connections, perceptual color constancy and contributions to memory formation.
Of course there are plenty of other things that the visual cortex is involved in, as far as we can understand it at least. I'm talking purely about the macrostructures that are involved in elementary vision functions like orientation.
I'm in the camp as well that conflating brains and ANNs is a dangerous and hype-prone line to walk, but I'm not sure you can categorically say that the layers in V1 and the layers in a CNN have nothing in common, with respect to the computational capabilities they model. Likewise with convolutions and columns.
However, I doubt that the layers in V1 and CNN have much in common for a couple reasons. First is because the layers in V1 will differ depending on who you're reading from and when they wrote it. The layer divisions are more structural than functional, so far as we know. You can't trace ever more abstract features as you move across V1 layers, not least because V2-V4 are also involved in this.
Next is that V1 sends projections to "higher" and also "lower" areas while connecting to other areas than just the visual. Within V1, connectivity from lower to higher layers are not obvious. Some inputs from the LGN might feed into layer 4 and others to differing layers depending on whether the computation is motion or color etc specific. Layer 5 might also get connections from more distant areas. This level of interconnectivity suggests that very different computations are going on even at the level of just V1. The connections are finer and more intricate than simple bipartite divisions. Basically, they're working out more than just features that are robust to rotations and translations (e.g. speed, direction, dual stream signal integration, oscillations...).
The prevalence and importance of feedback and recurrent connections is not yet well understood but are believed to be important in the sort of high level scene understanding that are still open problems.
And related to that, the notion of inhibition is turning out to be more and more important in biological networks though they aren't really key to any neural nets. Some of their repeating patterns or motifs, have a place in directional and size selectivity but more intriguingly, are figured to be key in the top down flow of information that allows for our (mammalian) complex scene understanding.
There are feedback and recurrent connections in the brain but what (2) means to say that the specific mechanism is not biologically plausible. I know there are prominent Deep learning researchers who are trying to work around this but those explanations have never struck me as particularly parsimonious. It's better to start an explanation from experimental data on brains than from them most important aspect of neural nets.
Mammalian brains are so densely packed that we don't really know what their internal architecture actually looks like--that's why estimates of the number of neurons keeps increasing over time [1]. Most of our models are based on some autopsies, and rely on things like fMRI or electroencephalography (not often you get to spell that word!) to work out which portions of the brain are involved in certain tasks. We have no hope of being able to build a connectivity graph between neurons at our current state of technology.
The short answer is that the cerebral cortex is usually identified as having 6 layers. These can't be correlated with layers in a neural network, however. I do seem to recall a spitball number of 30-ish in relation to the visual cortex, but I've been unable to find any references as to what that number would actually represent, let alone if it's been superseded.
[1] The current estimate is around 100 billion. If you want to know the number involved in, say, the visual cortex, that appears to be on the order of a few billion.
That short answer is very misleading. Those 6 layers are histological layers of density and structure in the cerebral cortex (eg how the cross sections look a little different). You might have many (30+) layers of cells with high interconnectivity in each of those 6 layers. Current estimate is 100 billion for the number of neurons, but the estimate of inter-connectivity is on the order of 100 trillion synapses. The human brain complexity many orders of magnitude higher than anything we have ever "simulated" in silicon.
The brain's layers are both much more loose in their allowed interconnections but also more modular in their functionality. There are different visual areas (usually 5) at different locations of the neocortex that connect to each other (feedforward and back) in complex ways. As residents of the neocortex, each visual area will thus have 6 layers which are in turn dividable into sublayers (often 1 - 4). These layers are structural but are also partially organizational tools for the scientists.
The human brain is also very efficient so it also makes a very good use of those neurons, and modern networks aren't. However, a human brain trains over a very very long time and also prunes itself constantly, while such efficiency techniques at scale for very large networks are being explored in deep learning only recently.
When I read this I interpret it to mean they have no idea how deep learning works. (More specifically: why it works as well as it does.)
And the part about being modeled on the human brain just seems to be a fortunate (?) coincidence.
http://arxiv.org/pdf/1412.6544v4.pdf
http://arxiv.org/pdf/1412.0233.pdf
Btw, "why" questions are always a bit ill-defined, it's like asking for some kind of narrative to satisfy the questioner. "well in the beginning there was the big-bang..." So anyway I'm quite impressed at these papers.
"OK I found this on the web for 'whats next for artful shell in tell gins'"
My basic intuition is that consciousness arises as a byproduct of any sufficiently interconnected (i.e. Complex) network which processes input from the outside world. The problem with AI right now is bottlenecking of both kinds — we don't have computers powerful enough to support the level of complexity of the human brain (100bn neurons with an average of 7k synapses each), and we don't have good enough sensors for capturing the real world (hint: touch is a BIG one).
What sort of game do you have in mind? I definitely think a game in which humans interact with an AI in an artificially-supported information-rich environment (say, the VR of 5-10 years from now) could help provide a suitable environment for "growing" such an AI. Consider the insane amount of Stimulation and calibration human babies require for development! It takes years for the visual system to stablilize, for example.
I think you are underestimating the importance of the motor/action system for the development of consciousness. Many neuroscientists and robotic researchers think that the sensory-motor-environment-sensory loop plays an important role for consciousness. This loop allows you to unsupervised learn all sorts of laws of physics and allows you to think that your body actually is you and not just an extension of your mind and therefore also allows the development of the concept of self awareness. Some researchers might call this sensory-motor-contingencies or embodiment. I think when talking about consciousness there it is very important to take into account this loop and not only the sensory perception and building a neural representation of the world...
The only proof of consciousness case we have (i.e., consciousness arising from Big Bang gases and getting to this conversation) incorporated sense-motor-environment-sense loops. Don't know if it can happen without such interactions, but the one route we know worked, used them.
I definitely should have clarified that point, though, as it's a huge one, I agree!
https://psmag.com/rise-of-the-robot-artist-5aa0e6e1b361?gi=c...
My worry: AIs held only to the moral standards now expected of corporations. Optimize for shareholder value. We're close to this now with machine-learning assisted hedge funds.
It is my worry, as well.
So you call up Nick Bostrom and he very reliably gives you a quote about the existential threat of a superintelligence, even though no one in the industry thinks we're anywhere close to that. (What's next for AI is not superintelligence...) And you force a great researcher like Andrew Ng to talk about job loss among truck drivers, because that's what will make it relevant to people outside AI.
We should be thinking about job loss and how the job market will change, but this type of article never gets past the "Oh No AI Will Destroy US" stage of thinking. But a lot of the questions raised by AI are actually relevant now, in an economy where AI hasn't even made a big impact. That is, they're not AI issues, but we're treating them as though they are. How should our societies treat and support the humans who have become unnecessary? They obviously will not all become data analysts and robot caretakers.
What's next for AI is better natural-language processing. Right now, chatbots are pretty dumb, but in the next few years, they'll get much better, and in more languages.
What's next for AI is the wider deployment of mature technology. Many problems such as image recognition have been solved, but developers and companies have not figured out how to deploy the solutions yet. We still have chokepoints in the number of data scientists who can tune and train models, and the number of engineers who can plug them into existing stacks. So AI will be felt directly, rather than just talked about.[0]
What's next for AI is an arms race. The major powers will be escalating the how they deploy AIs against AIs, embedded in drones or through the creation of adversarial data to slip through filters. Commercially, many smaller arms races will occur in different industries, as AI drives down the costs of interpreting data, and allows rival organizations to compete on price.
What's next for AI is the combination of the flavor of the month, deep learning, with other extremely powerful algorithms like reinforcement learning and Monte Carlo Tree Search to create goal-oriented strategic decision-making agents.
[0] http://deeplearning4j.org/use_cases.html
Yes and no. Neural networks have a lot of nice properties like being easier to grasp, easier to parrallelize(?) and have better tools which i guess is because neural networks are applicable to more problems than SVM's because they are extensible.( see https://en.wikipedia.org/wiki/Recurrent_neural_network for example).
Either way SVM's still do what they do very nicely and in the field of "Human Activity Recognition" where i did my thesis neural networks are practically never used but SVM's pop up from time to time.
Apparently there are things such as structured SVM's and SVM networks, and models where they attach an SVM instead of the output neurons.
It isn't dead, and people are coming up with some pretty cool stuff, but NN seems to be where it is at right now.
Industrial robots have become much faster. Here's Bosch's packaging robot from 2014, looking at small objects and putting them in order for packaging.[1] Then Fanuc built a faster one.[2] Faster CPUs allow using modern control theory that considers dynamics, and fast machine vision. The resulting machines are much faster than humans.
Those are production machines, able to run for long periods. Research robots are even faster.[3] That's from 2012. Progress continues.
[1] https://www.youtube.com/watch?v=BAF-ALWwlLw [2] https://www.youtube.com/watch?v=vtAEIKJLHGw [3] https://www.youtube.com/watch?v=U2sUvQ_HsU8
Everyone seems to believe that achieving artificial general intelligence is inevitable. I'd argue that it's only inevitable if humanity survives long enough to make it happen. I'm not a pessimist, but the next 300-400 years will be the most difficult humanity has ever faced. In addition to climate change, population expansion, nuclear weapon proliferation, and naturally increasing inequality, humanity will face many more threats that haven't yet been perceived.
I believe building strong intelligence will optimistically take 3-4 centuries. To complete a "system that could successfully perform any intellectual task that a human being can," it is first necessary to understand what defines the human intellect and conscience. Although there is much ethical debate on what defines a conscience, scientifically it is a product of experience, and can be emulated if three requirements are met:
- The system can accept every input the human body can. - It can process every combination of stimuli in the same way a human would. - It can to the stimuli in every way a human would.
For these requirements to be met by software, we must either:
- Aquire a nearly full knowledge of the human brain (and probably body)'s information-processing mechanisms, and figure out how to implement this in software. - Build enough processing power to completely simulate the human brain/body, atom by atom.
Both will not be feasible for an extraordinary amount of time, and it's probably better to spend our time worrying about the current existential threats to humanity.
A much more relevant ethical problem for humanity is that of eugenics. Unlike AI, recent advances like CRISPR/CAS-9 make it viable now, through modification of sperm-generating stem cells (not embryos), and like AI, it offers world-changing benefits (the eradication of diseases, lengthened life spans, increased knowledge and strength, etc), while also providing the keys for modern humanity's destruction (designer babies, lack of diversity, separation of humans into casts, etc).
Perhaps increases in intelligence driven by eugenics will even cause computer systems and AI to become obsolete.
This is scary because it means we don't "design" an A.I., we design an adaptive system and allow it to emerge. Nothing could be more dangerous in the long term.