What is the difference between data scientist and statistician? Have you ever seen data scientist, like with a PhD? Have you ever seen statistician with PhD like https://statistics.wharton.upenn.edu/programs/phd/ ?
My best guess is that a statistician is a focus more on mathematics/pure statistics while data scientist is involved in more algorithms.. but the lines are blurred and I don't know what we would call people like Nate Silver. According to Wikipedia he's a statistician.
Statisticians are usually preoccupied with understanding the properties of a data set. There is a penumbra of activities which are associated with statistics that have become relatively more important as technology has changed - getting the data, managing the data and transmitting the understanding of the data to humans. Most of the people who successfully market themselves as pucker data scientists do have Ph.D's - often in Machine Learning, but sometimes in Maths or HCI.
It is simple to see the progression of the nomenclature; companies used to employ staticians, then analysts, and now data scientists. There are subtle differences that have evolved recently, today's data scientists often are employing ml and spending time on feature engineering.
I would like to add where I teach[1], our data scientists get hired at places like tesla . Many of our students tend to be bachelors or phds. You don't need an phd to do data science. It is more about hands on skills.
Edit: Cabinpark is right and I should clarify. Statisticians typically don't have the proper computer science fundamentals to be able to deal with the programming required to run the right experiments. They may have an understanding of the distributions, t tests, ... but may not be able to use the tools out there that the broad field of data science requires.
When I hear the term data scientist, I assume that the person has extensive training and background in statistics and mathematical modeling. I work with large data sets all the time as a scientist and can run all the standard statistical tests but I would hardly consider myself a data scientist.
Realistically, most "data scientists" that people hire aren't going to have that full background.
I think the problem with a lot of data science teams out there today is the hiring. Not a lot of people understand what the role of a data scientist should be and they expect these people who can do hadoop end to end, extensive CS skills, know all of the latest advanced machine learning algorithms, and know stats like the back of their hand.
Many employers will not need or use that full pipeline, and if they do, they are probably capable of hiring hadoop engineers as well as more traditional analysts who are deeper in the math with the models.
That analyst is someone with a specialized background and some form of training in scientific computing. This does not need to be a full blown masters/phd.
Realistically, you can get away with having a decent programming background, a clue about the landscape of the machine learning algorithms, and enough statistics to know when you're going down a rabbit hole with respect to research.
I think it more comes down to having the right mindset with problem solving. Much of this is also going to be domain specific.
The term data science is ambiguous at best, as it is a new field. I think over the next few years we'll come to see more specialized roles over time that will help clarify the sandbox that is data science vs data engineering among other disciplines.
I think a subset of that program would be rightly called data scientists. There is significant overlap, but statisticians are stronger in mathematical theory while data scientists are stronger in programming and computation.
At least that's how I see it. It's not particularly well defined at the moment, and a lot of people throw these words around because they're new and fun.
Thinking people (as oppose to those dedicated to promotional activities) should stop using the term "data scientist". All scientists are data scientists, otherwise we would call them philosophers. Data for the sake of data is not a science. While you are at it, please also stop using the term "big data", (often people mean: do something with the data), if you need to use a computer cluster and MapReduce because the data doesn't fit in your Mac, then refer to distributed data stores and computing systems. Also, please drop the term "deep learning" when you refer to using more compute power to run more complex models. Thanks.
Deep learning is suss, I think, but I think that the name is apt - learning more than 3 layers in a network is Deep...
I spent a lot of time ~1995 trying to learn complex neural networks and I have an appreciation of the difficulties and the opportunity that big modern clusters therefore afford us.
"Deep learning" has an actual meaning, which is the use of neural networks with multiple hidden layers. (Networks with one hidden layer can theoretically approximate any mathematical function, but it's the investigation of deeper networks, with more, that has reinvigorated neural net research over the past few years). I'm sure it is being misused, but there is a legitimate, technical meaning to it.
"Data scientist" seems to be a way for mathematically literate programmers to separate themselves from the teeming masses of commoditized ScrumDrones. It seems to mean, "this person is smart enough to deserve dibs on the most interesting work". Perhaps it's an attempt to back to the R&D culture that existed before biztards commoditized us and our work.
Most of the fuss around "data science" makes me think of the Fundamental Theorem of Employment. If you're hired for a job, it's typically either (1) to do a job the person hiring you can't do for himself or (2) to do a job he doesn't want to do. Type-1 workers are respected and have autonomy. Type-2 workers are generally ill-regarded (because the boss thinks he can do the worker's job). "Data Scientist" seems to be a way for a programmer to say, "Only hire me for Type-1 work".
I can't say I'm a huge fan of the title's existence, because most companies use "data scientist" as Biztard for "person who does watered-down machine learning", but I suppose the current climate is an improvement over the AI winter.
Yes, the latest work on NN is a breakthrough for sure. So are the advancements in distributed computing and storage that make low-cost scalability possible. However, we should resist getting sucked into marketing terms and buzz words. Terms like "self-driving car" are good because they are descriptive, accurate, and imply a paradigm shift. On the other hand, I may be wrong, for example the term "microprocessor" seems to have trascended relative size and is used to refer to a type of computer processor on an integrated circuit. Language evolves but perhaps we can influence by choosing good meaningful names when we can.
I am a scientist and these silly buzz words, while they're great for salesmen that work for businesses, drive me up the wall!
Regarding "deep learning", you are totally right. Deep learning refers to a particular formulation that may inspire development of a different class of algorithms.
Hi, I'm an adjunct instructor at the data science bootcamp Zipfian Academy[1].
It seems there's still a lot of confusion as to what a data scientist is.
Data Scientists are typically analysts who know some combination of matlab/python/R that come up with predictive models to achieve some sort of business objective.
This is usually related to a businesses' profit center. The work is typically anything beyond A/B testing using basic classifiers to figure out things like churn prediction, handling data quality, all the way to doing object recognition.
Data Engineers typically work on the JVM/distributed systems to handle data at scale and implement models for data scientists in production.
With respect to deep learning, I'm also the author of a java based distributed deep learning solution called deeplearning4j[2]
I think people throw around deep learning because it's the new thing, but it really can be more accurate.
This is due to not needing to do feature engineering[3] and feature extraction[4]
Also, if you're interested, I will be giving talks at both hadoop summit[5] and OSCon[6] this year around distributed deep learning if you have any specific questions as to what all of this stuff is. Data Science is an amazing field to be in right now.
For those of you who are interested in neural networks in general, Ersatz holds a great meetup (videos/tech talks recorded!)
Distributed GPUs ;). I'll be adding GPU support here shortly. First I'm working on the core nets, since everything is vectorized, swapping out the matrix implementations will be easy. Mainly focusing on a point release first. Just need to add Recursive Neural Tensor Nets[1]. Afterwards, distributed GPUs will be cake with the support that both NVIDIA[2] and AMD[3] support.You will find distributed compute is what google does[4]
Since Ersatz got mentioned, I'd like to point out that we don't do true distributed GPU yet (it's planned). Most of the models support a single GPU with the RNN models supporting multiple GPUs on the same machine. Of course, that's still way way faster than your laptop.
Also here: http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC... In this case, they are using infiniband, a high speed ethernet card/bus/whatever-you-call-it, in order to share data very quickly between multiple systems. This is basically like google's experiment but w/ GPUs (and thus way cheaper?)
I'll let Adam speak to his strategy. It's an open research question but that doesn't mean there aren't any options or that those options are even terribly difficult given the right resources. Of course, real world performance is another story... whether you need and/or can use all that capacity, etc.
A. Krizhevsky put out a pretty interesting paper recently on this same topic. Also, it has the best research paper title I have seen [1]
By changing the type of parallelism depending on where in the network you are (fully connected vs. convolutional layers), you can take advantage of layer properties to optimize the sharing between GPUs. Whether that reduction in data movement is enough to make it feasible for AWS/local LAN links is up for debate.
Sure! So my stack currently uses iterative reduce[1]. The algorithm can then be swapped out which can either be gradient descent or conjugate gradient. The default is conjugate gradient. I also have all of the knobs like adagrad, drop out,....
In general, the idea is that you split up the training in to mini batches and then average the parameters after all of them have been computed.
I will be working on downpour SGD as well.
In my case, I use akka underneath for all of the distributed computing. Everything is inherently parallelizable using an optimal batch size algorithm (very similar to hadoops). From there, it's usually just a case of figuring out what you want your matrix computations made of.
I also built it from the ground up to speak pretty much any data format.
The idea is using a DataSetIterator that speaks hdfs, local file systems,.... This allows you to handle data locality independent of the algorithms.
If you want more on the architecture, here's a pretty broad overview[2]
Admittedly, not optimal, this is a very early release of the lib. After RNTN have been impled, I will be focusing on optimizations and improvements.
Little late on the reply. It's non stop 9-6pm everyday for 3 months. This would be you quitting your job to learn the whole data science stack (SQL all the way through machine learning)
Since the focus is on practical/hands on, with only the needed theory, concepts are retained enough on a practical level to become a productive junior data scientist.
Note that our acceptance rate is also very low though. We are bringing in people who already will have phds or some sort of a software engineering background. The job placement is worth it though. We are seeing avg salaries of 115-120k starting. The problems being solved are also really interesting.
31 comments
[ 2.4 ms ] story [ 69.4 ms ] thread[1] http://www.zipfianacademy.com/
Edit: Cabinpark is right and I should clarify. Statisticians typically don't have the proper computer science fundamentals to be able to deal with the programming required to run the right experiments. They may have an understanding of the distributions, t tests, ... but may not be able to use the tools out there that the broad field of data science requires.
When I hear the term data scientist, I assume that the person has extensive training and background in statistics and mathematical modeling. I work with large data sets all the time as a scientist and can run all the standard statistical tests but I would hardly consider myself a data scientist.
I think the problem with a lot of data science teams out there today is the hiring. Not a lot of people understand what the role of a data scientist should be and they expect these people who can do hadoop end to end, extensive CS skills, know all of the latest advanced machine learning algorithms, and know stats like the back of their hand.
Many employers will not need or use that full pipeline, and if they do, they are probably capable of hiring hadoop engineers as well as more traditional analysts who are deeper in the math with the models.
That analyst is someone with a specialized background and some form of training in scientific computing. This does not need to be a full blown masters/phd.
Realistically, you can get away with having a decent programming background, a clue about the landscape of the machine learning algorithms, and enough statistics to know when you're going down a rabbit hole with respect to research.
I think it more comes down to having the right mindset with problem solving. Much of this is also going to be domain specific.
The term data science is ambiguous at best, as it is a new field. I think over the next few years we'll come to see more specialized roles over time that will help clarify the sandbox that is data science vs data engineering among other disciplines.
I think a subset of that program would be rightly called data scientists. There is significant overlap, but statisticians are stronger in mathematical theory while data scientists are stronger in programming and computation.
At least that's how I see it. It's not particularly well defined at the moment, and a lot of people throw these words around because they're new and fun.
I spent a lot of time ~1995 trying to learn complex neural networks and I have an appreciation of the difficulties and the opportunity that big modern clusters therefore afford us.
"Data scientist" seems to be a way for mathematically literate programmers to separate themselves from the teeming masses of commoditized ScrumDrones. It seems to mean, "this person is smart enough to deserve dibs on the most interesting work". Perhaps it's an attempt to back to the R&D culture that existed before biztards commoditized us and our work.
Most of the fuss around "data science" makes me think of the Fundamental Theorem of Employment. If you're hired for a job, it's typically either (1) to do a job the person hiring you can't do for himself or (2) to do a job he doesn't want to do. Type-1 workers are respected and have autonomy. Type-2 workers are generally ill-regarded (because the boss thinks he can do the worker's job). "Data Scientist" seems to be a way for a programmer to say, "Only hire me for Type-1 work".
I can't say I'm a huge fan of the title's existence, because most companies use "data scientist" as Biztard for "person who does watered-down machine learning", but I suppose the current climate is an improvement over the AI winter.
I am a scientist and these silly buzz words, while they're great for salesmen that work for businesses, drive me up the wall!
Regarding "deep learning", you are totally right. Deep learning refers to a particular formulation that may inspire development of a different class of algorithms.
It seems there's still a lot of confusion as to what a data scientist is.
Data Scientists are typically analysts who know some combination of matlab/python/R that come up with predictive models to achieve some sort of business objective.
This is usually related to a businesses' profit center. The work is typically anything beyond A/B testing using basic classifiers to figure out things like churn prediction, handling data quality, all the way to doing object recognition.
Data Engineers typically work on the JVM/distributed systems to handle data at scale and implement models for data scientists in production.
With respect to deep learning, I'm also the author of a java based distributed deep learning solution called deeplearning4j[2]
I think people throw around deep learning because it's the new thing, but it really can be more accurate.
This is due to not needing to do feature engineering[3] and feature extraction[4]
Also, if you're interested, I will be giving talks at both hadoop summit[5] and OSCon[6] this year around distributed deep learning if you have any specific questions as to what all of this stuff is. Data Science is an amazing field to be in right now.
For those of you who are interested in neural networks in general, Ersatz holds a great meetup (videos/tech talks recorded!)
Happy to answer questions!
[1]: http://www.zipfianacademy.com/
[2] http://deeplearning4j.org/
[3]: http://www.cs.princeton.edu/courses/archive/spring10/cos424/...
[4]: http://en.wikipedia.org/wiki/Feature_extraction
[5]: http://hadoopsummit.org/san-jose/speakers/
[6]: http://www.oscon.com/oscon2014/public/schedule/detail/33709
[7]: http://www.meetup.com/SF-Neural-Network-Afficianados-Discuss...
Edit: I'd like to mention ersatz[5] here as well.
Edit: Thanks for the correction David.
[1]: http://nlp.stanford.edu/sentiment/
[2]: http://www.jcuda.org/jcuda/jcublas/JCublas.html
[3]: http://developer.amd.com/tools-and-sdks/heterogeneous-comput...
[4]: http://research.google.com/archive/large_deep_networks_nips2...
[5]: http://ersatzlabs.com
Distributing computation across GPUs has been addressed here: http://techblog.netflix.com/2014/02/distributed-neural-netwo... We looked at a similar scheme, but didn't see it as practical on something like AWS
Also here: http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC... In this case, they are using infiniband, a high speed ethernet card/bus/whatever-you-call-it, in order to share data very quickly between multiple systems. This is basically like google's experiment but w/ GPUs (and thus way cheaper?)
I'll let Adam speak to his strategy. It's an open research question but that doesn't mean there aren't any options or that those options are even terribly difficult given the right resources. Of course, real world performance is another story... whether you need and/or can use all that capacity, etc.
By changing the type of parallelism depending on where in the network you are (fully connected vs. convolutional layers), you can take advantage of layer properties to optimize the sharing between GPUs. Whether that reduction in data movement is enough to make it feasible for AWS/local LAN links is up for debate.
[1] http://arxiv.org/abs/1404.5997
In general, the idea is that you split up the training in to mini batches and then average the parameters after all of them have been computed.
I will be working on downpour SGD as well.
In my case, I use akka underneath for all of the distributed computing. Everything is inherently parallelizable using an optimal batch size algorithm (very similar to hadoops). From there, it's usually just a case of figuring out what you want your matrix computations made of.
I also built it from the ground up to speak pretty much any data format.
The idea is using a DataSetIterator that speaks hdfs, local file systems,.... This allows you to handle data locality independent of the algorithms. If you want more on the architecture, here's a pretty broad overview[2]
Admittedly, not optimal, this is a very early release of the lib. After RNTN have been impled, I will be focusing on optimizations and improvements.
[1]: https://github.com/jpatanooga/KnittingBoar/wiki/Iterative-Re...
[2]: http://deeplearning4j.org/architecture.html
Since the focus is on practical/hands on, with only the needed theory, concepts are retained enough on a practical level to become a productive junior data scientist.
Note that our acceptance rate is also very low though. We are bringing in people who already will have phds or some sort of a software engineering background. The job placement is worth it though. We are seeing avg salaries of 115-120k starting. The problems being solved are also really interesting.