Not to sound overly critical, but I don't enjoy the tone of this paper: presenting the concepts (e.g. graph operations) and features (e.g. automatic differentiation) of TensorFlow as new and novel when in fact very similar systems like Theano have existed since at least 2010 (which they finally get round to mentioning 14 pages in). They really should have shorn the paper in half and just focused on the distributed bit, which is the really novel and exciting bit.
Addendum/Edit: With that said, I don't want to disparage the amazing technical achievement of the Google Brain team and the way TensorFlow really smartly reuses and improves on the concepts that make Theano really powerful. I'm following the project very closely and TensorFlow has a very high chance of being the foundation for my future projects :)
Where does it say that its automatic differentiation feature is new and novel? It actually calls out Theano in the paper:
"Like Theano and Chainer, TensorFlow supports symbolic differentiation, thus making it easier to define and work with gradient based optimization algorithms."
EDIT: Okay, I see that you already saw the reference. I guess I didn't find the tone as bad as you did. :-/
It's confusing but automatic differentiation, symbolic differentiation, and finite differences are all different approaches. "Automatic differentiation" typically refers to reverse order chain rule. Higher level libraries do really neat stuff with it. For example autograd can differentiate much of your numpy calls, even of composed of many complex functions.
Not to sound overly critical ;), but the word "novel" doesn't appear in the paper and the word "new" only appears in tehnical contexts like "quickly experimenting with new models".
The arxiv paper is just a repost of the preliminary paper, which is really more a design outline and rationale -- we may work on a more substantial paper later, we've been busy building the software.
It would be nice if you changed it from "white paper" to something more accurate, like "provisional".
The reason is that "white paper" has a very specific connotation, and this paper is not consistent with the white paper.
Personal opinion: in science the word publication now means "to make accessible" rather "to publish in a conference" (the conference = publication thing is mostly CS) or "to publish in a journal" (like biology and a lot of non-arxiv physics). Pre-pub peer review isn't even required any more. arxiv (and scientists putting preprints on web sites) changed this. Of course, this is personal opinion, based mainly my observations and doesn't necessarily represent the mainstream opinion (yet).
No. But in this case, it doesn't matter much - the paper isn't really making any claims in relation to other systems. The authors can reasonably be trusted to describe the design of their own system, particularly since the source is out there. Peer review would improve the paper, undoubtedly, but wouldn't have any effect on the system design itself or Google's reasons for building it. The latter two are the focus points of the whitepaper.
> it's not a white paper, it's a full scientific publication.
I believe the parent referring to the common usage of "white paper" to refer to B2B marketing materials that are not often written in an academic style.
I was pleasantly surprised to find this was a self-published scientific article and not what I and dekhn usually think of when we hear white-paper.
It is not a full publication yet. The experiment section is left as to-do and that is a very important section in any paper.
They will most likely publish it in the near future.
>> "A computation expressed using TensorFlow can be executed with little or no change on a wide variety of hetero- geneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards."
Are there other comparable systems that deploy using the same build across all platforms?
If not, putting aside other advantages like doing computation across multiple machines, being able to using the same build is a major step forward in my opinion.
Are there other comparable systems that deploy using the same build across all platforms?
SystemML[1] doesn't, AFAIK, claim to scale down to mobile devices, but it does scale from a single desktop machine up to an arbitrarily large distributed cluster, using the same code. If you write code in DML or PyDML, the runtime will take care of distributing it across the cluster. And it's "smart" enough to take the cluster characteristics into account, and can also dynamically re-evaluate stages of the computation as the job is running, so it's always trying to use resources as efficiently as possible.
30 comments
[ 3.0 ms ] story [ 76.0 ms ] threadAddendum/Edit: With that said, I don't want to disparage the amazing technical achievement of the Google Brain team and the way TensorFlow really smartly reuses and improves on the concepts that make Theano really powerful. I'm following the project very closely and TensorFlow has a very high chance of being the foundation for my future projects :)
"Like Theano and Chainer, TensorFlow supports symbolic differentiation, thus making it easier to define and work with gradient based optimization algorithms."
EDIT: Okay, I see that you already saw the reference. I guess I didn't find the tone as bad as you did. :-/
A Google Beta by any other name would smell so sweet.
The reason is that "white paper" has a very specific connotation, and this paper is not consistent with the white paper.
Personal opinion: in science the word publication now means "to make accessible" rather "to publish in a conference" (the conference = publication thing is mostly CS) or "to publish in a journal" (like biology and a lot of non-arxiv physics). Pre-pub peer review isn't even required any more. arxiv (and scientists putting preprints on web sites) changed this. Of course, this is personal opinion, based mainly my observations and doesn't necessarily represent the mainstream opinion (yet).
I believe the parent referring to the common usage of "white paper" to refer to B2B marketing materials that are not often written in an academic style.
I was pleasantly surprised to find this was a self-published scientific article and not what I and dekhn usually think of when we hear white-paper.
Are there other comparable systems that deploy using the same build across all platforms?
If not, putting aside other advantages like doing computation across multiple machines, being able to using the same build is a major step forward in my opinion.
SystemML[1] doesn't, AFAIK, claim to scale down to mobile devices, but it does scale from a single desktop machine up to an arbitrarily large distributed cluster, using the same code. If you write code in DML or PyDML, the runtime will take care of distributing it across the cluster. And it's "smart" enough to take the cluster characteristics into account, and can also dynamically re-evaluate stages of the computation as the job is running, so it's always trying to use resources as efficiently as possible.
[1]: https://systemml.apache.org/
Edit: I see http://mlpack.org/ - anyone have experience using this?
https://github.com/tensorflow/tensorflow/blob/master/tensorf...