Seriously cool. Great work. I did get a glitch every now and then in the rendered output (say 1 out of 5 times) using Safari 10.1.2, MBP touchbar 2016 15", Radeon Pro 460 4096 MB.
TensorFire is up to an order of magnitude faster than keras-js because it doesn't have to shuffle data back and forth between the gpu and cpu. Also TensorFire can run on browsers and devices that don't support OES_TEXTURE_FLOAT.
I'm really interested in using Smartphones / Mobile devices for inference. Can this work with react-native so that I can build it without a bridge ? I would assume I would create a webview that would load a local website.
We're really excited to finally share this with you all! This is the first of a series of demos that we're working to release this week, and we're hoping you'll keep us to that promise :)
Sorry if it doesn't work on your computer! There's still a few glitches and browser compatibility problems that we need to iron out, and we're collecting some telemetry data with LogRocket (https://logrocket.com/) to help us do so (so you all know what kind of data is being collected).
We'll open source the library under an MIT license once we finish writing up the API docs, and fixing these bugs.
Yes, you are correct! Training benefits much more from available memory through batching and, since in many cases you only need to train once, it usually makes sense to train on beefy GPUs.
TensorFire is useful in situations where you want to perform inference, but you don't want to ship user-supplied data to your servers, either because you would run out of bandwidth, you would run out of compute power, or your users want to keep their data private.
I'm not sure if it's working with my browser or not. It says "Compiling network", then shows a lot of flashing rectangles, then stops and displays a single grey rectangle. Is that what it's supposed to do?
It's quite unreal. I remember when the paper and initial implementations came out less than 2 years ago, you had to go through this really long setup process that only worked on certain operating system and was a huge fuss. A few services came out online that would do it for you, but they were slow and limited, with huge queues.
Now, as you mention, you can run it in a few seconds on your phone, or in my case, on my Chromebook, right in the browser, with zero installation. Truly amazing.
I quit working at shoprite and now I make $35h - $80h...how? I'm working online! My work didn't exactly make me happy so I decided to take a chance on something new…after 4 years it was so hard to quit my day job but now I couldn't be happier.
I quit working at shoprite and now I make $35h - $80h...how? I'm working online! My work didn't exactly make me happy so I decided to take a chance on something new…after 4 years it was so hard to quit my day job but now I couldn't be happier.
Very nice work, folks. Impressive, and very well-put-together demo. That's the easiest neural style transfer demo I've ever used - and the most fun. (Other than a minute of worrying that my poor 2013 MBP was about to melt down, but that's not your fault. :-)
This is great! Keep up the good work!
A link to your github repo would be great.
I don't know if it was intentional but I did find your library on npm: https://www.npmjs.com/package/tensorfire
It´s just a demo of an upcoming open source API that allows running deep neural network models on the browser.
Steps (disclaimer: I´m not related to the creators, so this is just what I understand it does)
1.- You upload your image
2.- Select an image to be the origin of the style
3.- Downloading Model: downloads a trained (on style
transfering) deep neural net
4.- Colorful artifacts: the model is applied to your image. Probably the artifacts are a visualization of the network weights being transformed to WebGL shaders, or just a simple visualization of the internal hidden steps of the transformation
It's showing a visualization of all the intermediate activations of the style transfer network. The intermediate pictures are 4D, so they're visualized as a sequence of tiles.
There's a sequence of 9x9 and 3x3 convolutions that transforms that one big input image into a bunch of smaller images. They're processed by a sequence of residual convolutions. Finally, these tiny tiles are merged together back into a stylized image of the same size as the original input with a few deconvolution operations.
From the github issue referenced in the FAQ, I think they mean that because TensorFlow only natively supports CUDA, TensorFire may outperform TensorFlow on computers that have non Nvidia GPUs, such as the new MacBook Pro.
I think the goal is to allow people to develop webapps with models built using neural network libraries like keras and tensorflow. This would greatly improve the distribution of applications that are powered by deep learning because you wont have to install a bunch of dependencies in order to use the app.
This is amazing. I can't use GPU Tensorflow (natively) on my Macbook Pro because it doesn't have an NVIDIA graphics card. But I can... in the browser! Honestly didn't see that one coming.
To be clear, you can use it - just without GPU acceleration. The CPU-only build is supported and should work for you. If it's not, please let (me, us) know. Be sure to compile with AVX2 if you're on haswell or later; it helps quite a bit with some models.
Actually it seems like WebGL is doing it even faster. Which makes sense - machine learning involves a lot of matrix math, which GPUs are made for and CPUs aren't.
This is actually what I'd expect, but the website feels quite misleading. Advertising that a GPU-based approach can outbeat a CPU for neural nets is not a very strong commercial claim :)
I've played around with doing some computation in WebGL, but it was rather tedious and difficult with my limited knowledge about the topic. It's possible, but you can't even rely on floating point texture to be available on all systems, especially mobile. And for anything more complicated, you probably need to be able to render to floating point textures, which is even more rare than support for plain floating point textures.
This only makes it more impressive when people do cool computational stuff in WebGL, but I'd wish there were some easier ways for non-experts in shader programming to do some calculations in WebGL.
Cool! Awesome! New MIT grads playing with JS who think "this whole deep learning thing is pretty neat"!
Most "normal" humans -- and application developers -- will just see this as "yet another webapp" running some crazy ass code in a browser tab with no apparent rhyme or reason.
We've gotta stop the insanity! We need regulation! We need: "Linux on the Web" (a web-based OS that I've been developing for the past 5 years, see it here: https://linuxontheweb.org) to start putting this kind of crazy, experimental stuff into actual use.
I'm really not trying to be too much of a jerk, but I've been working on my project for about as long as it takes to get an entire college degree, and I'm going to start needing some help here!
90 comments
[ 3.3 ms ] story [ 172 ms ] threadWe will probably release it under an MIT license.
It would be good if you had a comparative benchmark on the website.
[0]: https://mil-tokyo.github.io/webdnn/
You can also sign up for the mailing list if you'd like us to email you when the repo goes live!
We're really excited to finally share this with you all! This is the first of a series of demos that we're working to release this week, and we're hoping you'll keep us to that promise :)
Sorry if it doesn't work on your computer! There's still a few glitches and browser compatibility problems that we need to iron out, and we're collecting some telemetry data with LogRocket (https://logrocket.com/) to help us do so (so you all know what kind of data is being collected).
We'll open source the library under an MIT license once we finish writing up the API docs, and fixing these bugs.
It looks like it (like keras-js) is only for inference (running already-trained models) and not for training. Is this correct?
Are the operations or memory required for training very different?
TensorFire is useful in situations where you want to perform inference, but you don't want to ship user-supplied data to your servers, either because you would run out of bandwidth, you would run out of compute power, or your users want to keep their data private.
Extremely impressed. Keep it up!
Now, as you mention, you can run it in a few seconds on your phone, or in my case, on my Chromebook, right in the browser, with zero installation. Truly amazing.
........... http://ow.ly/iBXm30dNtIZ
........... http://ow.ly/iBXm30dNtIZ
The download link failed, as others have noted.
Thanks so much for sharing this!
Steps (disclaimer: I´m not related to the creators, so this is just what I understand it does)
1.- You upload your image
2.- Select an image to be the origin of the style
3.- Downloading Model: downloads a trained (on style transfering) deep neural net
4.- Colorful artifacts: the model is applied to your image. Probably the artifacts are a visualization of the network weights being transformed to WebGL shaders, or just a simple visualization of the internal hidden steps of the transformation
5.- You get your image with the style applied
The network being run is defined here https://github.com/lengstrom/fast-style-transfer/blob/master...
This post provides a pretty good explanation of what's happening: https://shafeentejani.github.io/2017-01-03/fast-style-transf...
There's a sequence of 9x9 and 3x3 convolutions that transforms that one big input image into a bunch of smaller images. They're processed by a sequence of residual convolutions. Finally, these tiny tiles are merged together back into a stylized image of the same size as the original input with a few deconvolution operations.
could you elaborate on this statement ?. What kinds of architectures does this hold true for ?.
If so that's amazingly clever!
Tensorflow should add a WebGL backend that runs in NodeJS.
[1] https://aigrant.org/#finalists
>> framebuffer configuration not supported, status = undefined
> You can learn more about TensorFire and what makes it fast (spoiler: WebGL)
Does this mean that using a GPU in a browser through WebGL yields the same speed than a desktop CPU?
I wanted to download the resulting image but got a "Failed - Network" error :(
Happening in both Firefox and Chrome on Ubuntu. What exactly am I missing here?
Firefox: about:support
Chrome: chrome://gpu
This only makes it more impressive when people do cool computational stuff in WebGL, but I'd wish there were some easier ways for non-experts in shader programming to do some calculations in WebGL.
It should only take five minutes or so to apply.
Quick question: is the code compiled from js to webgl in browser as well, or do I need to compile beforehand?
I see this as a great way to learn and teach AI without having to bring a large toolchain.
Edit : it seems it is just a runtime for now for Tensorflow models!
Most "normal" humans -- and application developers -- will just see this as "yet another webapp" running some crazy ass code in a browser tab with no apparent rhyme or reason.
We've gotta stop the insanity! We need regulation! We need: "Linux on the Web" (a web-based OS that I've been developing for the past 5 years, see it here: https://linuxontheweb.org) to start putting this kind of crazy, experimental stuff into actual use.
I'm really not trying to be too much of a jerk, but I've been working on my project for about as long as it takes to get an entire college degree, and I'm going to start needing some help here!
Windows7, Firefox 54(64bit)