100 GFLOPS can process deep learning predictions on HD video in real time? I thought even GPUs like the GTX 1080 (8.8 TFLOPS) had difficulty with that.
It's not very much. 100 GFLOPs is about 1-2 Haswell/Broadwell cores (32 FLOP/s * ~2 GHz AVX frequency). Even 2015 era mobile GPUs like the Apple A9 GPU (iPhone 6S) is higher throughput (~150-200GFLOP/s achievable).
If you're executing these models at 30FPS on an HD stream (say ~1mm pixels/frame), you have roughly a 100 * 10^9 / (30 * 10^6 ) = 3,000 flops/pixel budget if you're operating at peak, which is about the same flops/pixel of a 16 channel -> 16 channel 3x3 conv. In practice of course you can usually operate on a much smaller spatial input.
With customized model u can do that. Models like MobileNet/SqueezeNet are designed specifically for fast inference on constrained device. Although during inference, the precision of your model can be quantized as low as single digit bits. This is an active area of research, but a lot of exciting stuff is happening and the progress made is massive.
Most of the time you don't need pixel-level classification. Suppose you want to make a cat tracker. How small can you shrink an HD image and still reliably detect a cat (using your own brain)? Pretty small. Probably less than a quarter of the size at least. Do you need colour? Often you can get away without it, so that's a third of the data gone too. For full-image classification you could go even smaller and still be accurate.
This is neat but sort of strange - I love the idea of a hardware package that makes fiddling with deep learning easy, but I wonder why Amazon went through the trouble of producing such a relatively niche product?
If I had to guess, this is going to be a great "user education" tool for AWS, designed to get new developers on the platform as early in the learning process as possible.
Because they open everything they develop and in this case they want to see innovations happen around their new home cameras! Which they use already with deliveries.
Do you really think its niche? Sure its not going to be a GoPro consumer product but the market for "smart video cameras" or whatever you want to call video+insight is going to be pretty darn big. This seems like this easiest foray into combining a video camera with learning with limited bandwidth issues. My only concern is whether or not Google will come out with something similar in the near term.
Surely Google will come with one. Or something under Android umbrella. Only concern is, Google's hardware is at best meh level, while Amazon had better reputation in that front.
Considering the engineering and manufacturing investment in this product, it's hard to see where they are expecting their ROI. It certainly won't be from the product sales themselves, and since the target market seems to be individual as opposed to corporate developers, I wonder if increased use of AWS is going to generate enough revenue to offset the costs.
Jetson TX1 is much expensive than this one, though. As with most phones. And most importantly, this looks like a Dropcam style camera, I think the expectation of use case is different here.
Jetson is a dev kit. Pricing for a consumer product with the same chip is lower. You can get a Shield TV box for $180 today; plug in a webcam and it's equivalent to this Amazon thing but cheaper with 10x the performance. For a video processing application that extra performance is going to be essential.
This is great for my chicken coop project :) I have been working with a raspberry pi and camera to train a system to recognize and respond to squirrels (close the feeder to stop them eating food). There are many other options to build this, but it is nice that this is an integrated system that is easy to train!
Sure will do, mostly I've been training squirrel recognition models on my laptop and also learning what it takes to run those models on the raspberry pi (low frame rates) with OpenCV. I did also buy one of these though https://www.adafruit.com/product/997 so once I get everything running I may also have videos of squirrels being sprayed by the garden hose :)
If I may give some dog trainer advice, you are going to spend water and energy making some squirrels just a bit more challenged to the feat, consider just attracting the offending ones to elsewhere where they find some cheap/disposable food source so they let the chicken's alone :-) — this approach is usually cheaper and renders more results with dogs, cats and even chickens themselves! Say, once you detect them, you lure them to a certain path or place, maybe with some LEDs or scent or something you can automate dispensing (I don't known squirrels).
It's it kind of funny... one of the first home-brew CV projects I ever saw was an engineer's system that would unlock the cat flap door when it detected the cat's silhouette. A desirable side effect is that it would not open the door when the cat was carrying a rodent corpse.
Nice! I bet you could start a business with this if you get it robust. I have been thinking about a bird feeder that works only for birds below a certain weight like hummingbirds. We often have crows taking our feeder apart.
That, as well as this new "AWS DeepLens", look like interesting kits/toys to try out new on-device and hosted systems for voice processing and computer vision.
Then again, I assume you have to buy these knowing that the hardware will likely cease to work in the next year or two or three... or whenever the hosted aspects change, and AWS and Google move on to their next trend.
Well, with the Google AIY kit, as it's Raspberry Pi based, you have the ability to do what you want with the hardware. It's really just a microphone and speaker on top of it.
At some point you could run offline models to take voice input and output intent. What you then do with that is up to you. What the cloud services provide are common lookups, such as searching wikipedia, your calendars, etc. You'd need to implement that yourself.
AWS DeepLens seems to run Ubuntu 16.04 under the hood, but it's unknown whether you can get SSH access to it. If so, it's could be a really nice hackable camera on which you could run anything you want.
Intel Atom processor? What a silly choice! A Tegra would be literally 10x faster for machine learning. Why on Earth would anyone choose Atom for this application?
Much better hardware for this application would be an NVIDIA Shield TV box and a USB webcam. 10x the performance for cheaper.
You can do much more with your phone's camera. I think a better use case would have been an SDK for Android/iOS. There are several open source libraries that you do a lot more interesting that simple object recognition.
Perfect! I will train one of these to watch my pained facial expressions when Alexa is triggered without using a keyword. Then it will fire a speech script "Bad Alexa."
Shameless related plug : We recently developed a deep-learning powered 2-axis-gimbal for a webcam : https://github.com/GistNoesis/Linn-Photobooth . We do use deep-learning either for deep-art or for pose tracking.
I guess we can probably control the gimbal with this new camera though I'm not sure It will be able to process deep art.
In my other usages, usually I have multiple cheap webcams with cheap board (IP cams or other) streaming it (over wifi/ethernet) to the GPU processing station located somewhere. With powerful GPU boards at €650, it's probably a better strategy once you start having plenty of cams.
That sounds cool, but after reading the webpage I am still not sure what it actually does. It prints a picture, you can change art style, what's the point of the gimbal?
The gimbal allows you to orient the camera (avoid people to touch the lens with greasy fingers), it can be substituted by a cheap 3d printed piece for manual adjustment if the functionality is not needed.
It has some awe effect, and is fun with kids. It makes the camera more alive, specially in the tracking mode.
The gimbal allows you to shoot videos where you are always at the center. We think this may grant some freedom of movement to some video makers and make their video more dynamic without the need of a buddy filming them.
Also this project has a DIY component to it, so doing the gimbal is a great introduction to robotics.
I know at the military level it does in the Phalanx https://en.wikipedia.org/wiki/Phalanx_CIWS which basically shots a stream of lead which must require one hell of a gimble!
I was imagining hooking up an old airsoft gun to one, legal, cheap, effective deterrent for small animals.
This is another example of tech beating the hell out of laws.
The tiny JeVois smart camera has been doing this for a while now, and lots of other kinds of computer vision too, although in a smaller and lower resolution device, also much cheaper at $49 and fully open source.
48 comments
[ 1.7 ms ] story [ 120 ms ] threadIf you're executing these models at 30FPS on an HD stream (say ~1mm pixels/frame), you have roughly a 100 * 10^9 / (30 * 10^6 ) = 3,000 flops/pixel budget if you're operating at peak, which is about the same flops/pixel of a 16 channel -> 16 channel 3x3 conv. In practice of course you can usually operate on a much smaller spatial input.
If I had to guess, this is going to be a great "user education" tool for AWS, designed to get new developers on the platform as early in the learning process as possible.
https://web.archive.org/web/20080913222639/http://www.quantu...
That, as well as this new "AWS DeepLens", look like interesting kits/toys to try out new on-device and hosted systems for voice processing and computer vision.
Then again, I assume you have to buy these knowing that the hardware will likely cease to work in the next year or two or three... or whenever the hosted aspects change, and AWS and Google move on to their next trend.
Any other similar kits/toys others have found?
At some point you could run offline models to take voice input and output intent. What you then do with that is up to you. What the cloud services provide are common lookups, such as searching wikipedia, your calendars, etc. You'd need to implement that yourself.
AWS DeepLens seems to run Ubuntu 16.04 under the hood, but it's unknown whether you can get SSH access to it. If so, it's could be a really nice hackable camera on which you could run anything you want.
Much better hardware for this application would be an NVIDIA Shield TV box and a USB webcam. 10x the performance for cheaper.
I was imagining hooking up an old airsoft gun to one, legal, cheap, effective deterrent for small animals.
This is another example of tech beating the hell out of laws.