Launch HN: Piccolo (YC W18) – Camera for controlling your home with gestures
The two of us have had an interest in computer vision for a long time and were in Udacity’s first self-driving car nanodegree cohort in 2016. We started this as a side project to control one lamp and soon had our entire house connected. For some actions, we found gestures to be much faster and more intuitive. For example, pointing at a lamp to turn it on is way more natural than saying “Hey Alexa, can you turn on my left living room lamp?”
To set up Piccolo, you can place it anywhere (near the TV is usually best), and then on the app you can indicate with bounding boxes where the devices are. After that, you connect those same devices (Chromecast, Hue lights, smart plugs, etc.), and you’re good to go. Some processing happens on-device, but the more complicated models are run in the cloud. Since we’re not a security camera, there’s no need to store video and so no image/video data is ever stored.
We’re excited about the experiences you can build when you have a camera and apply computer vision techniques. With recent progress in human pose estimation, object classification, and object tracking, there’s really a lot you can do. We’re starting out with gestures, but our goal is to build a platform that lets anyone create and deploy vision apps. Here's a few things we're excited about:
- New apps. For example an app that detects medical emergencies (like an elderly person falling). We'd also love an app that can tell you where you left your phone and keys.
- App integrations. For example, letting Netflix know which people are in the room to get tailored recommendations for everyone vs. just the person signed in.
- Smarter hardware. For example, an Espresso machine that, with one click, makes your favorite drink because it knows who pressed the button.
- Voice-vision fusion. You should be able to trigger Alexa just by gazing at the Alexa device instead of saying "Alexa". You should also be able to hold something and say "Order 5 more of these".
We're giving away 20 pre-release units next month to anyone that joins the waitlist. We’re happy to answer any questions and look forward to your feedback. If you want to follow up, our emails are marlon@piccololabs.com and neil@piccololabs.com.
56 comments
[ 5.8 ms ] story [ 112 ms ] threadWe did the course together but met many years before, in high school in Canada in '07.
Looks very neat. If you open the API, I’d love to play with it!
EDIT: you answered my question with DensePose. Had no idea that exists!
I'm slowly adding home automation to my house, but I'm a little more privacy oriented. I totally recognize I might not be your target audience with some of my constraints. Couple questions: If the service is paid (monthly or for a 1 time app purchase); how much is getting published to your servers? Because I like the control over something as intrusive as cameras in my house I'd prefer to self-host as much as I can.
Does it have to be the camera you provide/sell or could a high enough quality camera do that?
This is a massive problem with home automation tools today. I need to rely on Comcast to stay alive to be able to turn lights on and off in my house? To me, it's an absolute dealbreaker. I've spent more time than I feel like I should have had to in order to control devices I own without any data leaving my LAN. Am I the only one who feels like it's a totally absurd situation?
It cost $300 I think. That's about the same price as the Google WiFi 3-pack. /shrug
Give me (a power user) something I can run in a docker container locally, or on any of my local Macs or PCs.
Or, assume as a constraint that smart homes need to be able to run themselves. I would appreciate if everyone in the space made an effort at detente while the technology matured to a point where a massive privacy and security hole wasn't required in order to have the thing work at all.
To me, the bigger worry is having cloud connected video cameras (or microphones) in my home at all. Obviously many people are happy to trade privacy for convenience, but this this is a red line for me. There are too many ways for it to get deeply dystopic really quickly.
It depends on your definition of "usable".
For instance, if I want to tie 3 bulbs together (e.g. treat them as a single bulb for on/off/brightness/color) in SmartThings, that has to go to the cloud. That was when I stopped trying to make things work in SmartThings.
Comparing ST (which costs $$) to HA (which is free) is a no-brainer; HA will run Python scripts (or shell out) on hardware I own, on my LAN.
We, the users, should refuse to pay for anything that doesn't actually let us control our homes. The case for FOSS is stronger here than it's ever been.
I did my final year engineering project with the Kinect (years ago!) and controlled a robotic arm to pick up an egg. Didn't know how else to apply this, but it seems like you guys really thought outside the box.
Only thing is the name...not sure about naming after the green alien from DBZ.
I think something that people don't realize until they start using motion sensing technology is that how intuitive it can really be. It might look awkward at first, but once it's seamlessly integrated (ie. stepping into the sensor's view), it's really just like magic. A good enough sensor can pick up and understand very specific motions.
Anyway, good luck guys.
PS, please make this gesture an easter egg: https://qph.fs.quoracdn.net/main-qimg-3671587cdc2afad44da412...
Anyway, I'd be interested to know how you're dealing with the depth problem. For example, if there's a light behind you and a light above you, pointing up looks like you could be pointing at either light.
(For the convenience, I ended up using a Google Home, and I'll likely switch to an Apple HomePod next as they seem the most trustworthy with your data.)
If Piccolo will make money through data (like Amazon or Google), there's virtually no way for it to be non-intrusive unlike with audio, as we'd just be using it for turning off our lights rather than searching for the nearest French restaurant. Data would have to come from visual recognition. Hopefully the lack of connection needed to a cloud in this situation will result in more security? Anyway, curious to see how it works.
Like does the cloud just trigger events and this it, or does it keep a log of where people are located in the scene and whether they have their arms crossed, etc?
Of course these models are getting smaller over time and I'm incredibly impressed that these guys have put together the hardware, computer vision, and cloud setup. I also think they've nailed the MVP - not too easy but not too complicated either assuming they have decent models.
I'm signing up!
Was the Kinect even a neural network? I don't think it was.
A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wave bands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive--you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure, of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same program.
Zaphod waved a hand and the channel switched again.
To understand the problem faced here it's worth trying to just watch people. Guess what people are saying to each other.
I got a 502 and "Oops! Something went wrong while submitting the form" initially though but it worked on the second try.
Hope you guys all the best. This is so cool, I am on the waiting list!
It could recognise a couple in a dancing pose and randomly play something appropriate. It could recognise heroic arms aloft and play applause. So much potential.
There are countless things we do daily that have loads of room to be simplified. Even getting an Apple TV where the remote can switch on the TV and the device, and auto-change the TV's source to the device input felt like a game-changer, but this could ramp that up significantly.