So yeah, we need money for servers, and to pay for rent / food / etc for the several months it'll take us to make this thing.
When you start doing the math, a lot of the money we're asking for disappears. 10% of what we raise goes to Kickstarter and Amazon (for payment processing). Another 30% - $35% goes to paying for the Kickstarter rewards, and then the government takes it's cut. What's left is a minority of the money.
Awesome! I'd always wanted to do something like that with a site of mine, http://meteo-veneto.net/ but simply don't have the skills to do the prediction part of it very well.
We're not meteorologists, and we're not even fancy-pants computer science academic types. We just got sick of this app not existing...
So we hunkered down and learned about Computer Vision and experimented a whole lot (which resulted in another fun project: http://tinyfaceapp.com). There are so many awesome open-source data-processing libraries out there, which when combined with cloud-based computing, gives individuals so much tremendous power to pursue their weird passions.
I played around with OpenCV and the data a while ago, but in the end decided that for the time being it's probably not worth it. 1) I only have data for the Veneto, which is not that big 2) the data can be intermittent - if people start relying on you, and your data dries up for a few hours, they're going to be pissed off 3) storm prediction algorithms are much more complicated than just tracking the image. I looked up a bit of the literature on it, and... "it's complicated" if you want to really try for accuracy.
There's definitely a use case for this. It's awesome.
However, I wonder if it makes more sense to build the business around providing the data on the backend rather than creating a beautiful app.
If the point is this to let users know whether they're about to get rained on, all they need is a bit of text to get the message, rather than a beautiful flowy animation. It looks awesome - I just wonder if it's just going to get in the way after the 10th time checking it. A home screen widget would be ideal (which is why there's lots of weather app widgets already.)
I experimented with something like this in grad school to tell me when it was "safe" to walk home, by scraping the past hour's NOAA radar images, coursely gridding them as a poor man's way of reducing computational needs and noise, and throwing them into some regression algorithms to get a prediction of rain probability for [0,10] mins, [10,20] mins, etc. out to an hour.
It wasn't useless, but I gave up on it when it became clear that some significant computer-vision work was needed to get good accuracy, mainly object-tracking algorithms: correlating when a blob in one radar image is "the same" as a blob in a previous radar image, even though the blobs change shape/size semi-rapidly (and split and merge), in order to extract motion vectors. At least with the regression techniques I tried, it doesn't seem that they were able to implicitly extract the motion information from just the sequence of images as input data, because prediction was much better in a small set of images I manually labeled with motion vectors. Hopefully they have solutions for that!
In any case, I'd definitely use an app like this. I often predict short-term weather by pulling up one of those radar animations and extrapolating the motion of a storm in terms of "finger widths on monitor per 5 frames of animation" or something, which feels like a not-very-21st-century way of doing it.
Can you even get away with object tracking? It always seemed to me that there were a lot of latent phenomena you'd need to model in order to get the dynamics of cloud motion. There's wind producing a line linear drift, but also a lot of pressure and humidity interaction.
If you want a real physical simulation, there's tons of stuff yeah, but on a short term you can find considerable statistical regularities just by looking at how things move, while being agnostic about why they move that way. Over a 30-minute period, storm tracks tend to be semi-regular (though varies by region); if a storm's been moving ESE at 3 mph, and you have a bunch of historical data, you can come up with a decent predictive distribution over its likely heading/speed for the next 30 minutes. The uncertainty in the distribution is basically where all the un-modeled physical phenomena fall. If uncertainty is too high, then you can try to add in more complex models, or additional data (e.g. wind readings at various points, in addition to radar images).
It's always seemed to me that weather clouds don't seem to move as much as evolve directionally. That's where this intuition is coming from, though I suppose I'm not honestly surprised to find that gross motion is predictive as well.
A quick-and-dirty way I've seen a friend do it is to compress the images into h.264 and then extract the motion vectors. It's not the optimal algorithm but it does a pretty serviceable job and you're building off a large body of previous image tracking work.
MVTools is a dedicated Avisynth plugin designed to detect real motion, which is probably more suitable. Encoders only minimize residual, not find real motion vectors.
This is basically the problem I solved with my own Android live wallpaper. I drive a Jeep and I was constantly frustrated with getting caught in the rain.
So... I made a live wallpaper for my phone that would show the latest NOAA radar image for my area. I find that I can do a far better job of short term weather prediction with just a quick glance at my lock screen than I was getting by scanning short term forecasts.
I've set up something similar for myself, using WebLiveWallpaper to fetch a PNG of the latest local (Finnish) radar + temperature map.
It's rather surprising how easily having a bird's-eye view of the weather sort of fades into the background of your consciousness and becomes an accepted fact. I use my phone often enough that it just feels like I have an extra sense that tells me what the weather's like. Very odd.
How is Dark Sky able to predict weather on such short timescales? My impression has always been that weather prediction is fairly imprecise and so it only make sense in larger time scales (i.e. it will rain today, it will rain tonight).
We're going to write a blog post that goes into the technical details soon.
Basically, we rely on the fact that while weather is chaotic and turbulent in the long run (by "long run" I mean hours to days), it's fairly linear on the timescale of minutes.
So we're able to extract velocity data from NOAA radar images, and use that to project the storms into the future. Some storms are more coherent than others so we're constantly monitoring how accurate our predictions are and adjusting the projections accordingly.
Sometimes this technique breaks down, but it's surprisingly rare.
As far as I can tell, they are only trying to predict the type of rain showers which originate from thunderclouds, ie. Cumulonimbus clouds. These clouds are caused by a chain reaction starting in the lower atmosphere, where a huge mass of warm, moist air is sucked up to an altitude of 10000-60000 feet, depending on temperature, humidity and latitude.
The moisture (gaseous water) in the rising air condenses at altitude due to low temperatures, and hence frees additional heat which increases the upwards vertical motion of the airmass. This causes the cloud to grow. At some point, there is too much condensed water in the cloud and the water falls back down - a rain shower.
The development of Cumulonimbus clouds can be observed on weather radars. I'm guessing that they have an algorithm which, given the known conditions of temperature and humidity computes the time and size a Cb cloud has to be in order for a rain shower to start. The growth rate of a Cb cloud can be inferred given the vertical temperature distribution in the atmosphere.
It could also be that things are much simpler and that precipitation can be predicted from knowing the density of water in a cloud, which is what a weather radar measures. I am not a meteorologist.
I'm a complete layman here, but here's my guesses.
The first problem is that generally when talking about weather predictions you're talking about a large area. Asking, "will it rain in New York?" can be a loaded question if you're talking about a scattered storm. My understanding is that a lot of the time when you hear 60% chance of precipitation, the weatherman doesn't mean 40% chance it won't rain in the area, but that roughly 60% of the area or population will see rain. So the first advantage this app has is that it's forecasting for you individually in a very specific location, rather than for an entire TV or newspaper market.
Secondly, I think the short time frame is actually much easier to predict. On a short timescale you basically just look at the radar for the past 15 minutes, and see where a storm if any is heading. I do this all the time with weather.com. Open up the radar, see if a storm is nearby and where it's headed. It seems much harder to predict what a storm will do hours or days out, than what it'll do in 15 minutes.
In Slovakia we have been using similar feature for couple of years now. It is far away from being that pretty and interactive. It serves it's purpose for knowing how the weather will be evolving during the day: http://www.shmu.sk/sk/?page=1&id=meteo_num_mgram Charts from top to bottom: temperature (in Celsius), clouds (total, red-lower, green-middle, blue-upper), precipitation, atmospheric pressure, wind speed and last is wind direction.
Very useful for example when I want to go inline skating on the Danube dike in Bratislava: how much time do I have until it starts raining? :-)
This is really interesting-- Formula 1 has systems like this whenever they go to a track; whenever there's the chance of rain, you always see television shots of the local weather radar, combined with "Rain expected at 3:30 local time, intensity 1" overlaid on the map. Teams use this information to formulate their strategies-- do we bring the car in to switch to wet weather tires around 3:30, or do we wait to see what other teams are doing? It's pretty invaluable input to how races are run on days that are "mixed conditions".
This reminds me of that, only a much better display.
To be fair, even the expensive weather tracking systems they use get their fair share of the calls wrong and they employ watchers at different parts of the track (down the back straights, at the corners of the track, themselves at the pit straight, etc) to get a better idea. And that's over a relatively small area of space too.
It looks like Dark Sky takes this data and uses it to predict up to an hour ahead. It may not be revolutionary but at a fair price it could be a nice to have.
Nothing really too remarkable here though I do like the smooth radar map. Android already has a percent chance of rain graph built into it that though it wont say "rain will start in 5 minutes and last 15" you can see this info and predict it yourself. When I want more I love the graphs found here: http://forecast.weather.gov/MapClick.php?lat=35.82200&lo...
Anyway I guess I'm just spoiled with this type of info already built into android so guess its needed on the iphone which explains why its an iphone exclusive.
I wonder in all sincerity, how will it compete with looking out the window and making your own educated guess? Or looking at the NOAA radar hourly precipitation loop, as I do, and planning accordingly?
Because what appears to be missing is any sort of feedback mechanism by which to improve the accuracy of it's predictions on the same scale as which it is making its predictions.
In other words, if the prediction was rain in 8 minutes for 15 minutes, how can the program determine if it was accurate? (And suggests the question, is rain in 4 minutes for 18 minutes an acceptable level of inaccuracy?). Where does the data that it was 18 minutes of rain come from?
We constantly monitor how accurate our predictions are, and try to adjust accordingly. We'll have a blog post soon that explains more about how it works.
As for competing with looking at NOAA radar images, try a little experiment: Next time rain is headed your way, look at a radar animation and try to quantify when the rain will start. The fact that I'm completely unable to do that (even while looking at the damn radar animation) with any accuracy is the reason I started building this app.
Actualy the Wundermap on wunderground.com does this very well. It does a Google maps overlay so you can see your exact location. Also there are various options for highlighting the rain clouds and estimating the direction in which they are moving. Though I agree that using this product could be much easier when all you want to know the rain is coming and how long it might last.
Concerning your first question: This is largely a topological problem: If you're surrounded by mountains, the weather behaves distinctly different than on the open plain. In the second case, high wind speeds lead to rapid changes in conditions. For that scenario, such an app might give you some useful information.
I, myself, happen to work in a meteorological company and have an easy indicator: When our meteorologists all take off at the same time, rain is imminent.
It is very popular and it beats looking out of the window by a large margin. When it is raining you can see very accurately how long it's going to take for it to stop so you can decide if it's worth waiting. When it's not raining but it looks like it could be going to rain you can similarly make good predictions.
Note that they're not doing actual prediction, they're just showing radar images of the clouds of the last hour.
One other use case is if you have significantly different weather conditions a short distance from one another. I live near the coastal mountains; a 20 minute drive could take me from nice and sunny to cold and rainy.
Your points about feedback are valid, however.
Does anyone think it could also be useful to tie a twitter feed in with weather information? I feel like weather is one of those things that ppl complain about on twitter all the time; some simple geolocation and keyword matching and you could pull up recent geolocated tweets near you. Could be a nice qualitative weather report to accompany the actual data (people like a personal face on the weather).
Even the name of the app points to its competition. If the sky is dark I am most likely to bring an umbrella. It is however beneficial if I know I need to walk somewhere in a given amount of time. Seeing when the sky will be dark but no rain would be nice, but Ill still have my umbrella with me regardless.
There's no suggestion that the app will have a subscription cost and yet their data and processing costs are going to continue monthly. It's like a high-tech ponzi scheme. I just don't think it's sustainable so for that reason I couldn't back the project :(
Our current thinking is that we're going to charge for API access to our data. But I must admit, we don't have a well thought out business plan beyond "make the best f-ing weather app and charge a decent amount for it".
Sorry if I came across too negative. I've just got very militant about investing my time/energy/money in things which don't make financial sense.
I was also building an app based on weather data do I have some idea of the expenses involved and I quickly realised that my (admittedly niche) product couldn't be supported by small once-off payments like the app store.
Could optimize the data-processing such that it could be done on the device itself? You still need a weather service that you can get radar data from freely.
Scenario where it's useful: I walked to the grocery store and it's raining outside. Should I wait it out by staying inside or is it going to continue for more than 30mins?
The whole idea from Dark Sky came when I was stuck at a rest area off Interstate-90 in western New York. It started to downpour and I had no idea how long I'd have to wait if I wanted to stay dry.
Even looking at the radar map online didn't help. It was nearly impossible to extrapolate mentally the stuttery radar animation.
Here in The Netherlands we have buienradar(http://www.buienradar.nl), or drizzle radar. Works like a charm for short term rain predictions in our wet little country!
Do they really need $35,000 to do this? It seems like so many people produce apps first and get paid later, I'm not sure what makes their situation different.
Thinking about this more, its brilliant that they put their project on kickstarter. It didn't click for me that people are using kickstarter this way. They get some extra marketing and some extra $. Interesting also that they aren't afraid of someone taking their idea because they have a head start.
We've been working on the app for a while, but our day jobs always came first. We eventually realized that without a source of funding, it wasn't going to happen any time soon. And we're completely in love with the idea of Kickstarter, so we figured we'd start there and only approach more conventional sources of funding as plan B.
We figured out how much we needed to survive for 3 - 4 months, plus the cost servers for a certain amount of time, then factored in Kickstarter / Amazon's cut and the cost of the backer rewards.
We kept getting a number that was too high. So we eventually decided "Screw it! $35,000 it is!". It's still probably too high... but we wanted to avoid, at all costs, asking for too little money and then not being able to deliver to our backers.
If this was combined with GPS and push notifications, it could resonate with an awful lot of folks who would appreciate the heads-up. I can only imagine what something like this could do if with was plugged into something like Siri.
Talk about a personal assistant--it could remind you to bring an umbrella, or tell you to take an alternate route because precipitation 5 minutes from now might lead to delays on your regular commute, or change the way we learn about and deal with flight delays.
Of course, this is predicated on its ability to accurately predict the weather. I took a lot of meteorology in college and I don't think it's outside the realm of possibility, especially when coupled with historic data of past weather trends for specific locations.
It only offers the live radar map animation, based on your GPS location, but that's often pretty good for eyeballing when the rain is going to hit you, or when you are going to hit it if you are moving. Free, displays ads.
Not sure how useful is it as a separate app, but give me notifications about possible rain 30 minutes before a planned event, or whenever I'm more than X km from home, or just leaving and I'll be very very happy.
83 comments
[ 2.9 ms ] story [ 154 ms ] threadGot a spare fiver? Let them know you agree :)
Why do they need so much money? for servers?
So yeah, we need money for servers, and to pay for rent / food / etc for the several months it'll take us to make this thing.
When you start doing the math, a lot of the money we're asking for disappears. 10% of what we raise goes to Kickstarter and Amazon (for payment processing). Another 30% - $35% goes to paying for the Kickstarter rewards, and then the government takes it's cut. What's left is a minority of the money.
We'll be writing more about how it works for our fellow nerds soon.
Good luck, please come back and post here when you are done. You already have one future customer here :)
We're not meteorologists, and we're not even fancy-pants computer science academic types. We just got sick of this app not existing...
So we hunkered down and learned about Computer Vision and experimented a whole lot (which resulted in another fun project: http://tinyfaceapp.com). There are so many awesome open-source data-processing libraries out there, which when combined with cloud-based computing, gives individuals so much tremendous power to pursue their weird passions.
However, I wonder if it makes more sense to build the business around providing the data on the backend rather than creating a beautiful app.
If the point is this to let users know whether they're about to get rained on, all they need is a bit of text to get the message, rather than a beautiful flowy animation. It looks awesome - I just wonder if it's just going to get in the way after the 10th time checking it. A home screen widget would be ideal (which is why there's lots of weather app widgets already.)
I experimented with something like this in grad school to tell me when it was "safe" to walk home, by scraping the past hour's NOAA radar images, coursely gridding them as a poor man's way of reducing computational needs and noise, and throwing them into some regression algorithms to get a prediction of rain probability for [0,10] mins, [10,20] mins, etc. out to an hour.
It wasn't useless, but I gave up on it when it became clear that some significant computer-vision work was needed to get good accuracy, mainly object-tracking algorithms: correlating when a blob in one radar image is "the same" as a blob in a previous radar image, even though the blobs change shape/size semi-rapidly (and split and merge), in order to extract motion vectors. At least with the regression techniques I tried, it doesn't seem that they were able to implicitly extract the motion information from just the sequence of images as input data, because prediction was much better in a small set of images I manually labeled with motion vectors. Hopefully they have solutions for that!
In any case, I'd definitely use an app like this. I often predict short-term weather by pulling up one of those radar animations and extrapolating the motion of a storm in terms of "finger widths on monitor per 5 frames of animation" or something, which feels like a not-very-21st-century way of doing it.
So... I made a live wallpaper for my phone that would show the latest NOAA radar image for my area. I find that I can do a far better job of short term weather prediction with just a quick glance at my lock screen than I was getting by scanning short term forecasts.
https://market.android.com/details?id=com.appidio.radarlivel...
(That's the free version. The paid version animates the last 10 frames.)
It's rather surprising how easily having a bird's-eye view of the weather sort of fades into the background of your consciousness and becomes an accepted fact. I use my phone often enough that it just feels like I have an extra sense that tells me what the weather's like. Very odd.
This is a nice little mobile app that is location aware.
Basically, we rely on the fact that while weather is chaotic and turbulent in the long run (by "long run" I mean hours to days), it's fairly linear on the timescale of minutes.
So we're able to extract velocity data from NOAA radar images, and use that to project the storms into the future. Some storms are more coherent than others so we're constantly monitoring how accurate our predictions are and adjusting the projections accordingly.
Sometimes this technique breaks down, but it's surprisingly rare.
1. http://en.wikipedia.org/wiki/Chaos_theory
2. http://en.wikipedia.org/wiki/Butterfly_effect
The moisture (gaseous water) in the rising air condenses at altitude due to low temperatures, and hence frees additional heat which increases the upwards vertical motion of the airmass. This causes the cloud to grow. At some point, there is too much condensed water in the cloud and the water falls back down - a rain shower.
The development of Cumulonimbus clouds can be observed on weather radars. I'm guessing that they have an algorithm which, given the known conditions of temperature and humidity computes the time and size a Cb cloud has to be in order for a rain shower to start. The growth rate of a Cb cloud can be inferred given the vertical temperature distribution in the atmosphere.
It could also be that things are much simpler and that precipitation can be predicted from knowing the density of water in a cloud, which is what a weather radar measures. I am not a meteorologist.
The first problem is that generally when talking about weather predictions you're talking about a large area. Asking, "will it rain in New York?" can be a loaded question if you're talking about a scattered storm. My understanding is that a lot of the time when you hear 60% chance of precipitation, the weatherman doesn't mean 40% chance it won't rain in the area, but that roughly 60% of the area or population will see rain. So the first advantage this app has is that it's forecasting for you individually in a very specific location, rather than for an entire TV or newspaper market.
Secondly, I think the short time frame is actually much easier to predict. On a short timescale you basically just look at the radar for the past 15 minutes, and see where a storm if any is heading. I do this all the time with weather.com. Open up the radar, see if a storm is nearby and where it's headed. It seems much harder to predict what a storm will do hours or days out, than what it'll do in 15 minutes.
Very useful for example when I want to go inline skating on the Danube dike in Bratislava: how much time do I have until it starts raining? :-)
This reminds me of that, only a much better display.
http://news.bbc.co.uk/weather/forecast/8?area=London
Anyway I guess I'm just spoiled with this type of info already built into android so guess its needed on the iphone which explains why its an iphone exclusive.
No, that's because your average developer is not using the same platform as your average smartphone user.
Because what appears to be missing is any sort of feedback mechanism by which to improve the accuracy of it's predictions on the same scale as which it is making its predictions.
In other words, if the prediction was rain in 8 minutes for 15 minutes, how can the program determine if it was accurate? (And suggests the question, is rain in 4 minutes for 18 minutes an acceptable level of inaccuracy?). Where does the data that it was 18 minutes of rain come from?
As for competing with looking at NOAA radar images, try a little experiment: Next time rain is headed your way, look at a radar animation and try to quantify when the rain will start. The fact that I'm completely unable to do that (even while looking at the damn radar animation) with any accuracy is the reason I started building this app.
I, myself, happen to work in a meteorological company and have an easy indicator: When our meteorologists all take off at the same time, rain is imminent.
Topological or topographical? It's kinda both in your example.
I'm no expert, but I don't think topology features much in the weather models. It's mostly numerical analysis and statistics.
It is very popular and it beats looking out of the window by a large margin. When it is raining you can see very accurately how long it's going to take for it to stop so you can decide if it's worth waiting. When it's not raining but it looks like it could be going to rain you can similarly make good predictions.
Note that they're not doing actual prediction, they're just showing radar images of the clouds of the last hour.
Your points about feedback are valid, however.
Does anyone think it could also be useful to tie a twitter feed in with weather information? I feel like weather is one of those things that ppl complain about on twitter all the time; some simple geolocation and keyword matching and you could pull up recent geolocated tweets near you. Could be a nice qualitative weather report to accompany the actual data (people like a personal face on the weather).
I was also building an app based on weather data do I have some idea of the expenses involved and I quickly realised that my (admittedly niche) product couldn't be supported by small once-off payments like the app store.
Scenario where it's useful: I walked to the grocery store and it's raining outside. Should I wait it out by staying inside or is it going to continue for more than 30mins?
Even looking at the radar map online didn't help. It was nearly impossible to extrapolate mentally the stuttery radar animation.
Thinking about this more, its brilliant that they put their project on kickstarter. It didn't click for me that people are using kickstarter this way. They get some extra marketing and some extra $. Interesting also that they aren't afraid of someone taking their idea because they have a head start.
Good moves.
We figured out how much we needed to survive for 3 - 4 months, plus the cost servers for a certain amount of time, then factored in Kickstarter / Amazon's cut and the cost of the backer rewards.
We kept getting a number that was too high. So we eventually decided "Screw it! $35,000 it is!". It's still probably too high... but we wanted to avoid, at all costs, asking for too little money and then not being able to deliver to our backers.
Talk about a personal assistant--it could remind you to bring an umbrella, or tell you to take an alternate route because precipitation 5 minutes from now might lead to delays on your regular commute, or change the way we learn about and deal with flight delays.
Of course, this is predicated on its ability to accurately predict the weather. I took a lot of meteorology in college and I don't think it's outside the realm of possibility, especially when coupled with historic data of past weather trends for specific locations.
Only one way to find out.
It only offers the live radar map animation, based on your GPS location, but that's often pretty good for eyeballing when the rain is going to hit you, or when you are going to hit it if you are moving. Free, displays ads.
http://www.accuweather.com/radarplus-benefits.asp