Thoughts? We're thinking of individual-level prediction tools, but there are other ways of using this approach. e.g. at the community / city scale with mobile operator data.
I don't know if you'd get the same quality of data, but it seems like you could get the same kind of data from a user's Foursquare (location) and Facebook page.
Now that I think about it, I imagine most young people use Facebook messages/wall posts more than phone calls and texts.
Or perhaps a browser plugin that anonymously reports browsing habits.
You might see if this could get funded as a means to help seriously at risk populations reduce their risk, spot problems at an earlier stage and so on.
I have a form of cystic fibrosis, as does my 23 year old son. We have gotten a great deal healthier than doctors claim is possible largely by making dietary and lifestyle changes to protect us from illness, rather than waiting until we are ill and loading up on drugs. I'm extremely controversial in the CF community. But if you had funding from some "kosher" organization (unlike me -- I'm just some loudmouthed individual speaking from firsthand experience...aka "fruitcake"), well, there is real potential to do real good here, not just for the individual quality of life but also the economy: By my estimation, the 30,000 or so people with CF in the US consume roughly $3billion+ annually (edit: I mean in medical care -- doctors visits, drugs, hospitalizations), most of which does not come out of their pockets. Most of it comes from insurance, state aid, federal aid, charities, and even medical write-offs when patients who are routinely deathly ill simply can't pay. I don't know how to tap into that $3 billion, but it is in someone's best interest monetarily (if nothing else) to help this population (and others like it) stay healthier. And I know it is an achievable goal because I do it every day. I just don't know how to adequately explain to other people how to do what I do.
I hope this goes somewhere. I am excited to see something like this. It fits nicely with my views on what works, health-wise.
Thanks for the kind words, and good luck to your family. We really do want to make this into a tool that can help people, if possible.
My intuition is that being able to quantify and visualize 'quality-of-life' automatically, will help support diet and lifestyle changes.
What kind of features/UI would you like to see in such a tool? Would it be better is this was generic (e.g. like Facebook), or something specifically for patients with CF?
What kind of features/UI would you like to see in such a tool? Would it be better is this was generic (e.g. like Facebook), or something specifically for patients with CF?
The CF market per se is pretty small. I wouldn't look to customize it specifically for them. I also think that "good practices" generalize. If you can come up with some parameters that work well with vulnerable populations, it should work regardless of their specific issue.
Features that might be nice: A text message "alert" when early indicators are noticed and perhaps a means for the customer to put in custom "prompts" so that, whatever their medical condition, they can self-remind (for example: that they might have forgotten to take a particular medication). People with medical conditions often become "forgetful" as their condition starts heading into a downward spiral and the sicker they get, the harder it gets to think about what they most need to do to stop the downward spiral. A reminder to do the right things and an alert that their patterns suggest they might be in need of extra care could well be the stitch in time that saves nine.
Once we mockup our UI, would it be possible to have you give feedback / provide suggestions? Could you help us put together a user-trial with say 10-15 tech-savvy parents like you?
You can contact me via the email address on any of the websites in my profile. I belong to some health lists. I would want to ask the list-owners permission first, but I might be able to help you put together a trial. We can discuss more details via email.
My personal opinion: if you can REALLY do what you claim then you should get every country's equivalent of the CDC working with you to MANDATE that this be part of every phone, and then of course you get the revenue from IP licensing.
In fact, I would broaden the scope a bit: make it like the "Emergency Broadcast System" for phones, only two-way.
Lots of privacy implications and the carriers and handset manufacturers would hate it, but I can certainly see it doing long term good.
>When Madan trained software to hunt for this signature in the cellphone data, a daily check correctly identified flu victims 90 per cent of the time.
I'm very surprised by this - it strikes me that I'd expect it to be hard to tell from such data that it wasn't a stomach upset, tiredness, onset of disease or simply a heavy cold. Indeed 90% sounds like it might be higher than correct identification via an interview with a doctor.
>[...] in Rwanda between 2006 and 2009. He saw a clear reduction in people's movement, which may have been due to the disease. But the outbreak was caused by floods, which also limited mobility. Distinguishing between the two possible causes on the basis of phone data alone was impossible, he says.
Makes me the 90% figure highly suspect.
That aside I wonder if it wouldn't be easier to have a smart phone app that gives you epidemiology info in return for entering your health status for the day. To encourage continued use a sort of diary function and stats could be added. Wouldn't this be more predictive than monitoring movement, and less intrusive?
With a range that large, I would be highly suspect that when you did get 90%, it was just fitting the data. And how would you even confirm if they had the flu - even a doctor could misdiagnose mild flu, which adds more noise.
60% really means very little if it's a binary choice - flu or no. You'll be right 50% of the times no mater what.
With severely unbalanced classes, overall accuracy is not a good metric. So we use recall of the symptom class, and how it varies with the precision. Recall = 0.6 to 0.9. All results are based on cross-validation.
There is a lot more sophistication we can add (e.g. markov properties), these are almost first-order results.
Nonetheless important to validate with other populations. That would be a big part of testing any product.
Trends are spotted in the first 48-hour window, in our sample. There is the big open question of validation outside the sample population of course.
By precise, do you mean precision versus recall? The tradeoff depends on what you want to do with the inferences. If you send the analysis to say a nurse or family member (i.e., human in the loop, which we think is the right approach) then recall should be optimized.
I've just been away in the Midlands (UK) and used Google Maps on a mobile for the first time. The triangulation of position was only accurate to about 1500m radius and then would switch between positions quite often. Such info would look much like a user that was moving quite a lot - I expect it could be filtered out but this sort precision (and lack of accuracy) appears common in the UK.
I think a very strong possibility lies with the very experiment described in the article. Flu is a huge issue at college campuses, high schools, etc. Every level of education involves cramming a bunch of individuals in to small spaces with high amounts of contact (more so than in a prison, for example). In higher education, sexual activity and drinking (i.e. sharing cups) further increases risk of infection. My campus of 5,000 undergrads turned into a textbook example of mass hysteria during last year's swine flu scare. I can only imagine how many hours of education must have been lost through the whole ordeal.
So build an enterprise level system and sell it to individual universities. The school installs your application on each student's phone and receive an aggregate view of anonymous data through a back-end interface. This could give them a real-time look of how the infection is spreading through their community and allow them to be far more responsive in prevention measures. What if they could receive warning that the 3rd floor of Residence Hall A had signs of a potential outbreak? Just bring in those students for a preemptive checkup and stop the bug dead in its tracks.
Of course there is a boat load of legal and privacy issues to sort out for this type of intrusive monitoring. Nothing that couldn't be overcome I don't think (IANAL). I know that my school installs an app on all of our athlete's Facebook accounts for monitoring purposes so this isn't too far off from that.
Good luck with whatever path you choose! I'm always open to discussing startup strategy so feel free to drop me a line.
just make an app that warns you if any of you friends cell or computer use changes. I'd pay to see which of my friends might be getting sick. Luckily, each new health scare will drive up demand.
At the direct-to-consumers level, think "I've fallen and I can't get up!" Automatic detection of illness and alerts for the old and infirm could be a potential market, though research on college students is not going to be at all effective for monitoring old people. And, if it can only detect trends rather than a specific individual situation, then it won't work in this context.
As others mentioned, making it a standard part of all phones might be something you could sell to governments, particularly in areas where flu frequently becomes epidemic.
It's a public health issue, which isn't usually an extremely easy to go-to-market area, I would think. Since it doesn't cure people of the flu, and doesn't help prevent it in any particular individual, it has nearly zero market value for any individual. I wouldn't even have a reason to install it for free...what good would it do me?
Is there some element of your research and tech that can be generalized to spot other kinds of trends? Trendspotting is extremely valuable, particularly to marketers...Google would probably acquire a company that can effectively trendspot from mobile devices, and even if they didn't, a mobile ad platform with trendspotting would be incredibly lucrative.
The challenge with 'think I've fallen' is that you constantly have to ping the accelerometer. From a battery-life perspective, that kills the smartphone. Also, its hard to distinguish those pings from
For that particular application, its better to use custom body-worn hardware (e.g. fitbit).
Regarding spotting trends in face-to-face networks, we've done similar analysis political opinions during the 2008 election campaign.
very briefly. We're comparing the time + effort spent crafting an application (which apparently is quite large, for first-timers) and not fitting the ask VS simply building it and approaching consumers / partners.
If you have particular insight on any SBIR proposals related to our work, do share. Thanks!
Anmol - you're building an information business. And at scale, IMHO, they're fabulous. Health information seems like a massive market, and this seems like unique data. (Think about financial consumers who care about health data in addition to the pharmas.) I'll assume your app scales so that you get critical mass of input, but once you have that in your sights, here's how we constructed our information business.
In my experience, the best information businesses are founded on a core of primary research - data that you find/generate that no one else has. Once you build that database, you build the products. You might be surprised, but here's what many information products look like - I'm a fan of doing it in this order:
1. A big honking CSV file of the primary data. For customers that have their own models, which your early adopters will likely have. Ours did.
2. A CSV file of derivative analytics (the output of your prediction tools perhaps?). For customers who want your algorithmic take on the underlying data.
3. Reports (manually written). People buy your analysis of the data. We found a majority of the potential market doesn't want Tools or data files (even if they're Wall Street "quants"). They just want answers - and by virtue of having the data, you're an instant expert. Reports have the nice benefit of being A) tangible B) easy to consume and C) subscribe-able (repeat revenue!) They have a bonus of being delicious chum for the media sharks.
4. Reports (automated, for scale). Subscribers get the latest insights each week, month, etc. Best not to automate until you see which of #3 are most popular.
5. Now come the technology Tools. #3 and #4 above will be excellent test beds for your prediction tools. Doing it in this order, you won't have to build any front end applications until you understand what predictions people pay for. One way to build a tools-based product line is to give the tech away for free and have the customer subscribe to the latest data.
You might also find that a great consumer-facing website is powerful for the information business. We don't have those skills in our company, but have found that when we pull it off even a little, institutional buyers begin as personal, individual consumers. A great consumer facing site/app is a perfect way to reach them.
Seems like you have something that public health officials would want, and maybe mobile carriers, if the privacy issues can be solved and having it installed wins them some points with their regulators.
If you want to make something businesses want to pay for, see if the same predictive models work for forecasting product demand and other economic activity. What is the effect of a movie release on patterns of communication and movement? Can profiles be (semi-anonymously) correlated with other tweets/purchases, so you know that learning about a certain product, or buying it, changes behavior in some way?
30 comments
[ 3.1 ms ] story [ 84.3 ms ] threadNow that I think about it, I imagine most young people use Facebook messages/wall posts more than phone calls and texts.
Or perhaps a browser plugin that anonymously reports browsing habits.
You might see if this could get funded as a means to help seriously at risk populations reduce their risk, spot problems at an earlier stage and so on.
I have a form of cystic fibrosis, as does my 23 year old son. We have gotten a great deal healthier than doctors claim is possible largely by making dietary and lifestyle changes to protect us from illness, rather than waiting until we are ill and loading up on drugs. I'm extremely controversial in the CF community. But if you had funding from some "kosher" organization (unlike me -- I'm just some loudmouthed individual speaking from firsthand experience...aka "fruitcake"), well, there is real potential to do real good here, not just for the individual quality of life but also the economy: By my estimation, the 30,000 or so people with CF in the US consume roughly $3billion+ annually (edit: I mean in medical care -- doctors visits, drugs, hospitalizations), most of which does not come out of their pockets. Most of it comes from insurance, state aid, federal aid, charities, and even medical write-offs when patients who are routinely deathly ill simply can't pay. I don't know how to tap into that $3 billion, but it is in someone's best interest monetarily (if nothing else) to help this population (and others like it) stay healthier. And I know it is an achievable goal because I do it every day. I just don't know how to adequately explain to other people how to do what I do.
I hope this goes somewhere. I am excited to see something like this. It fits nicely with my views on what works, health-wise.
Best of luck.
My intuition is that being able to quantify and visualize 'quality-of-life' automatically, will help support diet and lifestyle changes.
What kind of features/UI would you like to see in such a tool? Would it be better is this was generic (e.g. like Facebook), or something specifically for patients with CF?
The CF market per se is pretty small. I wouldn't look to customize it specifically for them. I also think that "good practices" generalize. If you can come up with some parameters that work well with vulnerable populations, it should work regardless of their specific issue.
Features that might be nice: A text message "alert" when early indicators are noticed and perhaps a means for the customer to put in custom "prompts" so that, whatever their medical condition, they can self-remind (for example: that they might have forgotten to take a particular medication). People with medical conditions often become "forgetful" as their condition starts heading into a downward spiral and the sicker they get, the harder it gets to think about what they most need to do to stop the downward spiral. A reminder to do the right things and an alert that their patterns suggest they might be in need of extra care could well be the stitch in time that saves nine.
In fact, I would broaden the scope a bit: make it like the "Emergency Broadcast System" for phones, only two-way.
Lots of privacy implications and the carriers and handset manufacturers would hate it, but I can certainly see it doing long term good.
I'm very surprised by this - it strikes me that I'd expect it to be hard to tell from such data that it wasn't a stomach upset, tiredness, onset of disease or simply a heavy cold. Indeed 90% sounds like it might be higher than correct identification via an interview with a doctor.
>[...] in Rwanda between 2006 and 2009. He saw a clear reduction in people's movement, which may have been due to the disease. But the outbreak was caused by floods, which also limited mobility. Distinguishing between the two possible causes on the basis of phone data alone was impossible, he says.
Makes me the 90% figure highly suspect.
That aside I wonder if it wouldn't be easier to have a smart phone app that gives you epidemiology info in return for entering your health status for the day. To encourage continued use a sort of diary function and stats could be added. Wouldn't this be more predictive than monitoring movement, and less intrusive?
The actual accuracy is between 60-90%, depending on clusters of symptoms, and apparently that was stripped somewhere during editing. Oh well.
60% really means very little if it's a binary choice - flu or no. You'll be right 50% of the times no mater what.
What? Just because something is binary doesn't mean it happens half the time.
There is a lot more sophistication we can add (e.g. markov properties), these are almost first-order results.
Nonetheless important to validate with other populations. That would be a big part of testing any product.
Questions
1. How fast can you spot a trend? 2. How is the tracking working? 3. How precise is it.
I could imagine traffic would be a good area to look into.
By precise, do you mean precision versus recall? The tradeoff depends on what you want to do with the inferences. If you send the analysis to say a nurse or family member (i.e., human in the loop, which we think is the right approach) then recall should be optimized.
I've just been away in the Midlands (UK) and used Google Maps on a mobile for the first time. The triangulation of position was only accurate to about 1500m radius and then would switch between positions quite often. Such info would look much like a user that was moving quite a lot - I expect it could be filtered out but this sort precision (and lack of accuracy) appears common in the UK.
So build an enterprise level system and sell it to individual universities. The school installs your application on each student's phone and receive an aggregate view of anonymous data through a back-end interface. This could give them a real-time look of how the infection is spreading through their community and allow them to be far more responsive in prevention measures. What if they could receive warning that the 3rd floor of Residence Hall A had signs of a potential outbreak? Just bring in those students for a preemptive checkup and stop the bug dead in its tracks.
Of course there is a boat load of legal and privacy issues to sort out for this type of intrusive monitoring. Nothing that couldn't be overcome I don't think (IANAL). I know that my school installs an app on all of our athlete's Facebook accounts for monitoring purposes so this isn't too far off from that.
Good luck with whatever path you choose! I'm always open to discussing startup strategy so feel free to drop me a line.
As others mentioned, making it a standard part of all phones might be something you could sell to governments, particularly in areas where flu frequently becomes epidemic.
It's a public health issue, which isn't usually an extremely easy to go-to-market area, I would think. Since it doesn't cure people of the flu, and doesn't help prevent it in any particular individual, it has nearly zero market value for any individual. I wouldn't even have a reason to install it for free...what good would it do me?
Is there some element of your research and tech that can be generalized to spot other kinds of trends? Trendspotting is extremely valuable, particularly to marketers...Google would probably acquire a company that can effectively trendspot from mobile devices, and even if they didn't, a mobile ad platform with trendspotting would be incredibly lucrative.
For that particular application, its better to use custom body-worn hardware (e.g. fitbit).
Regarding spotting trends in face-to-face networks, we've done similar analysis political opinions during the 2008 election campaign.
Results here: http://web.media.mit.edu/~anmol/political-2.pdf
BTW, small world, my SO knows Eagle :P
If you have particular insight on any SBIR proposals related to our work, do share. Thanks!
ps. yes, Nathan is a friend.
In my experience, the best information businesses are founded on a core of primary research - data that you find/generate that no one else has. Once you build that database, you build the products. You might be surprised, but here's what many information products look like - I'm a fan of doing it in this order:
1. A big honking CSV file of the primary data. For customers that have their own models, which your early adopters will likely have. Ours did.
2. A CSV file of derivative analytics (the output of your prediction tools perhaps?). For customers who want your algorithmic take on the underlying data.
3. Reports (manually written). People buy your analysis of the data. We found a majority of the potential market doesn't want Tools or data files (even if they're Wall Street "quants"). They just want answers - and by virtue of having the data, you're an instant expert. Reports have the nice benefit of being A) tangible B) easy to consume and C) subscribe-able (repeat revenue!) They have a bonus of being delicious chum for the media sharks.
4. Reports (automated, for scale). Subscribers get the latest insights each week, month, etc. Best not to automate until you see which of #3 are most popular.
5. Now come the technology Tools. #3 and #4 above will be excellent test beds for your prediction tools. Doing it in this order, you won't have to build any front end applications until you understand what predictions people pay for. One way to build a tools-based product line is to give the tech away for free and have the customer subscribe to the latest data.
You might also find that a great consumer-facing website is powerful for the information business. We don't have those skills in our company, but have found that when we pull it off even a little, institutional buyers begin as personal, individual consumers. A great consumer facing site/app is a perfect way to reach them.
rock on.
If you want to make something businesses want to pay for, see if the same predictive models work for forecasting product demand and other economic activity. What is the effect of a movie release on patterns of communication and movement? Can profiles be (semi-anonymously) correlated with other tweets/purchases, so you know that learning about a certain product, or buying it, changes behavior in some way?