> LifeLog was.. to be "an ontology-based (sub)system that captures, stores, and makes accessible the flow of one person's experience in and interactions with the world in order to support a broad spectrum of associates/assistants and other system capabilities". The objective of the LifeLog concept was "to be able to trace the 'threads' of an individual's life in terms of events, states, and relationships", and it has the ability to "take in all of a subject's experience, from phone numbers dialed and e-mail messages viewed to every breath taken, step made and place gone".
> The SenseCam is a personal, wearable camera developed by Microsoft Research in Cambridge, UK, and used as a lifelogging device in projects like MyLifeBits.. is based on wearing the SenseCam for lifelogging of ‘events’ during your day, and generating a fast-forward movie of the event as the memory recall interface.
> Lifelogging technologies can capture both mundane and important experiences in our daily lives, resulting in a rich record of the places we visit and the things we see.. Previous work has demonstrated that Lifelogs can aid recall, but that they do many other things too. They can help us look back at the past in new ways, or to reconstruct what we did in our lives, even if we don’t recall exact details. Here we extend the notion of Lifelogging to include locational information.
> MylifeBits is a lifetime store of everything. It is the fulfillment of Vannevar Bush’s 1945 Memex vision including full-text search, text & audio annotations, and hyperlinks. The book Total Recall.. is the culmination of our thoughts regarding MyLifebits and the larger CARPE research agenda .. Gordon Bell has captured a lifetime’s worth of articles, books, cards, CDs, letters, memos, papers, photos, pictures, presentations, home movies, videotaped lectures, and voice recordings and stored them digitally. He is now paperless, and is beginning to capture phone calls, IM transcripts, television, and radio.
I think Recall is a window into the future of computers—but this version isn’t going to be successful. It may not even be possible to succeed with this model at all.
I think a lot of people would consider it a _feature_ if they could simply ask their computer “what was my banking password?” and not understand at all the difference between storing credentials in Recall vs storing them in LastPass.
For me the killer compromise is that 3 months isn’t _nearly_ long enough. This wouldn’t be useful to me until the history extends to more than a decade. I don’t always need that, but for anything less than that I very rarely need the help—it’s at most a convenience, not a new capability.
Perhaps a day will come when tools like Siri will ask for a confirmation whether you meant "fifteen" or "fifty" instead of making a guess when they aren't fully certain that they heard you correctly.
That’s really where I think the future of these features lives—in figuring out the trust model and security story for providing it. (Maybe it’s even impossible…)
The only way I can imagine being comfortable with it is if I'm using it on a machine that is permanently air-gapped from the internet.
That complete of a record is extremely sensitive and high-value. The potential damage from having it misused is far too high to take any actual chance with.
Likely, this will be illegal, especially if users aren't aware their screens are being recorded. Furthermore that screenshots of activity are stored in the Azure cloud.
Developers building it are probably thinking about Security Considerations, but are getting pushback from product managers. So data leak is imminent.
Not to mention that, even taking them at their word on that for the sake of argument, storing the data locally is far from sufficient. Can we be sure that the data is only ever processed locally and that the results of that processing stay local?
>> No the announcement was clear that the storage is local.
But I just bought a new computer and need to recall stuff from the old one.
Or, I am using my laptop / tablet computer and need to recall stuff done on my desktop comptuer. (Or similarly, I am using my smartphone and need to recall stuff done on my laptop.)
All they have to do is convince the user of the advantages of sending their recall data to the cloud and the user will agree to do so.
I was under the impression that Recall was designed to be local-only - all processing done locally, storage local too (the article quotes 25GB of local storage for three months of screenshots).
Microsoft isn't really specific about what is local and what isn't. Sure, they are very specific that the storage of the screenshots is local. What about all the processing? As far as I see it they're far less specific about that.
And who says this will stay local and isn't a small T&C update away from being shared with 'trusted partners'?
The article and the source video note these screenshots are stored locally: "Satya point out the processing and data storage is done locally on the device"
I believe the questionable legality also extends to any kind of video calls taking place on systems with this turned on.
If Recall indeed "does not perform content moderation", then having it on would include having screenshots of all video calls.
Especially in many-party consent jurisdictions recording video calls by taking screenshots at a sufficiently high frequency without the explicit consent, or at least option to opt out, of remote parties is probably illegal.
You have to turn on "sync across devices" and have a MS account. One of the many excellent reasons to avoid one. Just in case MS makes a "mistake" and turns this one on by default as well.
I can be wrong on this, but if the screenshots are being uploaded, they can be used to train AI models on how to effectively use the computer and navigate. Copilot will then become better at doing things on autopilot and Microsoft will profit.
But it got thinking, if git ever gets replaced would it be by something akin to this that is constantly monitoring all keystrokes in your repo folder and you never make a commit as such, the ai repo upstream just sorts it out..
This OS will, by design, waste 10% of rather limited SSD space, on a feature that I don't need, that may or may not work, and that also introduces a security risk on top of slowing things down.
It's going to be very useful™, as infinite browser history and bookmarks are to some people, I'm sure.
As for me, this sounds like a wonderful way to invent entirely new types of digital clutter.
Hoping there'll be a no-nonsense way to disable this "feature" in the "Pro" OS versions at least, along with Cortana/Copilot.
I’m highly skeptical of the conclusions here. Many corporations already record periodic screenshots of employee devices. There’s also a bunch of hand waving of “security incident here, therefore hackers will access your machine”.
The crux of the argument boils down to
> If you have malware running on your PC for only minutes, you have a big problem in your life now rather than just changing some passwords.
Ok. How often does malware run for just a few minutes? Honestly if there’s malware running on any of my machines, it’s not clear the outcome is any different here. Sure there’s a hypothetical scenario where it’s worse. But it’s also just as likely to grab a copy of my password manager while it has everything unencrypted. And it’s also way more likely to persist for days or months, making access to historical data moot as it’ll just install its own key logger.
> I’m highly skeptical of the conclusions here. Many corporations already record periodic screenshots of employee devices. There’s also a bunch of hand waving of “security incident here, therefore hackers will access your machine”.
In highly specific situations, like customer care, yes. Also these machines tend to be super locked down, no admin rights for the users.
And yes this practice is risky. I managed some PCs for customer care and one of them was taken in for repair by the service desk and replaced, and redeployed without wiping it. It ended up in payroll and when we found out their manager was furious (quite understandably so).
Yes malware can install its own keylogger. But it's an extra step once again.
To add to this - did you notice the terribly ineffective demo scenario?
In the video, the WSJ reporter searches for the term "Brown leather bag" and ends up finding what appears to be an AI generated / stock image of a brown leather bag inside a PDF that has zero relevant textual content about brown leather bags.
Of all the things a user might be looking for with that search term, it's not going to be this!
52 comments
[ 3.5 ms ] story [ 42.8 ms ] thread2003, https://en.wikipedia.org/wiki/DARPA_LifeLog
> LifeLog was.. to be "an ontology-based (sub)system that captures, stores, and makes accessible the flow of one person's experience in and interactions with the world in order to support a broad spectrum of associates/assistants and other system capabilities". The objective of the LifeLog concept was "to be able to trace the 'threads' of an individual's life in terms of events, states, and relationships", and it has the ability to "take in all of a subject's experience, from phone numbers dialed and e-mail messages viewed to every breath taken, step made and place gone".
2007 Microsoft Research, https://www.microsoft.com/en-us/research/video/the-microsoft...
> The SenseCam is a personal, wearable camera developed by Microsoft Research in Cambridge, UK, and used as a lifelogging device in projects like MyLifeBits.. is based on wearing the SenseCam for lifelogging of ‘events’ during your day, and generating a fast-forward movie of the event as the memory recall interface.
2010 Microsoft Research, https://www.microsoft.com/en-us/research/publication/now-let...
> Lifelogging technologies can capture both mundane and important experiences in our daily lives, resulting in a rich record of the places we visit and the things we see.. Previous work has demonstrated that Lifelogs can aid recall, but that they do many other things too. They can help us look back at the past in new ways, or to reconstruct what we did in our lives, even if we don’t recall exact details. Here we extend the notion of Lifelogging to include locational information.
https://www.microsoft.com/en-us/research/project/mylifebits/
> MylifeBits is a lifetime store of everything. It is the fulfillment of Vannevar Bush’s 1945 Memex vision including full-text search, text & audio annotations, and hyperlinks. The book Total Recall.. is the culmination of our thoughts regarding MyLifebits and the larger CARPE research agenda .. Gordon Bell has captured a lifetime’s worth of articles, books, cards, CDs, letters, memos, papers, photos, pictures, presentations, home movies, videotaped lectures, and voice recordings and stored them digitally. He is now paperless, and is beginning to capture phone calls, IM transcripts, television, and radio.
HN ranking history for this thread: https://hnrankings.info/40433884/
I think a lot of people would consider it a _feature_ if they could simply ask their computer “what was my banking password?” and not understand at all the difference between storing credentials in Recall vs storing them in LastPass.
For me the killer compromise is that 3 months isn’t _nearly_ long enough. This wouldn’t be useful to me until the history extends to more than a decade. I don’t always need that, but for anything less than that I very rarely need the help—it’s at most a convenience, not a new capability.
And the computer would reply "I'm sorry, I have no record of Baking Pastor", based on how well these AI assistants understand me.
At least they will learn a lot from this, in 6-12 months and perhaps a new generation of NPUs we might have something attractive.
But I'm not confident enough with any system or OS for that -- and I have zero confidence in the likes of Microsoft, Google, OpenAI, Facebook, etc.
That complete of a record is extremely sensitive and high-value. The potential damage from having it misused is far too high to take any actual chance with.
If it were entirely local and everything else on my computer was entirely local unless I was actively trying to connect to something external, fine.
But instead everything is being sent to the void and monetized as much as possible while telling me the absolute minimum about it.
Developers building it are probably thinking about Security Considerations, but are getting pushback from product managers. So data leak is imminent.
But Azure cloud? How can this be legal?
It's probably not illegal...
And if it is, they're probably not going to get caught...
And if they do, the government probably won't go after them...
And if it does, they're probably not going to be found guilty...
And if they are, they probably won't face any consequences...
And if they do, nobody's going to go to jail.
But I just bought a new computer and need to recall stuff from the old one.
Or, I am using my laptop / tablet computer and need to recall stuff done on my desktop comptuer. (Or similarly, I am using my smartphone and need to recall stuff done on my laptop.)
All they have to do is convince the user of the advantages of sending their recall data to the cloud and the user will agree to do so.
Where does it say that screenshots will go to the cloud?
And who says this will stay local and isn't a small T&C update away from being shared with 'trusted partners'?
Especially in many-party consent jurisdictions recording video calls by taking screenshots at a sufficiently high frequency without the explicit consent, or at least option to opt out, of remote parties is probably illegal.
Disclaimer: IANAL and this isn't legal advice.
The author does realize that windows 10+ keeps a timeline of all your copypasta right? Windows will even upload it to the cloud lol.
The keylogger is already here.
This is explicitly mentioned in the article too
You have to turn on "sync across devices" and have a MS account. One of the many excellent reasons to avoid one. Just in case MS makes a "mistake" and turns this one on by default as well.
I'm sure the cloud keeps all copypastas indefinitely, but you have to manually enable clipboard sync for that to happen.
Let's not be disingenuous.
How long before governments require all your data to be shared with them "for safety"?
How long before companies get to mine your data for advertising and "training better AI / ML assistants"?
How long before criminals get your data for hacking, extortion, etc. ?
How long before you cannot opt out?
Who is this feature really supposed to serve?
This is a good point IMO.
If you'd tell someone in the 90s that the government (or a corpo on their behalf) would scan all their drives they'd be laughing at you.
Now when everyone uses OneDrive and Dropbox this has become "normal".
But it got thinking, if git ever gets replaced would it be by something akin to this that is constantly monitoring all keystrokes in your repo folder and you never make a commit as such, the ai repo upstream just sorts it out..
This OS will, by design, waste 10% of rather limited SSD space, on a feature that I don't need, that may or may not work, and that also introduces a security risk on top of slowing things down.
It's going to be very useful™, as infinite browser history and bookmarks are to some people, I'm sure.
As for me, this sounds like a wonderful way to invent entirely new types of digital clutter.
Hoping there'll be a no-nonsense way to disable this "feature" in the "Pro" OS versions at least, along with Cortana/Copilot.
The crux of the argument boils down to > If you have malware running on your PC for only minutes, you have a big problem in your life now rather than just changing some passwords.
Ok. How often does malware run for just a few minutes? Honestly if there’s malware running on any of my machines, it’s not clear the outcome is any different here. Sure there’s a hypothetical scenario where it’s worse. But it’s also just as likely to grab a copy of my password manager while it has everything unencrypted. And it’s also way more likely to persist for days or months, making access to historical data moot as it’ll just install its own key logger.
In highly specific situations, like customer care, yes. Also these machines tend to be super locked down, no admin rights for the users.
And yes this practice is risky. I managed some PCs for customer care and one of them was taken in for repair by the service desk and replaced, and redeployed without wiping it. It ended up in payroll and when we found out their manager was furious (quite understandably so).
Yes malware can install its own keylogger. But it's an extra step once again.
It won't take long before the demand access to your stored search data.
Of course only to prevent major crimes like terrorism, child abuse and copyright infringement.
In the video, the WSJ reporter searches for the term "Brown leather bag" and ends up finding what appears to be an AI generated / stock image of a brown leather bag inside a PDF that has zero relevant textual content about brown leather bags.
Of all the things a user might be looking for with that search term, it's not going to be this!