I agree that the original article should be linked here, not my summary.
I take offense to the clickbait accusation. This post wasn't intended to be shared elsewhere, it was a bookmark on my personal website. I have 7,000 more of those here: https://simonwillison.net/search/?q=&type=blogmark
I understand and don't disagree with this sentiment, but I also think it's worth considering the Internet fame you've developed over the decades. Your blog isn't a diary. A lot of people are reading it and you might take that into consideration. It's going to be shared far and wide whether you intend it or not.
I'm not really a fan of Scott Alexander any more, but something he used to do years back that I liked was putting an "epistemic status" disclaimer at the top of every new post. Some were well-researched, evidence-backed treatises. Some were barely idle shower thoughts. Everyone's musings, whether they write them down or not, probably fall into categories like these. But when you know others are reading it, it can be helpful to make very explicit which is which so readers know what to take seriously and what to consider exploratory and possible but also only preliminary and maybe a deadend or even wrong.
I understand and don't disagree with this sentiment, but I also think it's worth considering the Internet fame you've developed over the decades. Your blog isn't a diary. A lot of people are reading it and you might take that into consideration.
Their blog can be anything they want? The post literally starts with the link to the original source. It's not like those typical clickbait sites that bury the link somewhere deep down. IMO, it's up to the submitter to link to the original source.
I see it more as quote-Tweets, someone with reach amplifying useful/interesting articles.
It's basically my name for a link blog. My biggest inspiration for this is https://daringfireball.net where most of the posts are links plus commentary but sometime they are full blown essays in their own right.
I've been wanting something like this for my entire life and it seems like computers are finally heading in a direction of becoming extensions of ourselves instead of just clunky tools we have to adapt ourselves to. This has been the kind of sci-fi I've dreamt of since I was a kid.
I get the privacy concerns. And security concerns. But honestly, if somebody has this kind of access to your computer, they have access to your entire life anyway. Setting up a keylogger, getting all your passwords, getting any vital documents, etc. It's trivial. When is it just pointless fear mongering? It's subject to the same security concerns as everything else and we don't suggest people switch to pen and paper instead.
> But honestly, if somebody has this kind of access to your computer, they have access to your entire life anyway.
Exactly. So please tell me why I would want to essentially allow something like this on my machine?
As for the scifi-esque abilities: All my important information is stored in a personal wiki, which is automatically backed up and encrypted on a backup server. It's fully tagged, its fully searchable. And you know what? I have even set up a pipeline to feed it into a locally run LLM if I want to. Yes, I can do RAG on my personal wiki.
> All my important information is stored in a personal wiki
Ah, there's the crux, no? I dream of a system that automatically captures and allows me to interact with and query against everything I do, every ebook I download, every image, every document, every video, every site. What you're describing isn't this, it's not even the same ballpark.
Recall, even if it isn't perfect, is in the same ballpark.
> Exactly. So please tell me why I would want to essentially allow something like this on my machine?
Because it's useful. I don't understand this. It's like a circular argument. I want a useful feature, I install it on my PC. The security concerns apply regardless of Recall existing.
> I dream of a system that automatically captures and allows me to interact with and query against everything I do
Well, I don't.
For starters, because not everything I do generates relevant information for me. All those hours spent surfing through random blogs or wikipedia articles, most news I read, or watching some random scifi series just isn't info that I will ever need to refer back to. It's just noise. Best case scenario, it's useless information that just bloats the database.
Alternative scenario: It messes with the results. Intelligence and statistics are not the same thing. So what if I read some scifi novel on my screen, and the content of that pages embedding just so happens to score higher to a future query than an actually useful document?
In my system, I decide what's relevant information and what isn't. Subsequently, when I use RAG on this, I use it on already curated data. That data is already having structure imposed on it, so it is accessible to me with our without the help of AI.
--------
Then there is also the whole issue about information that is simply too sensitive to store in my personal Wiki. For example: I would NEVER store my bank account details, or API keys in there, even though everything is encrypted. All passwords and similar information go into a separate system FOR A REASON; Single points of failure are bad, and this goes double in security.
Now, a system that takes a screenshot and runs OCR on everything? Cool. What if it does so when I am just running a testscript after setting a temporary envvar in preparation? I don't want that info anywhere outside of my password manager. And yes, this does matter, even if everything is stored locally.
--------
And, of course, we haven't even started on the whole issue of privacy, and how people will feel about a system that basically logs and stores everything the user does. It doesn't matter how useful it could be if people don't feel comfortable with it.
> The security concerns apply regardless of Recall existing.
Not quite. Because there is a world of difference between an attacker being able to grab information that is being entered or temporarily present, or permanently present and encrypted, and an attacker being able to, within seconds, grab everything the user ever did, watched, saw, entered, etc. on his machine in one fell swoop:
"Q. But if a hacker gains access to run code on your PC, it’s already game over!
A. If you run something like an info stealer, at present they will automatically scrape things like credential stores. At scale, hackers scrape rather than touch every victim (because there are so many) and resell them in online marketplaces.
Recall enables threat actors to automate scraping everything you’ve ever looked at within seconds.
During testing this with an off the shelf infostealer, I used Microsoft Defender for Endpoint — which detected the off the shelve infostealer — but by the time the automated remediation kicked in (which took over ten minutes) my Recall data was already long gone."
End Quote.
And this is THE core problem with such a system, that just hoovers up everything, everywhere, all at once: It creates a single point of failure so critical, it will instantly become the prime target of every attack, because no other target is needed any more.
Yeah, adding DUPE to this all of a sudden has made it disappear off the top spot (and main page) so people will be less likely to find out about this today.
And it’s kinda important (understatement) for everyone to know about… especially if they use Windows or have family and friends who use it.
22 comments
[ 5.7 ms ] story [ 56.8 ms ] threadThe blog post is insanely reductive and clickbaitish
I take offense to the clickbait accusation. This post wasn't intended to be shared elsewhere, it was a bookmark on my personal website. I have 7,000 more of those here: https://simonwillison.net/search/?q=&type=blogmark
I'm not really a fan of Scott Alexander any more, but something he used to do years back that I liked was putting an "epistemic status" disclaimer at the top of every new post. Some were well-researched, evidence-backed treatises. Some were barely idle shower thoughts. Everyone's musings, whether they write them down or not, probably fall into categories like these. But when you know others are reading it, it can be helpful to make very explicit which is which so readers know what to take seriously and what to consider exploratory and possible but also only preliminary and maybe a deadend or even wrong.
Their blog can be anything they want? The post literally starts with the link to the original source. It's not like those typical clickbait sites that bury the link somewhere deep down. IMO, it's up to the submitter to link to the original source.
I see it more as quote-Tweets, someone with reach amplifying useful/interesting articles.
A blog is exactly what the one who writes it wants it to be.
It probably needs a heading or something to make it clear it is a summary of another blog.
I was a bit confused I assumed this was your blog post at first.
Maybe because I am on mobile there is some missing context about what this is.
Have you written anything about that yet?
It's basically my name for a link blog. My biggest inspiration for this is https://daringfireball.net where most of the posts are links plus commentary but sometime they are full blown essays in their own right.
https://waxy.org is another link blog that I really like.
Here's mine: https://linkfeed.net/u/5rh44yzy
And RSS: https://linkfeed.net/u/5rh44yzy/feed
I didn't find it clickbaity at all.
I get the privacy concerns. And security concerns. But honestly, if somebody has this kind of access to your computer, they have access to your entire life anyway. Setting up a keylogger, getting all your passwords, getting any vital documents, etc. It's trivial. When is it just pointless fear mongering? It's subject to the same security concerns as everything else and we don't suggest people switch to pen and paper instead.
Exactly. So please tell me why I would want to essentially allow something like this on my machine?
As for the scifi-esque abilities: All my important information is stored in a personal wiki, which is automatically backed up and encrypted on a backup server. It's fully tagged, its fully searchable. And you know what? I have even set up a pipeline to feed it into a locally run LLM if I want to. Yes, I can do RAG on my personal wiki.
Ah, there's the crux, no? I dream of a system that automatically captures and allows me to interact with and query against everything I do, every ebook I download, every image, every document, every video, every site. What you're describing isn't this, it's not even the same ballpark.
Recall, even if it isn't perfect, is in the same ballpark.
> Exactly. So please tell me why I would want to essentially allow something like this on my machine?
Because it's useful. I don't understand this. It's like a circular argument. I want a useful feature, I install it on my PC. The security concerns apply regardless of Recall existing.
Well, I don't.
For starters, because not everything I do generates relevant information for me. All those hours spent surfing through random blogs or wikipedia articles, most news I read, or watching some random scifi series just isn't info that I will ever need to refer back to. It's just noise. Best case scenario, it's useless information that just bloats the database.
Alternative scenario: It messes with the results. Intelligence and statistics are not the same thing. So what if I read some scifi novel on my screen, and the content of that pages embedding just so happens to score higher to a future query than an actually useful document?
In my system, I decide what's relevant information and what isn't. Subsequently, when I use RAG on this, I use it on already curated data. That data is already having structure imposed on it, so it is accessible to me with our without the help of AI.
--------
Then there is also the whole issue about information that is simply too sensitive to store in my personal Wiki. For example: I would NEVER store my bank account details, or API keys in there, even though everything is encrypted. All passwords and similar information go into a separate system FOR A REASON; Single points of failure are bad, and this goes double in security.
Now, a system that takes a screenshot and runs OCR on everything? Cool. What if it does so when I am just running a testscript after setting a temporary envvar in preparation? I don't want that info anywhere outside of my password manager. And yes, this does matter, even if everything is stored locally.
--------
And, of course, we haven't even started on the whole issue of privacy, and how people will feel about a system that basically logs and stores everything the user does. It doesn't matter how useful it could be if people don't feel comfortable with it.
Not quite. Because there is a world of difference between an attacker being able to grab information that is being entered or temporarily present, or permanently present and encrypted, and an attacker being able to, within seconds, grab everything the user ever did, watched, saw, entered, etc. on his machine in one fell swoop:
https://doublepulsar.com/recall-stealing-everything-youve-ev...
Quote from the Q&A Section of the post:
"Q. But if a hacker gains access to run code on your PC, it’s already game over!
A. If you run something like an info stealer, at present they will automatically scrape things like credential stores. At scale, hackers scrape rather than touch every victim (because there are so many) and resell them in online marketplaces.
Recall enables threat actors to automate scraping everything you’ve ever looked at within seconds.
During testing this with an off the shelf infostealer, I used Microsoft Defender for Endpoint — which detected the off the shelve infostealer — but by the time the automated remediation kicked in (which took over ten minutes) my Recall data was already long gone."
End Quote.
And this is THE core problem with such a system, that just hoovers up everything, everywhere, all at once: It creates a single point of failure so critical, it will instantly become the prime target of every attack, because no other target is needed any more.
All the discussion seems to be here:
"Recall: Stealing everything you've ever typed or viewed on your own Windows PC" 171 comments https://news.ycombinator.com/item?id=40540703
And it’s kinda important (understatement) for everyone to know about… especially if they use Windows or have family and friends who use it.