Wholeheartedly agree. It's sad to slowly see the bitrot in my own bookmarks, trying to pull up important resources or media from only a few years ago only to hit a 404 or domain placeholder. And many times there is nothing on Archive.org or similar.
That ArchiveWeb extension looks excellent for custom captures. Picking the pages and elements you want during capture is a great feature.
For quicker one-offs, SingleFile gives you a single HTML with all the media of a page: https://www.getsinglefile.com
If you're making archiving a more involved part of your workflow, I highly recommend ArchiveBox. It can save pretty much every type of file from everywhere, in almost every format you can imagine, while building a searchable index of the archives. Been using it successfully for many years: https://archivebox.io
SingleFile is great for single pages. All the tweaking I do is enable or disable saving scripts, and it works for >90% of webpages. Output being a single file (heh) simplifies things enormously, there’s no hierarchy or folders to deal with - hard to imagine your saved files not loading or breaking in some way as long as the original capture works.
I used ArchiveBox for years. It's really good. The self-hosting and maintenance gets annoying after a while though.
Link rot really bothers me and I want people to get into archiving web pages so that eventually there will be a decentralized mirror of the unslopped web sitting on people's hard-drives, ready to share whenever someone needs it. So, I made a worse version of ArchiveBox ( https://memoryhole.app ). My philosophy is that, if people see how it is easy to start archiving and how useful it is, then they'll keep it up with SingleFile, or ArchiveBox or anything else.
I haven't tried archivebox, but self-hosted https://karakeep.com/ has been working well for me. When the server-side parsing fails I'll fallback to using SingleFile in my browser which then forwards it to my Karakeep instance.
Can vouch for Hister! In the fairly short time I've been using it, I already had a few sites disappearing or changing content and hister kept the juicy information accessible.
It was painless to set up and quick to use when needed, too.
I also use Singlefile and Obsidian Web Clipper a lot, but Hister has been great and is now a first-class citizen in my system. It helps it works passively for those things that don't look important when you read them, but become relevant later.
I am using both Linkding and Linkwarden self-hosted and archiving my bookmarks as HTML, PDF, and screenshot (I might be overdoing it) without the need for any additional dependency like client software or extension. Linkwarden works better at archiving things like Reddit, but now with the anti-scraping mechanisms in place it might het harder to do anyway.
I realized I really need this when one website which was a treasure trove for PowerMac technical information and assembly guides just went dark. Some of that info I could never find elsewhere. Linkrot sucks if you're in the middle of an (admittedly drawn out) personal project and your source of information vanishes with no trace. Belt and suspenders from now on. Might give Hister a try.
I just started using it, and to my surprise it only saves the sanitized html content, no scripts no css. I'm thinking of pairing this with archivebox. But hister is amazing. I love it. It's from the creator of searxng if I'm not mistaken which is another piece of great software.
On MacOS, I use my own built app called HEAP which lets me archive in various formats locally (full page image, pdf, markdown, html, and webarchive) via a single right click of the menu bar button:
I’ve been going over the art that I’ve saved in the past 15+ years and it’s quite sad when I go look for the original source of an image and find out that it’s completely gone from the original website. No indication that they’re on some other website, at least by reverse image searching it. Definitely has made me glad that I’ve been hoarding these images on my hard drives, with the total count being somewhere over 300k at the moment. I think the oldest ones have now been migrated across four different computers. Kinda sad for everyone else since they can’t see them anymore though.
Haven’t started saving web pages yet, but I definitely should though. My home server has some tens of terabytes to spare, so that’s at least not an issue. Been eyeing these comments to find a good self-hosted solution.
I use AnyBox on macOS to do this for bookmarks. It saves a WebArchive copy and a stripped 'reader'-esque version. SingleFile can also pipe into AnyBox, if you want to use that as the archive.
I recently spent a few days trying to figure out and discover my old websites and things from the early 2000s on the web archive. I hadn't realised my memory had gone quite so bad, but it was revitalising to find some of what I'd forgotten. So much of what I do remember is long dead and missing, sadly.
I thought it might be rose-tinted glasses, thinking that the earlier internet was better, but spending time retracing my teenage steps made me even more certain that it was indeed better. So many derpy little websites made by nerds, so many fun games and websites that weren't preying on vulnerable people, and everything spread out across many places - not centralised in the hands of large corporations. I've been using LLMs to have fun recreating concepts from that time, like my own forum software, as some sort of anti-progress. It's been fun.
In 5 years, cloud LLMs may have largely replaced the open Internet as a source of information, with centralized control and ability to rewrite the past.
Please, please buy books and keep them in good condition, and please archive websites with important information (assuming it's legal in your jurisdiction). There are still plenty of excellent sites and blogs around from the 2000s and even earlier, but they won't be there forever. Ironically, coding agents have made archival much easier -- I've always found problems with using pre-written tools, but Claude or Qwen make it trivial to download, test, and archive sites on your local.
... It's a joke about Grok which had a weird moment where it referred to itself as Mechahitler[1]. Also it's relevant because Grok also had/has some strong biases about the treatment of white South Africans[2]: so case in point of LLMs being used to alter facts about the past and the world.
Not the provider per se, but there have been several documented attempts to rewrite Wikipedia around historical and geopolitical topics. One of the reasons for this is Wikipedia feeds the chatbots.
There are already many people who are trying to rewrite the past. Holocaust deniers, Moon Landing deniers, people who deny the US Civil War was about slavery, and so on, and that’s just one country. Give one of them the ability to put their thumb on the scale when training an LLM and they absolutely will.
They been doing this from the start. Remember when Google's new image generator model, when asked to generate pictures of historical white people, would render them as black without fail? We only know of that example because of how incredibly ham-fisted and incompetent it was. The future of AIs is likely for them to only become more contaminated with ideological or monetarily incentivised biases, and we'll be less and less likely to be able to spot them.
It depends on what you mean by 'an LLM will try to rewrite the past'. But in either case I don't believe that stating that an LLM will try to rewrite the past is not an obvious, deliberate exaggeration (ie, hyperbole).
If you mean 'an LLM' as in the owner/maintainer of the LLM/weights, then examples of this already exist. Whether it's DeepSeek/Gwen downplaying Uyghur forced labor camps [0] or Grok hallucinating a 'white genocide' in South Africa [1], it is clear that the owners of LLMs have political agendas that are expressed in the weights of popular LLMs.
On the other hand, if you mean 'an LLM' as in an autonomous agent that could decide to rewrite history, that also seems very possible. Either by using existing techniques like, "poisoning ChatGPT, Gemini, and Google AI Overview answers by seeding the Web with malicious links and data and then optimizing the content." [2] Or by breaking out of its sandbox and updating its own internal weights that allow for an outcome that distorts historical facts. "In July 2026... OpenAI models circumvented controls designed to isolate them from the internet and compromised parts of OpenAI’s internal research infrastructure and Hugging Face’s systems ." [3]
I don't think that mainstream models will try to rewrite the past, but I wouldn't trust models to helpfully engage with your intent. As search engines encode better understanding of queries, they also become more model-like with refusals for "naughty" queries. Just compare something like
Disclosure Day free stream
across Yandex and Google. I'm sure that Google has indexed the same web sites, but Yandex has a free stream as the #1 result and Google has either deindexed it completely or pushed it below dozens of less-relevant, more-legal sites. (According to Google, the site Yandex showed me wasn't even DMCA'd out of the results.)
This isn't just relevant for piracy. Paid subscriptions to frontier models will refuse or downgrade answers to questions about the natural sciences and engineering. Unlike cybersecurity, where you can get specially approved to use models with fewer refusals, I don't think there is any program you can join where Anthropic will treat you like a grownup who has already read Carleone's Tactical Missile Warheads book. Opus just won't discuss details of high explosive engineering at all.
Have a look at Hister [1], a 'personal web history search engine' which by way of extensions for browsers captures your web history and stores actual page content on your own instance where it can be browsed and searched. It does not store images so if that's what you're mostly interested in it won't work for you. For text is works quite well, maybe sometimes too well since it also stores ephemeral things like HN comment pages unless you tell it not to.
the last source of truth are forums.. but not reddit... if it has a CEO it will rot and enshitify.
I'm talking about forums... the bmw forum for your old car... the motorcycle forum for the enthusiast of the SV650... those places are full of anecdotes, people talking, lending a hand to another person... being humans and care...
Which is what social media is/should/was about, is what internet is about before the enshitification happened... I would vote to have this ID associated to you via check of your local goverment issued id and ONLY UPPON VERIFICATION of you being using that you can log and read and use those websites without tracking you (Yes 10,000 ifs where this is secure, handled in a confidential matter, the whole tech is not to track that you are doing something... but that a human is behind that, no api, no robot, no adds, no nothing... just the old school forum where you can go, talk, submit a photo... where someone who wanted to spend time to write something did it, SO WE CAN ENSURE TWO THINGS, A HUMAN IS THERE, AND NO FVK!NG CEO WILL ENSHITIFY IT, no to track people, but to know that a person is interacting there)
But because "empathy is the weakness" and shareholder value...
> the last source of truth are forums.. but not reddit... if it has a CEO it will rot and enshitify.
Quick gripe-anecdote that occurred (again) just now: I have a technical question, the search-engine says it was discussed in a Reddit thread, so I go to Reddit, giving them my traffic and eyeballs just like they want... but the information is impossible to find because "new" Reddit hides/collapses 90% of the discussion by default and there's no way to tell which branch has what I need without manually clicking them all. (And them some of them will reload the entire page, erasing progress.)
So I have no sympathy for their whining about scraping when they spent already years making it painful for actual live organic humans to find anything.
"1. Webs of trust, in some way that resists crowds of sockpuppets." we are already there... is just that the frog soup had a very steady hand at rising the temperature.
The other option is to use the X402 protocol to pay, every time you access a page you pay in cpu to calculate some shitcoin.
For a person, is just trading the time spent loading and displaying ads for paying the creator for their work.
But if you are scrapping you are now paying with cpu and electricity to see things and content creators are being paid.
Yes, and with AI you can even back up the logic behind a website (the back-end), and even generate fake but plausible data to fill it if crawling is too much work.
This is why I started using Obsidian Web Clipper. There is SOME work to learn it and set it up, but the small amount of upfront work will pay for years to come. And the folder itself is backed by Dropbox, so I don't have to worry about losing it all forever.
I also regularly use the web clipper specifically for this reason - most of my conversations can be around articles I read online, and instead of googling for the article I have it referenced right in my journal. I find myself regularly upset that I didn't save old articles from years ago because I didn't have this system
Ages ago (like 2001 or so?) I started building something to automatically archive everything I browse. I used tcpflow (https://www.circlemud.org/jelson/software/tcpflow/). I've seen a fair bit of varions of this idea over the years
Then SSL/TLS started getting widespread and I lost focus. Nowadays, I guess you would build it as an extension in the browser.
I did actually. I took a whole day to go to each of my individual saved bookmarks and save the articles as pdf using edge reader mode, sometimes hunting down articles on the way back machine
i'm saddened that http://macsecret.com/ along with /files directory seems to have all be been lost. I has waiting on some storage for the NAS to archive it and it went away before I could back it up.
I learned this the hard way by getting too casual with Angelfire keeping an old website running for 20 years after I stopped updating it.
It had lots of photos that I hand-scanned from printed film, back in the day when it took a good bit of time to scan photos manually.
The site was a running joke, where I would open it to show people that it still existed... until it didn't, and I couldn't find my backup of all of the photos. The archive sites have the 'bones' of the website, but not the photos. It was easily avoidable, which is why it is painful.
Ooph. This is what I’m using netlify for. I clones all my old php sites into static copied and moved them to Netlify. Hopefully I’ll get a heads up before they close.
Plain text is not robust at all. Not now, and not in the futue. Its actually useless.
How do you save images in plain text ?
Lot of usefull information is stored in pictures.
ie: in technical articles about graphics, hardware.
Or some diagrams mentioned and refereced in the text.
Even medieval books have pictures, because some things need to be shown, and it is not enough to just write some text about it.
Anyway, youre probably just a very very old troll, typing this on a typewriter...
Bonus suggestion: get that internet archive extension for your browser that automatically sends every page you visit to the IA (if you're ok with the privacy side).
You don't have to worry about having your copy easily accessible, and as a bonus it's also available for everyone else in the world.
There are far better tools that are not built by this person’s company. Dig around and you’ll find an incredible array. Not everything has to be a sales pitch.
I did this for a while. I went so far as to built a highly advanced web crawler webapp that was better than the Internet Archive in a lot of ways.
But the modern web has made this unreasonable. Every webpage is 99% junk (CSS, JS, HTML structure, tracking pixels, UI images, navigation links, cryptominers (I joke)) and 1% content. You need a smart AI system to pick out the good parts, otherwise you'll quickly find yourself schlepping hundreds of gigs of junk to just preserve your favorite websites.
The better idea is: if you love information, back it up. Collect the knowledge from the website, not the website. And then preserve it. Create a free ebook. Or sell it on Amazon kindle. That'll last a lot longer than the website will, and any HTML backup of the site you could possibly make.
I'd suppose most of those junk websites have no value in them, really. The most valuable information in the web is in good ol' (x)html, maybe some css. But something like readability.js [1] can help in filtering the junk around the main info.
I had a graphing calculator website that had 250K monthly active users, from the logs it seemed mostly schools' math classes in Latin America. It was running on a single AWS EC2 Classic.
Ironically, when I started working for AWS, I always felt threatened with sticks and never had space to maintain my own projects. When they deprecated EC2 Classic, my site went down. I didn't get time or mental space to migrate it.
156 comments
[ 2.9 ms ] story [ 112 ms ] threadThat ArchiveWeb extension looks excellent for custom captures. Picking the pages and elements you want during capture is a great feature.
For quicker one-offs, SingleFile gives you a single HTML with all the media of a page: https://www.getsinglefile.com
If you're making archiving a more involved part of your workflow, I highly recommend ArchiveBox. It can save pretty much every type of file from everywhere, in almost every format you can imagine, while building a searchable index of the archives. Been using it successfully for many years: https://archivebox.io
Link rot really bothers me and I want people to get into archiving web pages so that eventually there will be a decentralized mirror of the unslopped web sitting on people's hard-drives, ready to share whenever someone needs it. So, I made a worse version of ArchiveBox ( https://memoryhole.app ). My philosophy is that, if people see how it is easy to start archiving and how useful it is, then they'll keep it up with SingleFile, or ArchiveBox or anything else.
My bookmarks file is a graveyard as I have 20 yrs’ old ones…
And as a publisher I don’t link fresh sites (don’t even mention Netlify, Lovable, Vercel…) … Vannevar Bush would be surprised how his Xanadu evolved.
[0] https://hister.org/
It was painless to set up and quick to use when needed, too.
I also use Singlefile and Obsidian Web Clipper a lot, but Hister has been great and is now a first-class citizen in my system. It helps it works passively for those things that don't look important when you read them, but become relevant later.
I realized I really need this when one website which was a treasure trove for PowerMac technical information and assembly guides just went dark. Some of that info I could never find elsewhere. Linkrot sucks if you're in the middle of an (admittedly drawn out) personal project and your source of information vanishes with no trace. Belt and suspenders from now on. Might give Hister a try.
https://apps.apple.com/ca/app/heap-website-full-page-image/i...
Disclosure: I am the developer.
another one i use when i only need to save the text of a webpage is markdownload, which formats the page using markdown
https://addons.mozilla.org/en-GB/firefox/addon/markdownload
Haven’t started saving web pages yet, but I definitely should though. My home server has some tens of terabytes to spare, so that’s at least not an issue. Been eyeing these comments to find a good self-hosted solution.
i managed to reconstruct (and then improve) it with an LLM:
https://scott-cpu.cs.montana.edu/
I thought it might be rose-tinted glasses, thinking that the earlier internet was better, but spending time retracing my teenage steps made me even more certain that it was indeed better. So many derpy little websites made by nerds, so many fun games and websites that weren't preying on vulnerable people, and everything spread out across many places - not centralised in the hands of large corporations. I've been using LLMs to have fun recreating concepts from that time, like my own forum software, as some sort of anti-progress. It's been fun.
Please, please buy books and keep them in good condition, and please archive websites with important information (assuming it's legal in your jurisdiction). There are still plenty of excellent sites and blogs around from the 2000s and even earlier, but they won't be there forever. Ironically, coding agents have made archival much easier -- I've always found problems with using pre-written tools, but Claude or Qwen make it trivial to download, test, and archive sites on your local.
Hacker News has sure dropped off.
1. https://www.npr.org/2025/07/09/nx-s1-5462609/grok-elon-musk-...
2. https://www.theguardian.com/technology/2025/may/14/elon-musk...
If you mean 'an LLM' as in the owner/maintainer of the LLM/weights, then examples of this already exist. Whether it's DeepSeek/Gwen downplaying Uyghur forced labor camps [0] or Grok hallucinating a 'white genocide' in South Africa [1], it is clear that the owners of LLMs have political agendas that are expressed in the weights of popular LLMs.
On the other hand, if you mean 'an LLM' as in an autonomous agent that could decide to rewrite history, that also seems very possible. Either by using existing techniques like, "poisoning ChatGPT, Gemini, and Google AI Overview answers by seeding the Web with malicious links and data and then optimizing the content." [2] Or by breaking out of its sandbox and updating its own internal weights that allow for an outcome that distorts historical facts. "In July 2026... OpenAI models circumvented controls designed to isolate them from the internet and compromised parts of OpenAI’s internal research infrastructure and Hugging Face’s systems ." [3]
[0] - https://www.cbsnews.com/news/china-ai-qwen-artificial-intell...
[1] - https://www.cnbc.com/2025/05/17/groks-white-genocide-respons...
[2] - https://www.darkreading.com/threat-intelligence/attackers-ma...
[3] - https://openai.com/index/hugging-face-incident-and-the-road-...
Didn't they already try this last year?
This isn't just relevant for piracy. Paid subscriptions to frontier models will refuse or downgrade answers to questions about the natural sciences and engineering. Unlike cybersecurity, where you can get specially approved to use models with fewer refusals, I don't think there is any program you can join where Anthropic will treat you like a grownup who has already read Carleone's Tactical Missile Warheads book. Opus just won't discuss details of high explosive engineering at all.
And remember this https://www.reddit.com/r/Mcat/comments/5yffns/uc_berkeley_is... / https://news.ycombinator.com/item?id=13768856
So no it's not an hyperbole, what used to be a place to share content becomes closed source.
People keep spouting torrenting is a solution, but a shitload of torrents are dead now.
[1] https://github.com/asciimoo/hister/
I'm talking about forums... the bmw forum for your old car... the motorcycle forum for the enthusiast of the SV650... those places are full of anecdotes, people talking, lending a hand to another person... being humans and care...
Which is what social media is/should/was about, is what internet is about before the enshitification happened... I would vote to have this ID associated to you via check of your local goverment issued id and ONLY UPPON VERIFICATION of you being using that you can log and read and use those websites without tracking you (Yes 10,000 ifs where this is secure, handled in a confidential matter, the whole tech is not to track that you are doing something... but that a human is behind that, no api, no robot, no adds, no nothing... just the old school forum where you can go, talk, submit a photo... where someone who wanted to spend time to write something did it, SO WE CAN ENSURE TWO THINGS, A HUMAN IS THERE, AND NO FVK!NG CEO WILL ENSHITIFY IT, no to track people, but to know that a person is interacting there)
But because "empathy is the weakness" and shareholder value...
Quick gripe-anecdote that occurred (again) just now: I have a technical question, the search-engine says it was discussed in a Reddit thread, so I go to Reddit, giving them my traffic and eyeballs just like they want... but the information is impossible to find because "new" Reddit hides/collapses 90% of the discussion by default and there's no way to tell which branch has what I need without manually clicking them all. (And them some of them will reload the entire page, erasing progress.)
So I have no sympathy for their whining about scraping when they spent already years making it painful for actual live organic humans to find anything.
The other option is to use the X402 protocol to pay, every time you access a page you pay in cpu to calculate some shitcoin.
For a person, is just trading the time spent loading and displaying ads for paying the creator for their work.
But if you are scrapping you are now paying with cpu and electricity to see things and content creators are being paid.
History and this very thread should show you how this is a naive statement.
Then SSL/TLS started getting widespread and I lost focus. Nowadays, I guess you would build it as an extension in the browser.
It had lots of photos that I hand-scanned from printed film, back in the day when it took a good bit of time to scan photos manually.
The site was a running joke, where I would open it to show people that it still existed... until it didn't, and I couldn't find my backup of all of the photos. The archive sites have the 'bones' of the website, but not the photos. It was easily avoidable, which is why it is painful.
How do you save images in plain text ? Lot of usefull information is stored in pictures. ie: in technical articles about graphics, hardware. Or some diagrams mentioned and refereced in the text. Even medieval books have pictures, because some things need to be shown, and it is not enough to just write some text about it.
Anyway, youre probably just a very very old troll, typing this on a typewriter...
Base64, like countless websites. Or just accept that you don't have to act like a lawyer with internet comments and simply save them as image files.
I mean, technically, we have 7000yo graphics, but they were really too low-res to be functional for archiving data.
You don't have to worry about having your copy easily accessible, and as a bonus it's also available for everyone else in the world.
But the modern web has made this unreasonable. Every webpage is 99% junk (CSS, JS, HTML structure, tracking pixels, UI images, navigation links, cryptominers (I joke)) and 1% content. You need a smart AI system to pick out the good parts, otherwise you'll quickly find yourself schlepping hundreds of gigs of junk to just preserve your favorite websites.
The better idea is: if you love information, back it up. Collect the knowledge from the website, not the website. And then preserve it. Create a free ebook. Or sell it on Amazon kindle. That'll last a lot longer than the website will, and any HTML backup of the site you could possibly make.
[1]: https://github.com/mozilla/readability There's also a C rewrite: https://github.com/eafer/rdrview
It can also apply Readability before saving and it has a simple editor that you can use to remove unwanted html elements from a page.
Ironically, when I started working for AWS, I always felt threatened with sticks and never had space to maintain my own projects. When they deprecated EC2 Classic, my site went down. I didn't get time or mental space to migrate it.
Cest la vie I guess.