HN Explorer

Ask HN: Add Archive.org link in past section of HN?

39 points by ss48 ↗ HN
In the past section, there are several older links that give 404 errors. It would be great to add an archive.org link from same or earlier date so it's more convenient to go to the saved copy of the article.

20 comments

[ 2.8 ms ] story [ 48.6 ms ] thread
BitwiseFool
Hopefully dang sees this and can help. I know the mods are able to unlock a comment for editing. However I think they would most likely allow you to post a followup comment with the archive.org link.
black_puppydog
I think the idea was to literally add a UI element next to the "flag", "hide" "N comments" ones that just goes straight to archive.
altairprime
If someone emails dang using the footer contact link, he'll certainly see this!
ZeroGravitas
I wonder if it's possible to trigger archive.org to save the exact version that gets submitted and then link specifically to that, rather than earlier or later versions.
captn3m0
Yeah, link to the date of submission snapshot. I think archive takes you to the nearest snapshot in that case?
krapp
I got that to work in Anarki's fork of the HN forum a while ago[0] - a page presents you with a link to archive the url, and it saves the snapshot with the url in a table somewhere. Once archived, that link becomes a link to the archived page.

[0]https://github.com/arclanguage/anarki/pull/179

squeaky-clean
I think they're asking if, upon submission, HN could request an IA snapshot. That way the nearest snapshot will be from the day of posting and not some unknown time
lucb1e
I was thinking the same yesterday, looking for "interview question" submissions (there have been quite a few good ones) using the algolia HN search.

Out of about ten links I clicked, I think there was only one that still worked. All the rest I had to run through the IA, whose homepage takes 5 seconds to load enough for the search box to show up, then wait for it to load the captures, and finally wait for it to fetch one of the captures. A direct link would be very welcome.

The addition should be easy, no need to query the IA site or anything because IA will find the nearest date to the request automatically iirc. Something like one week after the submission would allow post-submission updates without being likely to show a 404 capture.

(By the way, the question I ended up picking for our applicant was: "what did you like or dislike about the culture in previous places you've worked at or interned?" It went over well and I got some insight into what they'd expect (or enjoy) when joining the team. The colleague I did the interview with later said it was a good question :).)

sillysaurusx
I considered this a few months ago, but it's a bad idea for a few reasons. One is that it increases the load on archive.org. (There are a significant number of bots, and adding 30 million new links for them to spider is no small thing.)

The more important reason is that it's hard to do programmatically. For every link, you'd want to link to the most recent, active archive entry. As far as I know, this isn't easy to determine.

Have you ever had the experience of going to archive.org, plugging in a url, and seeing that the most recent one is actually dead, but then you can go back in time and find what you're after?

It's easy to say "Well, that's ok. Take me to the dead link, and I'll go back myself." Maybe. But with Dan's exacting standards, I somehow don't think that'll pass muster.

The ultimate reason this probably won't happen, though, is because people can do it themselves. It's not hard to copy the link and visit archive.org. Every other link (past, flag, hide, favorite) is there because it's hard to do.

EDIT: On the other hand, https://news.ycombinator.com/item?id=30384630 shows a way it could be done. I didn't know about the date wildcard feature.

ffpip
> most recent, active archive entry. As far as I know, this isn't easy to determine.

https://web.archive.org/$link will always take you to the most recently archived version of the link.

Eg: https://web.archive.org/https://example.com

sillysaurusx
Yes. The point is that if you try doing that for links far in the past, you'll often get a dead page.

By "dead" I mean different content than the one that was submitted to HN at the time. So you need a way of specifying the date.

There's a way to do that, though, which is interesting. My edit above points out a comment to get it.

ss48
This could be challenging for existing links, but at least moving forward, when a submission is made, HN could autorequest an archive be made on archive.org and then use that when link is available in past or search.
Fnoord
Should be doable via a user script in e.g. Tampermonkey or Greasemonkey. It would avoid 30 million links for bots but would allow the feature for power users.
joecool1029
>One is that it increases the load on archive.org. (There are a significant number of bots, and adding 30 million new links for them to spider is no small thing.)

I mean, Wikipedia did it and archive.org survived. That has to be a totally different ballpark with regards to backlinks and traffic than niche HN.

sillysaurusx
Niche? :)

Maybe a few years ago.

But you’re probably right about the load.

jka
> For every link, you'd want to link to the most recent, active archive entry.

Hrm, are you sure about that?

I think I'd generally prefer to see each article as it looked when it was posted to HN, at least by default.

(that would translate to: minimum archive.org timestamp that is greater than the HN post timestamp)

Edit: ah, I've read your reply where you mention this exact use case. All good.

sillysaurusx
Yeah, the more we talk it over, the better of an idea it sounds. It could even be restricted to submissions more than a year old.

It would be an interesting experiment. You could prototype it with a tampermonkey script and post it as a Show HN.

anonu
Sounds like a task for a browser extension! The minimalist approach to HN is desirable. Adding features clutters it up.
lxe
Or even... a hn-specific crawler+cache that caches pages as soon as they reach a certain threshold of votes or even just appear in HN.
KoftaBob
The Web Archives browser extension is great for this. It allows you to pull up archived versions of whatever website you're on, from a variety of archive sources. My go-to lately has been using the www.archive.is option when I use the extension.

https://github.com/dessant/web-archives