Am I the only one who is seeing this sudden surge of bbc.com linked articles ? Not that I'm complaining.
When people started making pages for Search Engine instead of Human Consumptions, Internet died. "Light on information and Heavy on keywords and linking" sites and pages flooded the internet, through which finding the actual content, needs another level of skill.
I think solution to this is, an AI based open source browser extension, which find relevant page from search engine result depending upon its usefulness index. ( or just show its usefulness index, along side the result, in short sabotaging the Google search results ;) ). Is there anything similar available ?
It's an Eternal September problem, not a search engine problem. Whenever I use search like a proper normie to buy something, check song lyrics, or weather, the results are excellent.
On that note, lately it seems Google results are more and more about what’s ”popular” in some sense instead of what’s relevant, especially when your search term has multiple meanings. Two recent examples of things I’ve run into are ”cloud” (pages and pages of cloud computing results, no mention about the meteorological phenomenon) and ”thrush” (nothing about the songbird family, just dozens of results about the yeast infection). Whether I’m logged in or use private browsing doesn’t make much difference. Not sure what has changed, but I’m used to Google giving a more representative sample of different meanings of a search term.
Duck Duck Go has a similar problem for cloud and thrush, but if you click the “meanings” tab, it lets you switch results over to what you meant, and then gives decent results.
More broadly, DDG implements UI features that allow users to clarify their intent, while Google focuses on using machine learning to infer intent.
When things go sideways, this means DDG empowers users, and google overrides them.
Disclaimer: I switched away from google search years ago, and now I find its user interface completely baffling.
Maybe I just don’t know how to use it any more, and there is some override I’ve overlooked.
Also, I’m kind of shocked by how many corporate logos and spam links fly by when I scroll down the Google results for “cloud”. It feels like the 90’s internet before animated gifs and the blink tag were invented.
In fairness to google, ddg has a similar number of logos, but they’re each approximately the size of one normal sized character, not one third the width of the screen like in google.
> Duck Duck Go has a similar problem for cloud and thrush, but if you click the “meanings” tab, it lets you switch results over to what you meant, and then gives decent results.
This is a good idea but not well implemented. For example, searching Python displays a relevant carousel of "meanings" - you have the snake represented, and the programming language. But choosing an item from the list simply changes the search term.
This is particularly bad with "Ruby". Perform a search for Ruby and then click the 'a pink to blood-red colored gemstone' meaning. Observe that the top result now that you have clarified is "Ruby Programming Language".
But are relevance and popularity not related? If more people are using Google to learn about cloud computing than the clouds in the sky, I’d hazard that that would make cloud computing the more relevant result, in terms of what Google sees anyway.
It’s tricky, though, because I’d say that the clouds in the sky are more relevant to humanity as a whole. Not everyone cares about or needs to think about cloud computing.
Both are relevant meanings and should be included in top results. Perhaps Google should have a disambiguation feature like Wikipedia does (and DuckDuckGo does according to another commenter).
I absolutely agree with you there. In the meantime, adding additional descriptors yields better results but is maybe a bit un-intuitive to some users (e.g. "thrush bird" rather than "thrush").
They used to - I remember pushing this feature hard when working on the search visual redesign in 2010, and getting the response that it was just a really hard engineering problem. The back-end teams delivered sometime in late 2011, and IIRC it was live in the left-nav (remember when Google had a left-nav?) for a few months to a year.
It was unlaunched eventually because nobody ever clicked on it. It turns out that disambiguation is a relatively rare feature for people to want when most of the query stream is navigational queries, and it's contrary to Google's goal of getting you to your answer as quick as possible (2 clicks if it were a UI feature vs. one click if the result were just in the results). AFAIK the backend is still live, but it triggers off keywords in the query rather than any UI element. If you search for [cloud weather] 100% of the results are about the meteorological phenomena, if you search for [thrush bird] 100% of the results are about the songbird, if you search for [jaguar car] 100% of the results are about the car, and if you search for [python snake] 100% of the results are about the reptile.
Interesting, thanks! Yeah, disambiguating with a more specific search query is, of course, a fine solution. I guess I've just became accustomed to Google basically reading my mind compared to the bad old days of search engines when figuring out the proper incantation to get the results you want was akin to a logic puzzle.
It's even worse than that - it also appears to learn synonyms from popular usage, even when it is incorrect or misleading.
For example, when it comes to firearms, there are two completely different parts called "extractor" and "ejector", that people often confuse because of similar sounding names. So now, when you search for "AR extractor removal", the first hit on Google is about ejectors; and it specifically highlights the word "ejector" as bold, as if it came from the search query:
Federated search (please don't say "AI") may eventually become part of the internet. It's garbage now, and the skills required to make it go are generally employed at google, bing or yandex, but it's one of those things that should have happened.
> Am I the only one who is seeing this sudden surge of bbc.com linked articles ? Not that I'm complaining.
I think it comes in waves. There was a surge of TechCrunch articles the other day. I noticed because they're blacklisted in pi-hole due to their crappy UI.
Websites got a lot less fun at some point too. A lot of the designs seem dated now, but I really miss the days when websites had brightly colored text over load backgrounds. Obviously modern sites are more usable, but I want <blink> tags back.
That and 2/3rds of websites are just Shopify templates, etc.
I suppose this is the natural progression as a given thing becomes more accessible, that the things themselves lose personality and flair because literally anybody can do it now. It reminds of cars, actually, how back in the early days every car was unique because when you bought a "car" what you received was a chassis, engine, wheels, etc. and then you'd take that to a coach builder who'd finish the job to your liking. Then as time went on, things got standardized (for cars, it was manufacturers and models, for websites, it's companies like Shopify, Wordpress, Joomla, etc. and then slowly everything gets paired down into what is generally agreed upon as the "correct" set of features, and they all just look a bit the same.
And then of course you occasionally have a teenager who fits a great stereo to a Mazda or something, but by and large, everything is pretty much standardized.
I know it's more efficient, but it's terribly boring.
> things themselves lose personality and flair because literally anybody can do it now.
I'd argue that fewer people can do it now (relative to the Internet population) than previously, because free webspace is dead, the desktop is dead (the laptop close behind it), ftp is dead, DMCA, GDPR, hackers are scary, HTML is much more complex (e.g. responsive design).
The web is a playing field for larger corporations now and most people and small entities have given up and just maintain a FB/IG presence.
All the people getting online around the developing world overwhelmingly own only phones. Their phones are the only electronic device they have. Even in developed countries it is becoming extremely common now for people whose don't create much text content to only use a phone or tablet. Desktops and tablets are something they maybe only see at work.
In the early days aka pre 1990 very few people where online or had access to computers worldwide. In the 90’s access kept increasing but mostly in the developed world which is ~1/6 of global population.
New PC sales makes it look like access is dropping, but the reality is people simply use older Computers on average. A 10 year old desktop in 1998 is relatively ancient, but in 2018 it’s not that far behind the curve. On top of that tablets and smartphones often replace secondary computers. If you have a powerful gaming desktop and a tablet then a laptop is less useful.
This is going to vary by area. In various locations I've lived, all in the developing world, phones are used for mindless social media and games, desktops are used for everything else. There are some cultural things that might make the numbers look misleading or just generally be very difficult to measure. For instance internet cafes tend to be ubiquitous in many parts of the world which provide easy and extremely comfortable access to high end computers which people in many cases use effectively as their own. With prices that start at some pennies per hour, it also often makes more economic sense in many cases.
Another big issue is that 'computer malls' are also still a thing. Think something like Fry's, but 6 stories tall and full of nothing but independent shops. These places drive two confounding issues. The first is that they build [lots] of computers a la carte. PC sales figures only come from major distributors like Dell, HP, etc. So even though there are millions of PCs sold at these malls, it'd count as 0 by most metrics. And another factor is that people tend to repair/refurbish/upgrade their computers rather than buy new ones. These malls again facilitate such things at very affordable prices.
I can count the number of tablets I've seen on one hand.
An aside but surely this must be positive in terms of environmental impact. Imagine the millions (billions?) of desktops and laptops that have not been manufactured or powered because of the widespread availability of phones.
Conversely, desktops and laptops can be upgraded much easier than phones, so I'm not sure the long-term environmental impact is necessarily positive, once all those millions of phones end up in landfills.
It has something like 40% of web traffic now, despite those with desktop PCs being more likely professionals in the field, so normal people simply don't really use a desktop/laptop anymore to browse the web.
I'm not going to say you're wrong, but I can't make sense of your post and it seems contrary to my experience.
> It has something like 40% of web traffic now
I'm not sure what your metric is here so I've assumed volume.
40% of web traffic now is an order of magnitude more traffic than all web traffic a decade ago. I'm not sure how something which has an increased volume and still has 40% of the total pie can be 'dead'. I might buy 'dying' if it was accompanied with more data.
> ...despite those with desktop PCs being more likely professionals in the field...
I'm not sure what you are saying. It seems like you're saying outside of tech, businesses don't use desktops. That does not jibe with my experience. I have seen an increase in POS systems using tablets, but those are primarily replacing cash registers, bespoke POS systems, etc.
> ...so normal people simply don't really use a desktop/laptop anymore to browse the web.
With the exception of the people I know who own a laptop but not a television, it seems that all normal people do with their laptops is browse the web.
The devices are still sold. However, they're increasingly just used as browsers. I was just saying to someone that it's something of an irony that Chromebooks seem to have settled into something primarily for the K-12 education market when they're really all the PC that most people need.
I actually travel on business with one a lot of the time but will probably replace it because no one really makes the hardware I want.
Historically, when people have talked about desktops and laptops they have generally been thinking in terms of a stateful device that ran applications locally. Increasingly they're just a convenient form-factor to run a browser. They're effectively functioning as a modern-day thin client.
When people say the desktop/laptop is largely dead. That's what they mean. [ADDED: That, and that web access is increasingly through a phone.]
Yep. The browser + web has effectively evolved into what all the deliberate attempts to create thin clients mostly failed at. (Just as app stores largely do what application virtualization was supposed to.)
If I could only have one consumer computing device to make a movie, or a drawing, or a anything involving photos, I'd pick one of those "consumption" devices over a desktop or laptop without hesitation. Even a phone, for anything involving the camera. The form factor and extra/better sensors make them great for all kinds of things.
I'd also rather have an iPhone helping me out in the workshop than a laptop or desktop, for a bunch of reasons. I can use it as a measuring device, a level, a note-taking/shopping-list machine (hands free, even!), a youtube how-to player, an instruction manual finder, and so on, and it fits in my pocket and reasonable amounts of sawdust and liquid won't hurt it. That's creation.
People spend most of their time on "real" computers consuming, too, by a huge margin. I still wouldn't say their (computers') primary focus is consumption. Anyway, the premise is kind of nonsense as consumption is a major part of many creative processes.
This is all true. But the barrier to go from consuming to creating is a lot higher now.
Making silly little programs in BASIC or Perl is a lot tougher on an iphone, and that's what sparked the interest for a lot of people (though maybe I'm wrong and it's getting easier?)
For the specific case of creating software, then? I'd say the Swift Playground is (much) easier to get started with and also more compelling than hacking around with Perl on (at first) Win98 was.
This was my point, though you put it much more articulately. People used to make websites because they were enthusiastic about The Smashing Pumpkins, Fight Club, or their EverQuest guild. Now they make sites for a business. Business ruined eBay, and then it spread to the rest of the web. If you have a passion you want to building for, you have to accept a terms of service of a big platform and build with their tools. It might be easier, but it sure has been less fun to view.
Nope, it's better than ever. You have non-profit providers like https://neocities.org/, regular shared hosting with free plans, and stuff like Github pages and Netlify for those with a bit more tech skills.
Way better than the times when you had Geocities and such injecting ads into your site.
> the desktop is dead (the laptop close behind it)
No need for those to build a website. Any cheap Android phone can do it.
> ftp is dead
Nope, the previously mentioned shared hosting providers still offer it.
> HTML is much more complex (e.g. responsive design)
Old-school HTML pretty much works the same, and is actually quite fine on both desktop and mobile screens.
--
Frankly, I don't think the problem is that making your own site has gotten any harder, it's that there was no FB/IG back then. Had there been FB pages from the start of the web, you'd probably would have never seen so many custom sites in the first place.
> Had there been FB pages from the start of the web, you'd probably would have never seen so many custom sites in the first place.
I’m not sure about this. AOL, Prodigy and CompuServe all had vibrant online communities. However, what you could do was much more limited than the federated web, and they were silos.
Without the competition of “I’ll host my own server, and search engines will index it”, I wonder if FB and it’s predecessors would have been able to figure out what to build.
Well Facebook was built on the failures of MySpace which in turn was built on the failures of Friendster, and so on. People who wanted their own space were always making their own space, and the rest would go to whichever social network was the most relevant at the time.
Facebook wasn't built on MySpace's failures, they were both operating at the same time. People moved to Facebook because it was more intimate and less commercialized while MySpace turned into a dodgy spam heap.
Half of those you call dead (desktop, ftp, simple html) are things i use all the time :-P. And i'm sure free webspace isn't dead as i used one about 3 years ago which was the first hit on google among many and i doubt all of them vanished so fast.
I miss the times when websites weren't being designed by designers, and even when they were the practice of web-design was in its infancy so the well-trodden tropes of today's internet were not established yet.
It was creatively speaking a very exciting space - there were almost no rules yet, and you could do whatever and people wouldn't be turned off because your site looked weird or unusable, because all sites looked weird and were equally unusable.
There are many upsides to the web we have today: accessibility, ease of use, ease of discovery, rich content, but that creative space has gone and something with that fresh creative scope combined with massive, new audience may not exist again in my lifetime (or even ever - the internet was the first time you could throw up a page and have people all over the World see it - a larger network of eyeballs can never exist).
I agree with everything you said except the part about rich content today. Maybe I'm nostalgic, but I think there's a much lower percentage of that today. Unless celebrity gossip and political self-promotion is considered rich content.
I know exactly what you mean - when I said rich content though I was talking more about images, animations, video, games. In the early web over a 28.8k (or slower!) modem you'd have to make every non-text asset count and compress it (3-phase loading jpegs, anyone?), now you can put 10Mb of javascript executable in there and no one bats an eye.
I misread your comment, of course! Yes, pictures were a calculated luxury. If you asked me to click to download a picture and it wasn't worth the five minute wait, you went on a certain list.
I think back in the day people created websites really for fun and can do whatever they want. Now you are judged against the best websites.
Probably the only remnants of the old web are professors who publish academic websites and are based on content rather than design. I do remember when I was about to graduate there was talk of standardizing all the professor websites. I would agree all websites would need accessibility, mobility etc but like you I miss the old web where you could just explore.
Nice. Captain Marvel's website wouldn't load without enabling javascript for three domains, and then wanted four more after it loaded. It looks retro, but it's the same modern web crap.
Perhaps the novelty of the Internet faded away and access is now viewed in the same light as "dial-tone" was in the decades before--it's just there when you need it.
The web platform evolved into a much more complex space. In the early days (late 90's) you could spend an hour in "HotDog Web Editor" and build a page that looked somewhat close to a mainstream website. Those days are long gone and most people now opt for platforms, templates and/or site builders, which make total business sense and are the right decision for most, but they lack the fun and excitement experienced by the early Internet "webmasters".
As bad as many sites were in the 90s, modern sites somehow manage to be worse. Just yesterday I visited somewhere on my phone and about 2/3rds of the screen was occupied by slide-in banners (top and bottom!) informing and or asking me for some shit I of course didn't read because I went there to see the actual content.
I'd take spinning flaming guitar gifs, gaudy tiled backgrounds, and <blink> over that crap any day.
I think the word you're looking for is "personality". Sure half the sites were blinding and had background music and blinking crap and were pretty terrible, but it perfectly reflected the person that built it, however shoddily.
Now, everything is sterilized. Run through a million filters to be picture-perfect, work on everything beautifully, etc. This is great for the bulk of the web (businesses and web apps and what-not), but personal websites disappearing was a _huge_ blow to the personality of the internet. Heck, MySpace pages were a disaster, letting users run arbitrary HTML and CSS everywhere, and when Facebook came along everyone breathed a collective sigh of relief that _finally_ we'd be free of those horrendous MySpace pages.
Now, looking back, I miss the crap out of those pages. I also ditched social media entirely but I consider everything I just mentioned as playing a large role in that. No charm, no personality. Rigid. Walled gardens. It's like a microcosm of the state of technology in 2019.
And I mentioned personal sites disappearing, and I still stand by it - yes, of course there are blogs and what-not, but they're not the same. They're hosted, fast, efficient, and typically use approved themes. If you want to get down and dirty and actually make your own site, it'd be looked at as a gigantic waste of time. Technology has come too far for it to be practical to build a fancy site out of pure flat HTML files that looks like the worst 90s sites. Why do that, when you could just have an out-of-the-box Wordpress or Tumblr site?
Plus, no one would visit it anyway. If it's not spammed or marketing, no one's visiting anything these days. It's just too saturated.
The average internet user visits a tiny fraction of the number of websites they used to. Typically, aggregators. Reddit and Facebook and whatever. Worse yet, aggregators that pull in only things they agree with, shared by people they agree with. And that is their version of the internet, and not to mention, the world. No dissenting opinions, nothing original. It's depressing and I hate it.
I grew up learning to build websites from copying source code into Notepad and running it locally. It was a hot mess, but so much fun and super rewarding. The 90s web, even early 2000s until ~2006 or so (when I personally consider it went "too mainstream" to put on my hipster hat, and everyone was suddenly on it), I miss it all.
Sorry for the rant but this all hits close to home and, as a web developer who got started in the mid-90s, I'm subjected to the "modern web" for many hours a day, and I'd give anything to see it regress.
>And I mentioned personal sites disappearing, and I still stand by it - yes, of course there are blogs and what-not, but they're not the same. They're hosted, fast, efficient, and typically use approved themes. If you want to get down and dirty and actually make your own site, it'd be looked at as a gigantic waste of time. Technology has come too far for it to be practical to build a fancy site out of pure flat HTML files that looks like the worst 90s sites. Why do that, when you could just have an out-of-the-box Wordpress or Tumblr site?
I still write my website by hand. I use a static site generator to do things like make a blog index and stuff, but I don't use any templates. Everything you see is designed by me.
Note that a lot of people in the 90s used WYSIWYG editors, both Netscape Gold (later Communicator) and Internet Explorer (as of Windows 98) came with WYSIWYG editors (Composer and Frontpage Express respectively). Personally i had my own site since the mid-90s (initially on Geocities, later on Tripod) and always used WYSIWYG editors (initially Netscape Composer, later Frontpage Express and after that Mozilla Composer) and AFAIK most people i know used and most oldschool sites i remember were made using such editors.
TBH i never understood, even at the time, the point of writing static pages by hand (i remember even proud web buttons like "Made in Notepad" :-P) - the main explanation i was given was that the WYSIWYG editors created messy code, but to me that never sounded like a good reason since the entire purpose of such an editor is to not have to bother with the code.
I only started making sites by hand when i got into (classic) ASP (i made single pages before, but mostly for javascript demos or learning HTML and the like) and later PHP. Even then, the first time i used ASP was for a hobby site i worked on with some friends over the net about (point and click) adventure games where we'd put articles and such and the articles themselves were written in Frontpage Express for formatting and such (with the code just copy/pasted into a textarea i had in the "admin" section :-P).
I'm still using Seamonkey Composer and Frontpage Express (well, more the latter than the former since i'm not using Seamonkey much anymore whereas the latter is just a 1.5MB zip file i carry on my external hard disk that works out of the box anywhere) for very simple pages like [0], [1] and [2]. Although my "bigger" sites (that is, sites with more than a couple of pages and a need for a theme) are made using custom generators (usually in Free Pascal or Python).
And FWIW one of my favorite software that isn't developed anymore is Apple's iWeb - i haven't seen a tool as easy to use for making static sites as this one. Sadly Apple cares much less than Microsoft ever did about backwards compatibility and i'm certain my copy of iWeb will stop working soon (and there isn't any Wine equivalent to fall back on whereas in the unlikely even that old Windows programs stop working, i can still use them via Wine) so after Apple abandoned it, i stopped using it myself.
I've been doing some research on videogaming in the 90s and I run across so many dead links to things that even archive.org doesn't have. The thing that is really frustrating is that there were a handful of psuedo-podcasts with interviews from game developers that are all gone because they were posted in real audio.
if I ever needed to show proof of my time there it would only be a Google search away
On that topic, has anyone discovered why Google deep-six'd Usenet archives it acquired with Deja News?
You used to be able to find specific posts from specific posters with by: and other operators. Sometime in the aughts it degraded quickly to the point where I can't find threads from which I have explicit excerpts and full author names.
Does someone high up in Google have an embarrassing usenet history? Did it just fall in disrepair?
I hated Google Groups, the interface seemed so overengineered compared to the simplicity of Usenet. I believe you couldn't even view the (plaintext) posts with Javascript disabled.
I have a suspicion that Stack Overflow's success can be attributed at least somewhat to the experience that was browsing comp.lang.* using Google Groups.
I haven't looked in a while, but for a long time, the single most-voted-for bug in Google's public bug tracker was "give me some API access to the message content in Google Groups."
Ironically, there is already a very simple API mechanism that could have been used to provide exactly all the information people wanted... NNTP. All Google would have to do is provide an NNTP server endpoint to Google Groups (even beyond its Usenet mirror). It's not even that hard to write an NNTP server: it's probably the easiest server to implement of POP3, SMTP, IMAP, and NNTP.
As soon as spam started hitting USENET there really was no way to shut it down. The entire system was held together by "netiqette" and when that died off during the Eternal September, it was only a matter of time before it collapsed.
Because they're Google. One might think that Google would see preserving the world's information as part of their mission. I believe they once said something along those lines. But that's pretty much gone by the wayside. And it's probably a reflection of the way Google is managed that even projects with absolutely trivial costs relating to things like RSS and Usenet just fade away because no one wants to be associated with such non-strategic things. Scholar and Books also pretty much went by the wayside though that wasn't really Google's fault.
I agree they're not really faithful to it.
I mean don't get me wrong. Google Search and Maps are two inventions that are so insanely useful, they may standalone be responsible for decades of faster progress. But I do think that if they were really true to the mission statement, they'd take things such as archival more seriously.
(If you explained the entire internet to an alien, and then told them that Google aren't the ones running the Internet Archive, they'd say "Seriously?")
>(If you explained the entire internet to an alien, and then told them that Google aren't the ones running the Internet Archive, they'd say "Seriously?")
+1000
Mind you. At some level, I'm happier that it's a non-profit pursuing this as its mission rather than Google. But, given the vast amounts of money that Google spends on all sorts of things, I don't really grok the mindset that doesn't really prioritize the preservation of information--and the organization of that information--as at least a sideline.
There are admittedly growing headwinds, especially in the case of Europe, about what information can be preserved, but that hasn't really been a big issue until recently.
It might be that they figure the Internet Archive is doing a great job, and there's no use to do the same job twice. They do contribute, being on of the largest (the largest?) book sponsor.
I suspect that at least part of it is that mirroring copyrighted content is a gray area of law. The fact that the Internet Archive is a non-profit archive may give them some leeway and, in any case, they're a less tempting target than Google would be. Look at the ongoing issues that Google has around news sites for example.
Ideally the Internet Archive stays out of jurisdictions that can force them to remove most types of content. The non-profit aspect won't help them in the least.
Being all over the planet in terms of business, infrastructure and physical presence is where Google acting as archive would fail very badly. They might be the absolute last organization you want serving as that entity.
The IA in theory could operate all of its infrastructure and organization out of a preferential jurisdiction (or a few, so as to have backups in case one favorable location goes bad legally/politically), and archive anything it wants to from around the world while entirely ignoring the local laws from a given place (eg the EU, or China, or Brazil, New Zealand, or Turkey, or wherever).
Though the legal situation is a bit murky even in the US. After all, I can't set up a "Comics Archive" and start populating it with all sorts of copyrighted comic strips and expect not to hear from the publishers. But as a non-profit who isn't making money off the content it mirrors, respects robots.txt even retroactively, and will generally honor takedown requests that are remotely legit, it gets cut a lot of slack that a corporation doing this for profit-making purposes wouldn't.
The short answer is that libraries do not get a magical exemption to make copies of copyrighted works although they have some limited exemptions (that seem to have mostly been written with physical artifacts in mind). For example, a library cannot rip a DVD and make it available to the public with no usage restrictions.
IANAL but there is maybe an argument to be made that the IA can mirror web sites for preservation purposes but then could only make it available to one researcher at a time.
As far as I know, there's no such registry. There are exceptions under Section 108 for institutions that fit a certain definition of library, but from what I can tell as a non-laywer, they don't allow the kind of indiscriminate reproduction that the IA engages on: https://www.law.cornell.edu/uscode/text/17/108
From Wikimopedia: "The Archive is a member of the International Internet Preservation Consortium and was officially designated as a library by the state of California in 2007."
>The non-profit aspect won't help them in the least. //
AIUI one of the tests for Fair Use [in USA] looks at whether use is non-commercial (not the same as non-profit; commercial use can be free-gratis, for example; nor is it a sufficient condition in itself), so it could be a key element in a court decision I feel; what's more important perhaps is that people are less inclined to sue non-profits because of the potential harm to their own public image.
Google are pretty canny, I'd expect them to let IA lead - eg assumed consent with old books - in order to set a non-binding precedent so that they can go to the press should they be challenged and say "well we just followed what the noble souls of IA are doing, and this court decision will harm the IA".
Last I looked, IIRC, Papua New Guinea wasn't signatory to copyright treaties, but I think they were planning on signing. There's probably a country in a similar circumstances that would be a reasonable place for holding a backup archive that includes the stuff less liberal regimes want you to ditch.
Ideally, any archive respects the wishes of copyright holders and we don't need to rely on legislation. I certainly want control over my data, and thankfully most legal systems are on my side. My rights over my data trump other peoples need to preserve absolutely everything, no matter how trivial. Like the collection of personal letters my grandparents wanted destroyed after their death, which did not end up in a library vault and the historical significance of which is lost to time.
I pity future historians who will need to wade through the petabytes of crap like so much landfill because we outsourced curating it to the future. Because just maybe the rubbish I spout on my personal blog will be of interest to future generations (hint, it isn't, and I'll be spinning in my grave from embarrassment if it is). I doubt they will wade through it, since we have the ability to leave future generations actual historic records and not force them to learn about us from fragments decoded by archaeologists.
Checking left and right are two jobs. And how often do you test each of your backups? It might get corrupted at any moment - is perpetual validation the answer?
Sure, why not? I do regular automated validation of my backups against each other using "rsync --checksum --dry-run" and get notified if anything beyond a tiny threshold is out of whack. (The threshold being due to small files updated between the two backup runs)
> Google Search and Maps are two inventions that are so insanely useful, they may standalone be responsible for decades of faster progress.
I don't know about this. Google Search was a great improvement in ranking by relevance, but an important invention itself?
As for maps: as far as I'm aware, Google Maps was "just" a combination of two existing technologies - online maps and car navigation systems. They launched Maps in 2005, even Germany had online maps by 2000. Granted, they didn't look as good as Google's and the UX was inferior, but it wasn't a horses vs cars situation imho.
And the iPhone is "just" a more expensive version of its predecessors with a slightly better touchscreen and UX, and Dropbox is "just" a ftp mount with svn on top of it.
Also cars are just faster horses so that works too :)
Honestly I find it hard to argue that Maps and Search haven't been some of the internet's biggest worldwide productivity boosts.
> ...but the wave of full-screen smartphones had already started.
I’ve heard it argued that someone would have gotten there eventually, but my suspicion is that without a big player committing all their resources to marketing and selling it a similar device would have failed to make headway. Phone manufacturers would never have pushed like Apple did against the headwinds of physical keyboards and flash and operator-managed app distribution (and phone crippling).
But I’ve never heard someone argue that the design was already established and going to become a tidal wave. There were a few devices which vaguely resembled a few superficial elements. What examples can you provide to illustrate a wave was already underway?
The first of its kind was the LG Prada, which came out slightly before the iPhone and sold a million units. The technology had all come together just enough to make devices like this possible, and they were starting out at barely good enough.
Batteries, screens, CPUs, all of those were advancing rapidly whether phones used them or not. And 3G was spreading rapidly. Even if it took two more iterations of moore's law, the market was growing more and more feasible every month. Even half-baked attempts 2-3 years down the line could easily have been more compelling than the original iPhone.
I guess a chunk depends on how critical the operator-independent apps were, but let's not forget that the iPhone was ATT-only for years.
> Maps... had much better scrolling than competitors? Being pretty isn't revolutionary.
It's a combination of being always immediately available and having so much useful information about every place, all presented in a UI that is accessible.
The digital maps we had in our country pale in comparison to current day Google Maps. It even has graphs showing working hours and crowding level and reviews, and picture-perfect 3D simulation. That surely wasn't possible for any GPS or phone maps of that period.
Started by who? Early pre-iPhone Android prototypes were modeled after the BlackBerry and RIM was preaching the need for keyboards long after the iPhone was introduced. Microsoft was also aiming phones at businesses.
That's a gross misrepresentation of what I'm saying.
My point is: the technology was there. Better UX, nicer integration into other services etc, those are valuable and good, and they alone can be enough to win a huge market share, but they are not innovations as in "introducing new concepts". A smartphone capable of using GPS to show your position on a map is great and way more accessible than a Magellen NAV 1000, but it's the GPS itself that is the actual breakthrough, the big innovation.
I agree with you insofar that "inventions" seem to imply that they invented the concept of Search Engines and Mapping Software from scratch, which is obviously not true (both existed for a long time before Google entered the market) but in both cases Google managed to offer a superior service for free (for the end user at least). I wouldn't call them inventions, but they're better implementations of an existing concept.
Compare Google maps to the mapping and GPS services of yore, you'd have to pay a fortune to get the same feature set. I remember when you had to pay to add regions to your GPS and then pay again to update them later.
The word "disruption" is quite a buzzword these days but in this case Google truly disrupted both these markets. Everybody had to play catch-up after that. I remember how all other search engines started copying Google's slick interface when they realized they were losing badly.
I do agree that they were vastly superior, but I believe that the new and disruptive thing that Google brought to the table in those markets was the monetization model. They offer valuable services without any visible price tag because the data those services generate are driving the profits from ads. I don't know if they were the first to do this, but that's an invention in my book..
The big innovation with Google Maps was the interactive zooming/panning interface, and it was a huge improvement over any other mapping site of the time. The popular map service of the time, MapQuest, had clunky scroll and zoom buttons around the sides of the map image, and moving around or zooming would reload the entire page. The GMaps interface wasn't just a marginal improvement over the existing pages, it was a completely new way of doing it that was 100x easier to use.
> To "organize the world's information and make it universally accessible and useful"
They still do a few things like that, very occasionally. When they got rid of Freebase (notice a pattern?) they did preserve that data and it was used to seed Wikidata, which now in turn feeds their "SERP boxes". But I absolutely agree that there's zero focus otherwise on that core enabler of their business-- they're just coasting on their earlier (very substantial) efforts, and managing to stay afloat somehow.
> If you explained the entire internet to an alien, and then told them that Google aren't the ones running the Internet Archive, they'd say "Seriously?"
They are running an internet archive. That's what their cache is. There's more than one internet archive; why would Google necessarily be running all of them?
Yeah, their new motto might more accurately be stated as "to organize the world's information, to feed it into our machine learning models, and to throw away anything we deem unprofitable."
> I mean don't get me wrong. Google Search and Maps are two inventions that are so insanely useful, they may standalone be responsible for decades of faster progress
I'm wondering if I'm the only one finding Google Search lacking lately. Increasingly, its search results are seriously out of date and linking to dead and/or ad-heavy sites. I've come to use DDG and, recently, Startpage.com more than Google Search.
>One might think that Google would see OWNING the world's information as part of their mission
FTFY
But also, I do like the theory that early high level Google hires, across the org likely have embarrassing Usenet histories -- I mean, c'mon - I've bee 'online' since the early 90s...
I've said some cringeworthy stuff: for example - I used to actively participate in the Haiku thread on Craigslist in the early 2000s -- and I thought I was /r/IamReallySmart
It was fun at the time - but I put way to much time into writing long Haikus, and once spent a ton of time attempting to write my masterpiece, a palindromatic senryu...
((HAHAH I just went back to CL and read some of my Haiku I wrote on there... from 2003!))
EDIT: for self deprecation, here is what I wrote on 9/11/2003 ABOUT 9/11/2001:
This makes me think that if any company has a mission to digitize/preserve the world's information, they should now be required to setup a proper non-profit organization/foundation around that. That way this prevents the company from later on exploiting it or killing it off due to a conflict of interest.
> Does someone high up in Google have an embarrassing usenet history?
I don't know if that's true (it probably is), but personally I am glad that almost all of my old Usenet posts have vanished. I was horrified when Deja News started up. That was the moment I realized the internet is forever and decided to never use my real name or to upload any pictures of myself to anything connected to the internet.
Of course, I'm still screwed because I use a smart phone and probably several entities have that data and could connect the dots, but the average person I encounter can find out very little about me with just my name.
Or don't post things that you'd be embarrassed to be associated with? Easy for me to say of course. My digitized articles from an undergrad newspaper at least went through editors. And anything from my BBS days is almost assuredly lost to the ages. But I've always used my True Name and don't have any problem with that.
Well, sure, but I was still pretty immature 25 years ago. And for younger people, the norms are shifting so rapidly that it's not certain that something posted in jest today won't be considered career-ending heresy in five years.
This is the real reason the internet isn't nearly as interesting as it once was. The concept of 'thinking out loud' that I grew up with is nearly dead.
That's certainly fair. TBH I'm glad there isn't a public record of everything I might have written on a chat board or other public forum as a teenager or college student. At a minimum, there would be things that would require explanations along the lines of "Times were different."
That said, I'm generally pro-True Name unless there's some strong reason for anonymity.
True to a degree. There are still some places left. I think it fascinating that people often wish for more content moderation...
I don't know why they don't just go to work or something like that. Anything that provides a rigid cage for anything that could be deemed controversial. But why force it on places that are optional to visit?
I think it fascinating that people often wish for more content moderation...
It's very easy for overmoderation to prevent useful/enjoyable discussion.
It's equally easy for a lack of moderation to prevent useful/enjoyable discussion.
Restaurants are optional places to visit, too, but I'd find it hard to enjoy my meal if people were shouting threats at me or showing me images of child porn.
>Or don't post things that you'd be embarrassed to be associated with?
I can't predict what will be embarrassing or harmful for me to be associated with in ten years. The only way to be (sort of) sure would be to censor everything I say on the internet to be as inoffensive as possible to everyone, which would suck.
> Do you have any opinions that you would be reluctant to express in front of a group of your peers? If the answer is no, you might want to stop and think about that. If everything you believe is something you're supposed to believe, could that possibly be a coincidence? Odds are it isn't. Odds are you just think what you're told[....]
> The most important thing is to be able to think what you want, not to say what you want. And if you feel you have to say everything you think, it may inhibit you from thinking improper thoughts.
Google allowed you to request your posts be deindexed. I helped somebody get some particularly embarrassing messages deindexed. It’s been more than a few years, so I forget the process, but I do remember that verifying their authorship was somewhat cumbersome.
I take the opposite approach: I always use my name on the basis I never need to be worried about being deanonymized. Also keeps me from posting something I might regret.
I take an entirely different approach: My name is basically a globally-unique identifier to me, and most of my aliases are probably easily traced back to me, so I am forced to conduct all of my online business with the threat of being "found out"
I have been thinking about that these past few months and asking myself when and why I started using my real name online even though I was raised in the pseudonym era of the 90's. I think it's around when Skype was introduced and it made sense to use real name for discoverability.
I split the difference.
My name namrog84
Looks like a pseudonym and number from previous era.
However it's my last name backwards. Gorman. And I was born in 84 so it's not just a random throw away. Also namrog without number is often taken.
So i still use it professionally because it's still closely related to my identity. Yet majority of people would never know if they didn't know my name.
> I don't know if that's true (it probably is), but personally I am glad that almost all of my old Usenet posts have vanished.
Same. If I could go back I wouldn't share any details about my life. Best case scenario, nothing happens. But everything can (and will) be used against you by the court of public opinion. The last few years shredded any '90s idealism I had about the internet.
> The internet is forever
That must be scary for upcoming generations. Now many of our early screw ups and thoughts are recorded in some form or another and there's little way to move on. There's no time/distance element that allows you to grow out of whatever you – or others – uploaded. In the worst cases (revenge porn, false accusations, lolcows) you can't put your old life behind at all.
It's a lot better now. In the past we had usenet, public forever. Now our wall gardens give ability to share semi privately and hide most info that hasn't been leaked yet.
I've mentioned on multiple occasions that the current post-Snowden security and privacy movement is creating a serious threat to the Internet history preservation, and to some extents, threat the understanding/insights of the human civilization in the digital age.
My personal interest is Internet culture and communities. And I'm not amused by this comment, let me talk about the problem briefly.
Online communication from 1970 to 1995 was almost completely public, archived indefinitely. You can still read every single comment by every hacker from the late 80s in Usenet archives, sometimes even back to the ARPAnet era. There are a million posts to read and no spam and low-effort posting at all (by modern standards, even many flame wars seem to be high-quality). You can easily lost days, months or even years in the Usenet archive.
Records like these are often the only remaining records of the online communities, a snapshot of great historical and cultural value. To me, even the controversial political flamewars are interesting as they reveal parts of the history I would not know otherwise (I guess if someone rereads Reddit threads about Donald Trump today in 2055, he/she may have a similar feeling).
On the other hand, you also have names, addresses, and even phone numbers of almost everyone posted on Usenet. It was not a big problem when the access to Usenet/Internet was exclusive to members of the academia, and at a time when there was almost no systematic, organized abuses of the personal information. But today's different, we have big and little brothers who have "Collect Them All" as their slogan, and they are actively trying to exploit the information available to the maximum extents.
What is the response then? People (at least many in the hacking community) start to prefer private, semi-private, or in-group communities over public communication, often protected by cryptography. Some people also actively erases/purges their footprints, for example, some would delete every single post when they left a community, no matter how insightful they are, others may even deliberately insert misleading or false information. And we have something roughly similar to Vernor Vinge's True Name (describes an underground hacking community in the cyberspace). Good, now personal privacy and security is more or less protected by using the cryptographic barrier.
But what is it doing then? We are now creating a unprecedented, HUGE GAP of information in history, within our life time, we are now entering a new digital Dark Age where no one has seen before.
Centralized and/or proprietary services often delete information when they go out-of-service, too, so we need to archive them, desperately. You can't imagine how many resources/memories that are extremely valuable to members of some communities exist solely on a single web server/service provider. I remember reading a post from Schneier's blog that says a website contains numerous posts of wine culture were gone forever when the hard drive failed, and one commenter said that he uses w3m/lynx CLI browsers, and records everything he reads to his hard drive so he would never lost a single piece of information he has seen.
Is it an act of little brother surveillance? Arguably, it can be seen as one. But is it justified? I would say yes, and even say we need more people doing this, systematically. Naturally, archive.org was born in this way.
But then it faces the same issue. On one hand, many archived information can be abused, on the other hand, the more archive-refusing people we have, the more damage to historical records is made.
I don't know how to solve this problem.
The only way I can think of, is (1) Cypherpunks were correct. Anonymity is crucial in the information age, and we should have more of it: never use True Name and reveal personal information unless absolutely necessary, use an anonymous network (e.g Tor) if possible, discard identities...
Even if you leave out email, IRC, etc. I'm sure the volume of discussions that took place on commercial online services like Compuserve and the thousands upon thousands of BBS systems far outweighed Usenet--and almost all of that is long gone (for better or worse). Personally I sort of wish I had more archives from my BBS days but so it goes.
> 99.999+% of history has been lost, and that's OK. The modern fetish of archiving everything is not necessary or healthy.
There are people who spend their lives trying to read between the lines of what survived, in order attempt to answer some question that could have been easily answered from some of the lost material.
The "modern fetish" of archiving everything is an attempt to avoid culling material that may later turn out to have been valuable. All but 1% of what's saved will always be worthless, it's just impossible to know for certain which 1% that will turn out to be.
Most communication throughout history has been ephemeral, and lost as soon as the people relevant died without relaying it to someone else.
Consider the 1800s, where much of our understanding of the attitudes of the day comes from newspapers and archived letters. Then consider how many more were discarded once they had served their purpose.
Today it's possible to archive all of that ephemeral information, but it has never been necessary.
Usenet, for example, was thought to be ephemeral because at best you had a few months worth of posts archived on your server and maybe your local machine, so if you said something boneheaded, it was going to naturally fall off the internet sooner rather than later. As it turns out, that was an incorrect view of the world.
Most forums are treated by their users the same way; a place for people to meet and talk about things in quasi-realtime, but not to archive those discussions for all time. Of course, as it turns out, those discussions are archived for all time, or until the forum closes or has a catastrophic data loss (e.g. this one we're on now).
Consider the 1800s, where much of our understanding of the attitudes of the day comes from newspapers and archived letters. Then consider how many more were discarded once they had served their purpose.
Don't you think the world would be a somewhat better place if we had more records, and therefore more understanding, of what people thought in these times?
Also consider the class implications here. Letter-writing (and archiving) was something more often practiced by folks who were "elite" in terms of wealth and/or education. The thoughts and opinions of these classes have disproportionate representation in our understanding, and those of people in lower classes are underrepresented or erased entirely.
Interesting perspective. With regard to the historical value, though, I think that the volume of information cheapens the value of most of it. The kinds of things that move people and events so that future generations can understand what happened are still preserved -- maybe not all the details, but enough. Of course, there could be some great catastrophe that wipes out most of the collective memory, but the broad outlines are still there and will continue to be there in some form, because ultimately the really important memories are preserved in the minds of those who were affected by them and passed along in some form to the next generation.
So, the burning of the library at Alexandria was a catastrophe for our understanding of ancient civilization, but enough was preserved that we still have what is essentially a Greco-Roman understanding and practice of government, philosophy, history, etc.
Closer to our day, I remember things that were told to me about events that happened over 100 years ago by people who lived through them or who knew people who lived through them. For example, my mom has told me about how her Uncle Henry wheezed when she was a kid in the 30s because he was exposed to chemical warfare in the trenches of France during the Great War. My dad's stories of growing up on a farm during the Depression and World War II likewise tie in nicely with things I have read about the larger events at the time; e.g., he once mentioned how they received extra gas ration stamps once they bought a tractor because they were farmers producing food needed for the war effort. (They farmed with horses until 1942.)
So I feel like I have a pretty good understanding of what life was like for my parents and grandparents because of what they told me. Do I have anything like a complete record of it? No, but I don't need it. Do my children need it? Sure, but they don't really understand what actual Nazis were because that word has been misused and misapplied ever since Gore Vidal used that tactic to "win" an argument with William F. Buckley 50 years ago. And I can tell them about that and why it's important, but it doesn't seem to register because they were never taught any details of what happened in the 30s in Europe, and don't really have a sense of what that generation suffered through as a consequence of the toxic ideologies that flourished during that era.
So please forgive me, but while long posts like this on the internet might be interesting to a tiny percentage of us, for most of the people who live in the future they will at the very best be reduced to a one-sentence quote by some future Ken Burns-like documentarian read by some future Morgan Freeman-like actor.
> So, the burning of the library at Alexandria was a catastrophe for our understanding of ancient civilization, but enough was preserved that we still have what is essentially a Greco-Roman understanding and practice of government, philosophy, history, etc.
I think that's a bold statement. This is circular reasoning: since we know resources that we have, obviously we know what they do contain. I remember university lectures on ancient Roman history, conducted by a serious researcher in the field, where he said how our general understanding of Roman political system could be changed if we had a couple inches less (or more) of papyrus of Festus: an author who lived way after the interesting stuff but managed to survive. In fact historical writings that are preserved for us seem to have seriously warped our understanding (in a pro-Senate & optimates, conservative way). Only relatively recently we are trying to correct that by reaching for some more obscure sources and reading more closely. Many important things we will never know.
Obviously people caring about history will be always a small portion of the population, just as with many other pursuits.
I don't know. We've got a lot of the main writers, and we've got references to some of the books that were lost forever in the fire. I agree that there was some valuable information lost in the fire. But in the main, I think the broad outlines were preserved, or we probably would't even have the concept of circular reasoning.
Obviously there was a period where much of that was irrelevant anyway, after the barbarian hordes overran the weak and degenerate remnants of the Roman empire. When the revival came over half a millenium later, did they get it all right? Probably not, but it was a definite change from what existed in the interim. And some of that culture was preserved through the Roman church, though in a muted way.
More to the point of the original discussion, are you really concerned that anything of lasting value would be lost if the entire internet were deleted? Maybe technical information, sure, but culturally not so much imo.
TBH internetization of reality is still very much an ongoing process, likely to expand much further in the future. At least if we want to extrapolate, there might be a backlash, who knows (I can imagine internet gaining a boring, authoritarian image like television around the turn of milennium). "Big" political events had an important online aspect in recent years. New cultural phenomena, art genres etc. emerge here.
I have to admit, imagining nuking the entire internet is funny intellectually. It would leave us with a low connectivity, low-res "slow" version of history, something more resembling "long 1990s" or what a stubborn pre-milennial person may still be experiencing now. Would it be warped beyond recognition? Probably not, not yet. You could still reconstruct Western societies in broad strokes. But I think the cultural history at least would suffer significantly.
Just wanted to say that I agree with your sentiment. It's supremely aggravating to browse eg. Reddit and find some potentially interesting posts wiped out by some memory-erasing bot. And it is amazing how much we managed to preserve from historical periods, despite (as siblings point out) the theoretical ephemerality of information. I think it is important for subsequent generations to be able to study social phenomena of the past, and at least have a chance to be wiser. We could move beyond anecdotes, speculation and quasi-historical garbage, as "decline of America mimics decline of the Roman empire" meme regularly popping out in newspapers and such.
From my experience people are fascinated when they find some saved communication/memorabilia from years back, and the only downside for them is the fear of being judged. The best dream situation would be where you can have changable identities, but there is an incentive to build respect for them over time (like on HN or Reddit), and there is a strong social convention that you could not be in any way harassed for what you said with such identity. You can also change it, like a mask (maybe in some minimal intervals), and it will be respected. Maybe also you can't refer to anyone's extra-online identity, and it concerns also people from the outside unless they're public figures.
Or, that you can freely say anything under your real name under some online circumstances. It would be similar to carnival in traditional culture, with role reversal, rule suspension and all that. Although for this to work it should have an aura of unseriousness and inherent lack of consequence for the normal reality.
I know what total anonymity with no continual identity leads to (vide *chans), but it is not even what we are talking about here. We could do better with some better social conventions, I think. Ie. have good amounts of both freedom and historical preservation.
I'd be really interested to hear some of your techniques for finding these interesting things in the actual archive. How do you find stuff, do you have to know what your looking for and search by exact text match?
Before I left a website I was writing for (in amicable terms), I asked if I could have a db dump of the posts authored by me. It's in my archive now, so that I know I can get back to years of writing even if the website won't be there anymore in the future.
You may or may not have the rights to post it yourself online, but very few pubs are going to have a problem with you keeping your own copies for personal use.
PSA: If you write for pubs, especially ones that have paywalls of some sort, it's definitely worth keeping your own copies of anything you've written. Fortunately I either kept direct copies of most things I wrote (or they were mirrored on a tech news site) at one job over the course of about 8 years or it would all pretty much be gone.
Even if your pieces are still online and/or backed up on the Internet Archive, they can still be very hard to track down in a systematic way.
I have pretty much everything I care about from the past 20 years or so but a lot of it would be effectively gone had I not specifically saved it.
> I have pretty much everything I care about from the past 20 years or so but a lot of it would be effectively gone had I not specifically saved it.
While you have it - for "us" it is "effectively gone" - unless you uploaded it somewhere.
In the future, after you die, it may become "absolutely gone" - as your relatives or whomever likely won't care, and will discard it.
Unless you make some effort to preserve it. Most people don't - they never did; it was always left up to future people to do so.
For instance, my mom and dad saved a ton of old photos and letters between them, many while he was in the service during the Vietnam War. My mom saved them - but not in a great way. She basically put them all in a box, then that inside a trash bag, and we found it in my old childhood home after both she and my had passed away. So I saved them.
But - unless I scan them - they will be gone forever. In a sense, though, that's ok - because they were never meant to be public anyhow.
But for stuff on the internet - it was public; it was most likely meant to be public. But if you save it personally, then the site or whatever goes away, what the public had is gone. Things like that didn't really happen in the past - I mean, they did - but there was always the possibility of recovery in the future.
More than a few times was something thought "lost forever" - only to be recovered; it found hidden away somewhere, or buried in some dusty archive in a back room of a basement storage area at a university or something. Sometimes found inside a wall.
But the digital? That is rare indeed. Even precious artifacts like tapes and manuals from times past get thrown out all the time, usually by relatives of some passed on engineer, often by their widows, as "so much of this old junk".
Occasionally, though, it gets saved (usually by bitsavers) - and if we are really lucky, it gets scanned or pulled from the tapes - old archives of code, sometimes of other stuff...
But we lose more than we save.
I'm just rambling now - this kind of thing, this loss of our computing history in general (it's more than just "the early internet" - we've lost and thrown away so much of our "early computing" history, it's not funny)...
Think about the inventions that have changed humanity in radical ways, that have altered our society greatly: The steam engine, the automobile, the airplane, etc. How many museums and such are there that celebrate, archive, and curate information about them?
The computer?
How many museums are there for it in comparison?
The difference is stark. A thought I just now had is that I wonder if it is like this, because the computer represents such a different "threat" to humans - that is, it's the closest we've come to a representation of "us" - our minds. And - as it becomes more tightly integrated into our society, if not our body collective (and literal) - if we unconsciously seek to marginalize it due to fear in some manner; fear of the other? fear of the artificial? fear of the usurper? Maybe we seek to forget what we have wrought?
>While you have it - for "us" it is "effectively gone" - unless you uploaded it somewhere.
The stuff I have that I considered still relevant and interesting, I put on my website and it is presumably mirrored in The Wayback Machine by now although I have not actually checked.
I agree with your basic points. I have a lot of stuff that's scanned and online but it's a job.
For example, I have a large format book chronicling a year in product development at Data General in the mid-80s. It's really a fascinating snapshot of the time. I'm guessing it's not the only copy still around but it's probably one of a relatively few and should really be online.
but yes, I agree the search results are not nearly as good as they once were. the search engineers who were keen on getting this running years ago have moved on to other things.
I think it's kind of insulting to say that. More realistically, maintaining things in the Google environment has a cost- the products are always changing, so just keeping an existing system running well takes a ton of time, and that time comes at the cost of launching new features. Realistically, usenet search results never represented a huge amount of traffic, and traffic is what gets attention at google.
That reeks of incompetence though. Systems shouldn’t be in so much flux that existing products keep breaking. Poor abstraction and tight coupling.
>and traffic is what gets attention at google.
This is likely what resulted in the huge collapse in engineering quality. People that spew out shit with flashy features for product managers get rewarded and move up while people who stabilize and maintain products are cast aside.
The size of the engineering group at google is more than 10x what it was back in the glory days of stuff like gmail, reader, maps, etc. Google now just makes incremental worse changes to existing products (look how slow maps and gmail are) and kills off things that the terrible engineering org can’t keep up with.
There are definitely great engineers at google, but the ratio of garbage to them has become untenable and Google is well on its path to mediocrity. A great example of this is that you have to do an interview again if you want to work on something actually innovative like self driving cars.
In the early days Google wanted to make information accessible to anyone and that fit right into that strategy and goal.
Unfortunately priorities at Google have changed significantly, and the focus is on things that can help sell more targeted ads at scale. Actually providing value is not that important anymore I'm afraid.
I'm imagining Jason Scott/textfiles on an Indiana Jones-like mission, where he finds a hard drive containing all of this, but then has to run to escape, including running from a boulder and jumping over snakes.
The reality is not far from this - AIUI, the Internet Archive does have a fairly sizable archive of very early Usenet that was extracted from backup tapes stored at a random zoology department somewhere in Canada. So dangerous snakes were very likely involved, at least.
One of the interesting points the story brings up is that you never know what future people will care about. In this case cultural and political discussions are typically more interesting than arcana about bug fixes in long ago systems.
Google used to have a timeline showing key points in Usenet history. The first mention of Madonna. The first reference to AIDS. The first mention of the Gulf war. There were articles in Wired on how they recovered this stuff from long-lost tapes. This was such an important piece of internet history and they just abandoned it.
As I recall it, they tried to incorporate Usenet into Google Groups. It was a mess, and instead of fixing it, they probably discovered that adoption of not-Usenet functions was more used and abandoned it.
This is a feature of development based on sprints vs. a bigger vision. That design/development model has alot going for it, but it only works well when the scope of the system is narrow.
My best guess is the company just doesn't see much value in doing the work. It's a niche community and by definition 20+ year old information. It's important historically maybe but not to Google's business. No one's going to buy or see ads on 1990s rec.arts.sf.tv.babylon5 content.
archive.org has several significant Usenet collections. Henry Spencer's UTZOO collection, stuff from Giganews, and some sort of dump for Google Groups / Deja itself. I don't think anyone's built a useful interface for them yet.
I think that's about right. For better or worse, at least the Google of today does not see itself in a historical archivist role to any significant degree.
In an article by the Atlantic it's stated that the books at least are kept around.
> it’s 50 or 60 petabytes on disk, and the only people who can see it are half a dozen engineers on the project who happen to have access because they’re the ones responsible for locking it up.
They invested non-trivial amounts of money in scanning those books, deleting them would throw that money away. As for Usenet, it's similar: storing the data is cheap, and acquiring it again is probably next to impossible, so unlikely they threw it away unless idk it contains proof that sergey and larry have stolen their ideas from some usenet post or something.
"Does someone high up in Google have an embarrassing usenet history?"
To me it's a lot simpler than that: on Usenet one could find people exchanging opinions about a product while on today's web searches all we find is companies selling that product. Just try searching for anything and see how deep you need to go until you get something that resembles a legit conversation about something rather than people selling it.
It was about monetizing every search results page, and that goal became clear to me when they removed the discussion search option from the search engine; that was the final nail in the coffin for the Internet as we knew it.
The answer is clear: It takes an effort to publish something on the internet. Companies and other organisations disappear and their web presence disappears with them.
It takes an extra effort to create a copy. The dark web - the parts of the web that aren't easily accessible because they consist of databases, paid access areas or perhaps simply facebook posts not published as "public" - remains hidden and inaccessible to preservation efforts.
There's also the problem of consuming old internet content. Do you still have a gopher client? Will old web pages look the same in your browser as they did in 1995? Flash will no longer work (hooray?).
On the other hand books, photographs and magazines stick around until you throw them away and we have public institutions that are dedicated to their preservation.
FWIW old web pages do look the same as they did in 1995, with the exception of the default background color changing from silver to white at some point for some reason. But, at least under Windows, the default fonts, colors, spacing, etc are 99% the same.
Flash also works if you are using Chrome. Sadly, Firefox does not support Flash plugin anymore (and i say sadly because just yesterday i was trying to find some files from an old site that was made using Flash and i had to use Chrome just for that).
Firefox still supports Flash, it just doesn't bundle it, unlike Chrome - you have to install the plugin manually. They're only slowly deprecating it starting on FF 69 (the "always activate" option will disappear, so users will have to click to activate it every time: https://bugzilla.mozilla.org/show_bug.cgi?id=1519434)
This is a big problem. Firefox dropped Gopher and Chrome never supported it. Most Gopher users, presumably, use a Gopher web proxy.
For fun I recently built a Gopher web client, implementing the entire protocol in a single page app (literally a single HTML file, no frameworks or dependencies). It supports most documented file formats including "modern" extras like video and audio.
For archival purposes, we might as well just archive the software that was used to read it. In fact, we already do to some extent - for example, Debian archives package repos for all its releases, so you can still get Potato (2000) up and running.
There's also the question of hardware, but so long as we have at least one up-to-date PC emulator, we have an answer to that. It might become more of a challenge going forward, as mainstream emulators drop BIOS support and other legacy parts of the platform; but even then there's stuff like PCem.
Whenever we get into web nostalgia I've got to post http://wiby.me which is a search engine for live retro websites, a mix of modern sites with very little except text and immaculate sites from the early web that time hasn't changed.
I used to be involved in multiple popular things on the internet and became quite famous, even having my own "fan club". All of that is gone. Googling my old nick name yields 0 results.
Same here. In the late 90's I was involved with some other kids from college creating and maintaining a site called "The Crazy Zone", where I designed some of the first JavaScript games ever made (as far as I know), including a crude breakout clone and a magic 8-Ball. We even had a chatroom applet made in Java!
While not famous myself, I do get a single page of memory lane [1] about the AOL communities (plural) around the SNES game Earthbound. It isn't much, and a VERY BAD EYE SORE, but thinking back about all the stuff I said / posted / got TOS'd... I think I'm fine with that getting lost to the Aether.
Often it's the domain renewal issue. People won't keep paying the domain for an irrelevant website, even if the hosting itself is free. OTOH, why would we want to keep everything that was ever written on the internet, forgetting is a natural process in culture.
One of the brain's most important abilities to maintain sanity is the ability to forget the unimportant and remember the important.
The internet will need that ability as it becomes more intelligent. Having AI reasoning based on vast pools of fake news and other disinformation would not be a good thing.
The web has nothing built-in for archiving and versioning. It's a gaping hole in this technological platform, one that has been noted and criticized for a very long time. The reality of this problem, however, is vehemently denied by the current generation of "technologists". Of course, those are the same people who get six-digit salaries for managing complexity they themselves create - partly through hyper-centralization. Good and resilient archival, on the other hand, necessarily implies some level of decentralization.
I singlehandedly maintain a 14-year-old website that used to be a modestly popular web magazine. It's not very expensive, but it's a pain. DNS system is horrible and it's easy to loose domain names to some nonsense. (I lost one that used to be a free 2nd level domain when it was converted to a paid-for zone. Not a matter of money, just paperwork.) Server management is a time drain. Stuff like adding SSL certificate to a legacy VPS can lead to a cascade of updates and config changes that can take days to make and test.
BTW, everyone sings praises to archive.org (and it's well-deserved), but most people here do not seem to realize that they are also a centralized platform that can collapse and take everything down with them. Who archives the archives, etc. Fortunately, they are not the only one of the sort. Unfortunately, it's all very ad-hoc.
If W3C weren't a bunch of corporate shills, there would be a standard for creating versioned web archives, like, 10 years ago. It's obvious that we need one.
Interesting post. So are you saying that if there was a good standard for versioned web archives, then you could stop maintaining your website and just point people to the archives?
I would still maintain the website, but it would be much easier, because I could lean on archival features when that makes sense, instead of trying to keep everything "stable" manually.
Nobody could have predicted the global growth of internet users and the sheer quantities of data being created on a per second scale. Exabytes when? And then what?
There were plenty of people predicting it, pointing out its deficiencies and explaining what needs to be done. In terms of very high level ideas, Alan Kay comes to mind.
Ted Nelson was complaining constantly about the deficiencies. Pretty much nailed the issues too. Unfortunately his solutions were difficult to implement.
I'm interested in IPFS for allowing sites to be archivable. If a site I was interested in was hosted with IPFS, then I could mirror their content and help serve them their content on IPFS. If the original host goes down, I'll still be able to help host the content at its original URL, and the URL will still work for anyone else in the world who tries to follow it. And then maybe people will re-host my own content in the same way, even to long after I'm gone if my content is good enough.
I looked into this awhile ago and came to a similar conclusion about the web. Here's the examples of pre 1996 websites I know of (some are recreations):
I got sued over copyright infringement a few years back, and most of the evidence against my company consisted of screenshots of my sites/blogs from the internet archive. It is a good research tool for "who came first".
It's going to get even worse now that entertainment media is being "streamed" and DRM is preventing users from making copies. Today's game servers and loot boxes are very short lived, and they aren't even selling physical disks for people to preserve and enjoy.
And this is just web. Don’t forget that we also have Apps. Some of the newer startups / media company don’t / choose not to have a web interface. We have archive for web, but not for apps. I looked at the screenshot of the first Uber & Amazon app, and it was amazing how different they look compared to today (2019)
Isn’t the distributed nature of web making this super hard? Everyone can buy a new domain, everyone can structure one’s website differently, everyone can stop publishing content in any date. Innovative ideas about distribution and structure pop out frequently and change how data is served. It’s different from usual publication where publishers are needed to publish content, structure has not changed since the B.C. era when books were invented, and data is physical. We started archiving publications long after they were invented; the web was made only 20 years ago. I’ll take a long time to find a way to reliably save data that is served in a distributed sense, and people will have to study web with ones that survived, just like how we study papyrus or the Rosetta’s stone.
> It’ll take a long time to find a way to reliably save data that is served in a distributed sense
That's actually easy - it's literally what blockchains are for. The whole issue is how to manage the tradeoff between reliably archiving everything (and expecting participants in the network to do the same, so that the system doesn't end up relying on a single point of failure), and delivering/serving the stuff efficiently.
I don't think a long term, major content archival project is a good idea for a blockchain. Since the internet is so large, you would have to have major, major sharding (as opposed to the naive approach in bitcoin where all nodes contain the entire history). And additionally if you archive everything in an immutable datastore, you are going to end up archiving and then hosting illegal content, and afaik it is still not settled whether that constitutes a crime (in my opinion, it should).
IIRC blockchains are for anonymized trust, not efficient archiving, right? I'm not sure if blockchains are going to become super useful when it comes to preserving the internet... :-(
"While digital storage has fallen drastically in price, archiving all this material still costs money. “Who’s going to pay for it?”"
This is really interesting. If you think about it, for all of our print archives, the creator bore some of the cost of archiving it. When you print a book you have to send a copy to the library of congress, so the "storage" is paid for by the publisher. Same with newspaper archives -- usually the newspaper is sent by the publisher, who pays for the "storage" (the paper and ink). I think the UK and France and many other countries have similar systems.
But in the digital age, the entire cost is carried by the archiver.
>When you print a book you have to send a copy to the library of congress
As best I can tell, you only need to send two copies of a book to the Library of Congress if you want to get an LCCN. Otherwise, there's no requirement to do so, certainly not to receive copyright protection.
The cost of printing and shipping the book is only part of the cost. There are additional costs as part of the initial storage, such as the administrative function of issuing the LCCN, cataloging, and shelving the book. Once shelved, the book continues to incur the cost of shelf space in a climate controlled building.
The cost of just keeping a book in a library is great enough that all but the richest academic libraries have an ongoing process of identifying and selling or destroying books with no circulation over a period of time. Usually they do so after checking whether they can borrow a copy from another library in their network should one of their patrons ask for it.
7zip compresses textual content extremely well (and it and other algorithms are always getting better... although I think we're close to the Shannon limit already), and hard drive space is always getting cheaper.
On a more basic level than that, creating a physical book costs money and in turn physical books are much more resilient to time. You don't need to pay for storage, the book will continue to exist unless someone makes an effort to destroy it.
The book still needs to be physically stored in an environment that prevents rapid decay (from humidity, vermin etc). And you do need to pay for that storage.
But, conversely, storage is so much cheaper in the digital age, per amount of data stored, that many things can be archived just for fun. E.g. you can archive a snapshot of Wikipedia in about 80 Gb, and that's with a full-text search index.
My favorite memories of the earlier internet was hijacking Yahoo chat rooms (guns and ammo was a favorite) and singing 80's karaoke while the normal denizens became increasingly irate. Because once you took control of the microphone you got to keep it as long as you held the button down ;)
Now the internet does not seem as fun and wholesome as it did back in the 90's; I feel bad that my kids don't get to enjoy that. On the plus side, maybe they can make some side income as influencers?
351 comments
[ 2.7 ms ] story [ 370 ms ] threadWhen people started making pages for Search Engine instead of Human Consumptions, Internet died. "Light on information and Heavy on keywords and linking" sites and pages flooded the internet, through which finding the actual content, needs another level of skill.
I think solution to this is, an AI based open source browser extension, which find relevant page from search engine result depending upon its usefulness index. ( or just show its usefulness index, along side the result, in short sabotaging the Google search results ;) ). Is there anything similar available ?
More broadly, DDG implements UI features that allow users to clarify their intent, while Google focuses on using machine learning to infer intent.
When things go sideways, this means DDG empowers users, and google overrides them.
Disclaimer: I switched away from google search years ago, and now I find its user interface completely baffling.
Maybe I just don’t know how to use it any more, and there is some override I’ve overlooked.
Also, I’m kind of shocked by how many corporate logos and spam links fly by when I scroll down the Google results for “cloud”. It feels like the 90’s internet before animated gifs and the blink tag were invented.
In fairness to google, ddg has a similar number of logos, but they’re each approximately the size of one normal sized character, not one third the width of the screen like in google.
This is a good idea but not well implemented. For example, searching Python displays a relevant carousel of "meanings" - you have the snake represented, and the programming language. But choosing an item from the list simply changes the search term.
This is particularly bad with "Ruby". Perform a search for Ruby and then click the 'a pink to blood-red colored gemstone' meaning. Observe that the top result now that you have clarified is "Ruby Programming Language".
It’s tricky, though, because I’d say that the clouds in the sky are more relevant to humanity as a whole. Not everyone cares about or needs to think about cloud computing.
It was unlaunched eventually because nobody ever clicked on it. It turns out that disambiguation is a relatively rare feature for people to want when most of the query stream is navigational queries, and it's contrary to Google's goal of getting you to your answer as quick as possible (2 clicks if it were a UI feature vs. one click if the result were just in the results). AFAIK the backend is still live, but it triggers off keywords in the query rather than any UI element. If you search for [cloud weather] 100% of the results are about the meteorological phenomena, if you search for [thrush bird] 100% of the results are about the songbird, if you search for [jaguar car] 100% of the results are about the car, and if you search for [python snake] 100% of the results are about the reptile.
For example, when it comes to firearms, there are two completely different parts called "extractor" and "ejector", that people often confuse because of similar sounding names. So now, when you search for "AR extractor removal", the first hit on Google is about ejectors; and it specifically highlights the word "ejector" as bold, as if it came from the search query:
https://www.google.com/search?q=AR+extractor+removal
I think it comes in waves. There was a surge of TechCrunch articles the other day. I noticed because they're blacklisted in pi-hole due to their crappy UI.
I suppose this is the natural progression as a given thing becomes more accessible, that the things themselves lose personality and flair because literally anybody can do it now. It reminds of cars, actually, how back in the early days every car was unique because when you bought a "car" what you received was a chassis, engine, wheels, etc. and then you'd take that to a coach builder who'd finish the job to your liking. Then as time went on, things got standardized (for cars, it was manufacturers and models, for websites, it's companies like Shopify, Wordpress, Joomla, etc. and then slowly everything gets paired down into what is generally agreed upon as the "correct" set of features, and they all just look a bit the same.
And then of course you occasionally have a teenager who fits a great stereo to a Mazda or something, but by and large, everything is pretty much standardized.
I know it's more efficient, but it's terribly boring.
I'd argue that fewer people can do it now (relative to the Internet population) than previously, because free webspace is dead, the desktop is dead (the laptop close behind it), ftp is dead, DMCA, GDPR, hackers are scary, HTML is much more complex (e.g. responsive design).
The web is a playing field for larger corporations now and most people and small entities have given up and just maintain a FB/IG presence.
New PC sales makes it look like access is dropping, but the reality is people simply use older Computers on average. A 10 year old desktop in 1998 is relatively ancient, but in 2018 it’s not that far behind the curve. On top of that tablets and smartphones often replace secondary computers. If you have a powerful gaming desktop and a tablet then a laptop is less useful.
Another big issue is that 'computer malls' are also still a thing. Think something like Fry's, but 6 stories tall and full of nothing but independent shops. These places drive two confounding issues. The first is that they build [lots] of computers a la carte. PC sales figures only come from major distributors like Dell, HP, etc. So even though there are millions of PCs sold at these malls, it'd count as 0 by most metrics. And another factor is that people tend to repair/refurbish/upgrade their computers rather than buy new ones. These malls again facilitate such things at very affordable prices.
I can count the number of tablets I've seen on one hand.
> It has something like 40% of web traffic now
I'm not sure what your metric is here so I've assumed volume.
40% of web traffic now is an order of magnitude more traffic than all web traffic a decade ago. I'm not sure how something which has an increased volume and still has 40% of the total pie can be 'dead'. I might buy 'dying' if it was accompanied with more data.
> ...despite those with desktop PCs being more likely professionals in the field...
I'm not sure what you are saying. It seems like you're saying outside of tech, businesses don't use desktops. That does not jibe with my experience. I have seen an increase in POS systems using tablets, but those are primarily replacing cash registers, bespoke POS systems, etc.
> ...so normal people simply don't really use a desktop/laptop anymore to browse the web.
With the exception of the people I know who own a laptop but not a television, it seems that all normal people do with their laptops is browse the web.
https://www.theverge.com/2018/7/13/17567760/pc-sales-growth-...
I actually travel on business with one a lot of the time but will probably replace it because no one really makes the hardware I want.
So what? A browser is all you need to make a website.
When people say the desktop/laptop is largely dead. That's what they mean. [ADDED: That, and that web access is increasingly through a phone.]
https://www.cnet.com/news/oracle-to-debut-500-network-comput...
I'd also rather have an iPhone helping me out in the workshop than a laptop or desktop, for a bunch of reasons. I can use it as a measuring device, a level, a note-taking/shopping-list machine (hands free, even!), a youtube how-to player, an instruction manual finder, and so on, and it fits in my pocket and reasonable amounts of sawdust and liquid won't hurt it. That's creation.
People spend most of their time on "real" computers consuming, too, by a huge margin. I still wouldn't say their (computers') primary focus is consumption. Anyway, the premise is kind of nonsense as consumption is a major part of many creative processes.
Making silly little programs in BASIC or Perl is a lot tougher on an iphone, and that's what sparked the interest for a lot of people (though maybe I'm wrong and it's getting easier?)
Nope, it's better than ever. You have non-profit providers like https://neocities.org/, regular shared hosting with free plans, and stuff like Github pages and Netlify for those with a bit more tech skills.
Way better than the times when you had Geocities and such injecting ads into your site.
> the desktop is dead (the laptop close behind it)
No need for those to build a website. Any cheap Android phone can do it.
> ftp is dead
Nope, the previously mentioned shared hosting providers still offer it.
> HTML is much more complex (e.g. responsive design)
Old-school HTML pretty much works the same, and is actually quite fine on both desktop and mobile screens.
--
Frankly, I don't think the problem is that making your own site has gotten any harder, it's that there was no FB/IG back then. Had there been FB pages from the start of the web, you'd probably would have never seen so many custom sites in the first place.
I’m not sure about this. AOL, Prodigy and CompuServe all had vibrant online communities. However, what you could do was much more limited than the federated web, and they were silos.
Without the competition of “I’ll host my own server, and search engines will index it”, I wonder if FB and it’s predecessors would have been able to figure out what to build.
It was creatively speaking a very exciting space - there were almost no rules yet, and you could do whatever and people wouldn't be turned off because your site looked weird or unusable, because all sites looked weird and were equally unusable.
There are many upsides to the web we have today: accessibility, ease of use, ease of discovery, rich content, but that creative space has gone and something with that fresh creative scope combined with massive, new audience may not exist again in my lifetime (or even ever - the internet was the first time you could throw up a page and have people all over the World see it - a larger network of eyeballs can never exist).
Probably the only remnants of the old web are professors who publish academic websites and are based on content rather than design. I do remember when I was about to graduate there was talk of standardizing all the professor websites. I would agree all websites would need accessibility, mobility etc but like you I miss the old web where you could just explore.
Having said that, the new Captain Marvel website is awesome. I don't think the page counter is real though :( https://www.marvel.com/captainmarvel/
http://runtimeterror.com/games/dino/
(and i think the main runtimeterror.com site has a bit of a late 90s/early 2000s style, but it is just that i prefer it that way)
The web platform evolved into a much more complex space. In the early days (late 90's) you could spend an hour in "HotDog Web Editor" and build a page that looked somewhat close to a mainstream website. Those days are long gone and most people now opt for platforms, templates and/or site builders, which make total business sense and are the right decision for most, but they lack the fun and excitement experienced by the early Internet "webmasters".
I'd take spinning flaming guitar gifs, gaudy tiled backgrounds, and <blink> over that crap any day.
Now, everything is sterilized. Run through a million filters to be picture-perfect, work on everything beautifully, etc. This is great for the bulk of the web (businesses and web apps and what-not), but personal websites disappearing was a _huge_ blow to the personality of the internet. Heck, MySpace pages were a disaster, letting users run arbitrary HTML and CSS everywhere, and when Facebook came along everyone breathed a collective sigh of relief that _finally_ we'd be free of those horrendous MySpace pages.
Now, looking back, I miss the crap out of those pages. I also ditched social media entirely but I consider everything I just mentioned as playing a large role in that. No charm, no personality. Rigid. Walled gardens. It's like a microcosm of the state of technology in 2019.
And I mentioned personal sites disappearing, and I still stand by it - yes, of course there are blogs and what-not, but they're not the same. They're hosted, fast, efficient, and typically use approved themes. If you want to get down and dirty and actually make your own site, it'd be looked at as a gigantic waste of time. Technology has come too far for it to be practical to build a fancy site out of pure flat HTML files that looks like the worst 90s sites. Why do that, when you could just have an out-of-the-box Wordpress or Tumblr site?
Plus, no one would visit it anyway. If it's not spammed or marketing, no one's visiting anything these days. It's just too saturated.
The average internet user visits a tiny fraction of the number of websites they used to. Typically, aggregators. Reddit and Facebook and whatever. Worse yet, aggregators that pull in only things they agree with, shared by people they agree with. And that is their version of the internet, and not to mention, the world. No dissenting opinions, nothing original. It's depressing and I hate it.
I grew up learning to build websites from copying source code into Notepad and running it locally. It was a hot mess, but so much fun and super rewarding. The 90s web, even early 2000s until ~2006 or so (when I personally consider it went "too mainstream" to put on my hipster hat, and everyone was suddenly on it), I miss it all.
Sorry for the rant but this all hits close to home and, as a web developer who got started in the mid-90s, I'm subjected to the "modern web" for many hours a day, and I'd give anything to see it regress.
I still write my website by hand. I use a static site generator to do things like make a blog index and stuff, but I don't use any templates. Everything you see is designed by me.
http://www.jdpressman.com/
TBH i never understood, even at the time, the point of writing static pages by hand (i remember even proud web buttons like "Made in Notepad" :-P) - the main explanation i was given was that the WYSIWYG editors created messy code, but to me that never sounded like a good reason since the entire purpose of such an editor is to not have to bother with the code.
I only started making sites by hand when i got into (classic) ASP (i made single pages before, but mostly for javascript demos or learning HTML and the like) and later PHP. Even then, the first time i used ASP was for a hobby site i worked on with some friends over the net about (point and click) adventure games where we'd put articles and such and the articles themselves were written in Frontpage Express for formatting and such (with the code just copy/pasted into a textarea i had in the "admin" section :-P).
I'm still using Seamonkey Composer and Frontpage Express (well, more the latter than the former since i'm not using Seamonkey much anymore whereas the latter is just a 1.5MB zip file i carry on my external hard disk that works out of the box anywhere) for very simple pages like [0], [1] and [2]. Although my "bigger" sites (that is, sites with more than a couple of pages and a need for a theme) are made using custom generators (usually in Free Pascal or Python).
And FWIW one of my favorite software that isn't developed anymore is Apple's iWeb - i haven't seen a tool as easy to use for making static sites as this one. Sadly Apple cares much less than Microsoft ever did about backwards compatibility and i'm certain my copy of iWeb will stop working soon (and there isn't any Wine equivalent to fall back on whereas in the unlikely even that old Windows programs stop working, i can still use them via Wine) so after Apple abandoned it, i stopped using it myself.
[0] http://runtimeterror.com/tech/jtf/
[1] http://runtimeterror.com/tools/gopher/
[2] http://runtimeterror.com/tools/ol/
Blink is easily implemented in pure CSS these days if you still want it. You can even customize its behavior!
I used to have a chrome and Firefox JavaScript bookmarklet to do it in one click but that doesn't work anymore.
The url is obviously http://web.archive.org/save/https://<url-to-be-saved>
On that topic, has anyone discovered why Google deep-six'd Usenet archives it acquired with Deja News?
You used to be able to find specific posts from specific posters with by: and other operators. Sometime in the aughts it degraded quickly to the point where I can't find threads from which I have explicit excerpts and full author names.
Does someone high up in Google have an embarrassing usenet history? Did it just fall in disrepair?
I have a suspicion that Stack Overflow's success can be attributed at least somewhat to the experience that was browsing comp.lang.* using Google Groups.
Ironically, there is already a very simple API mechanism that could have been used to provide exactly all the information people wanted... NNTP. All Google would have to do is provide an NNTP server endpoint to Google Groups (even beyond its Usenet mirror). It's not even that hard to write an NNTP server: it's probably the easiest server to implement of POP3, SMTP, IMAP, and NNTP.
https://www.google.com/search/howsearchworks/mission/
I agree they're not really faithful to it. I mean don't get me wrong. Google Search and Maps are two inventions that are so insanely useful, they may standalone be responsible for decades of faster progress. But I do think that if they were really true to the mission statement, they'd take things such as archival more seriously.
(If you explained the entire internet to an alien, and then told them that Google aren't the ones running the Internet Archive, they'd say "Seriously?")
+1000
Mind you. At some level, I'm happier that it's a non-profit pursuing this as its mission rather than Google. But, given the vast amounts of money that Google spends on all sorts of things, I don't really grok the mindset that doesn't really prioritize the preservation of information--and the organization of that information--as at least a sideline.
There are admittedly growing headwinds, especially in the case of Europe, about what information can be preserved, but that hasn't really been a big issue until recently.
Being all over the planet in terms of business, infrastructure and physical presence is where Google acting as archive would fail very badly. They might be the absolute last organization you want serving as that entity.
The IA in theory could operate all of its infrastructure and organization out of a preferential jurisdiction (or a few, so as to have backups in case one favorable location goes bad legally/politically), and archive anything it wants to from around the world while entirely ignoring the local laws from a given place (eg the EU, or China, or Brazil, New Zealand, or Turkey, or wherever).
I don't think it's all that murky.
It's my understanding that IA is allowed to have all that copyrighted stuff because it took the effort to legally register as a real library.
IANAL but there is maybe an argument to be made that the IA can mirror web sites for preservation purposes but then could only make it available to one researcher at a time.
Related newspaper article: http://old.post-gazette.com/pg/07175/796164-96.stm
AIUI one of the tests for Fair Use [in USA] looks at whether use is non-commercial (not the same as non-profit; commercial use can be free-gratis, for example; nor is it a sufficient condition in itself), so it could be a key element in a court decision I feel; what's more important perhaps is that people are less inclined to sue non-profits because of the potential harm to their own public image.
Google are pretty canny, I'd expect them to let IA lead - eg assumed consent with old books - in order to set a non-binding precedent so that they can go to the press should they be challenged and say "well we just followed what the noble souls of IA are doing, and this court decision will harm the IA".
Last I looked, IIRC, Papua New Guinea wasn't signatory to copyright treaties, but I think they were planning on signing. There's probably a country in a similar circumstances that would be a reasonable place for holding a backup archive that includes the stuff less liberal regimes want you to ditch.
I pity future historians who will need to wade through the petabytes of crap like so much landfill because we outsourced curating it to the future. Because just maybe the rubbish I spout on my personal blog will be of interest to future generations (hint, it isn't, and I'll be spinning in my grave from embarrassment if it is). I doubt they will wade through it, since we have the ability to leave future generations actual historic records and not force them to learn about us from fragments decoded by archaeologists.
And why bother looking both ways before crossing the street, or even testing your backups?
I think you're being downvoted because many folks are touchy about the word censor being applied to non-government organizations.
OTOH, when is it that an organization becomes the defacto governing body?
...because I hope that's the most uncharitable interpretation of someone else's words that I read today.
I don't know about this. Google Search was a great improvement in ranking by relevance, but an important invention itself? As for maps: as far as I'm aware, Google Maps was "just" a combination of two existing technologies - online maps and car navigation systems. They launched Maps in 2005, even Germany had online maps by 2000. Granted, they didn't look as good as Google's and the UX was inferior, but it wasn't a horses vs cars situation imho.
Also cars are just faster horses so that works too :)
Honestly I find it hard to argue that Maps and Search haven't been some of the internet's biggest worldwide productivity boosts.
Without the iPhone it may have taken another year or two but the wave of full-screen smartphones had already started.
For Dropbox I'm unsure but they definitely have a lot of competitors doing the same thing at this point.
Maps... had much better scrolling than competitors? Being pretty isn't revolutionary.
Google search itself might qualify.
Cars are not faster horses, but if you removed any particular car company from history we would still have cars.
I’ve heard it argued that someone would have gotten there eventually, but my suspicion is that without a big player committing all their resources to marketing and selling it a similar device would have failed to make headway. Phone manufacturers would never have pushed like Apple did against the headwinds of physical keyboards and flash and operator-managed app distribution (and phone crippling).
But I’ve never heard someone argue that the design was already established and going to become a tidal wave. There were a few devices which vaguely resembled a few superficial elements. What examples can you provide to illustrate a wave was already underway?
Batteries, screens, CPUs, all of those were advancing rapidly whether phones used them or not. And 3G was spreading rapidly. Even if it took two more iterations of moore's law, the market was growing more and more feasible every month. Even half-baked attempts 2-3 years down the line could easily have been more compelling than the original iPhone.
I guess a chunk depends on how critical the operator-independent apps were, but let's not forget that the iPhone was ATT-only for years.
LG would have never built an entire ecosystem.
Apple was AT&T only in the US.
It's a combination of being always immediately available and having so much useful information about every place, all presented in a UI that is accessible.
The digital maps we had in our country pale in comparison to current day Google Maps. It even has graphs showing working hours and crowding level and reviews, and picture-perfect 3D simulation. That surely wasn't possible for any GPS or phone maps of that period.
My point is: the technology was there. Better UX, nicer integration into other services etc, those are valuable and good, and they alone can be enough to win a huge market share, but they are not innovations as in "introducing new concepts". A smartphone capable of using GPS to show your position on a map is great and way more accessible than a Magellen NAV 1000, but it's the GPS itself that is the actual breakthrough, the big innovation.
Compare Google maps to the mapping and GPS services of yore, you'd have to pay a fortune to get the same feature set. I remember when you had to pay to add regions to your GPS and then pay again to update them later.
The word "disruption" is quite a buzzword these days but in this case Google truly disrupted both these markets. Everybody had to play catch-up after that. I remember how all other search engines started copying Google's slick interface when they realized they were losing badly.
They still do a few things like that, very occasionally. When they got rid of Freebase (notice a pattern?) they did preserve that data and it was used to seed Wikidata, which now in turn feeds their "SERP boxes". But I absolutely agree that there's zero focus otherwise on that core enabler of their business-- they're just coasting on their earlier (very substantial) efforts, and managing to stay afloat somehow.
They are running an internet archive. That's what their cache is. There's more than one internet archive; why would Google necessarily be running all of them?
I'm wondering if I'm the only one finding Google Search lacking lately. Increasingly, its search results are seriously out of date and linking to dead and/or ad-heavy sites. I've come to use DDG and, recently, Startpage.com more than Google Search.
Why would one might think that? It seems obvious that their objective is to control information at best.
FTFY
But also, I do like the theory that early high level Google hires, across the org likely have embarrassing Usenet histories -- I mean, c'mon - I've bee 'online' since the early 90s...
I've said some cringeworthy stuff: for example - I used to actively participate in the Haiku thread on Craigslist in the early 2000s -- and I thought I was /r/IamReallySmart
It was fun at the time - but I put way to much time into writing long Haikus, and once spent a ton of time attempting to write my masterpiece, a palindromatic senryu...
((HAHAH I just went back to CL and read some of my Haiku I wrote on there... from 2003!))
EDIT: for self deprecation, here is what I wrote on 9/11/2003 ABOUT 9/11/2001:
https://forums.craigslist.org/?ID=8734800
I don't know if that's true (it probably is), but personally I am glad that almost all of my old Usenet posts have vanished. I was horrified when Deja News started up. That was the moment I realized the internet is forever and decided to never use my real name or to upload any pictures of myself to anything connected to the internet.
Of course, I'm still screwed because I use a smart phone and probably several entities have that data and could connect the dots, but the average person I encounter can find out very little about me with just my name.
This is the real reason the internet isn't nearly as interesting as it once was. The concept of 'thinking out loud' that I grew up with is nearly dead.
That said, I'm generally pro-True Name unless there's some strong reason for anonymity.
I don't know why they don't just go to work or something like that. Anything that provides a rigid cage for anything that could be deemed controversial. But why force it on places that are optional to visit?
It's equally easy for a lack of moderation to prevent useful/enjoyable discussion.
Restaurants are optional places to visit, too, but I'd find it hard to enjoy my meal if people were shouting threats at me or showing me images of child porn.
I can't predict what will be embarrassing or harmful for me to be associated with in ten years. The only way to be (sort of) sure would be to censor everything I say on the internet to be as inoffensive as possible to everyone, which would suck.
> Do you have any opinions that you would be reluctant to express in front of a group of your peers? If the answer is no, you might want to stop and think about that. If everything you believe is something you're supposed to believe, could that possibly be a coincidence? Odds are it isn't. Odds are you just think what you're told[....]
> The most important thing is to be able to think what you want, not to say what you want. And if you feel you have to say everything you think, it may inhibit you from thinking improper thoughts.
iOS has joined the room
However it's my last name backwards. Gorman. And I was born in 84 so it's not just a random throw away. Also namrog without number is often taken.
So i still use it professionally because it's still closely related to my identity. Yet majority of people would never know if they didn't know my name.
Same. If I could go back I wouldn't share any details about my life. Best case scenario, nothing happens. But everything can (and will) be used against you by the court of public opinion. The last few years shredded any '90s idealism I had about the internet.
> The internet is forever
That must be scary for upcoming generations. Now many of our early screw ups and thoughts are recorded in some form or another and there's little way to move on. There's no time/distance element that allows you to grow out of whatever you – or others – uploaded. In the worst cases (revenge porn, false accusations, lolcows) you can't put your old life behind at all.
Not on long time scale, I think. Services will disappear, disks will be reformatted, bit-rot will erode archives.
Repeat after me: "no I did not make those posts"
I've mentioned on multiple occasions that the current post-Snowden security and privacy movement is creating a serious threat to the Internet history preservation, and to some extents, threat the understanding/insights of the human civilization in the digital age.
My personal interest is Internet culture and communities. And I'm not amused by this comment, let me talk about the problem briefly.
Online communication from 1970 to 1995 was almost completely public, archived indefinitely. You can still read every single comment by every hacker from the late 80s in Usenet archives, sometimes even back to the ARPAnet era. There are a million posts to read and no spam and low-effort posting at all (by modern standards, even many flame wars seem to be high-quality). You can easily lost days, months or even years in the Usenet archive.
Records like these are often the only remaining records of the online communities, a snapshot of great historical and cultural value. To me, even the controversial political flamewars are interesting as they reveal parts of the history I would not know otherwise (I guess if someone rereads Reddit threads about Donald Trump today in 2055, he/she may have a similar feeling).
On the other hand, you also have names, addresses, and even phone numbers of almost everyone posted on Usenet. It was not a big problem when the access to Usenet/Internet was exclusive to members of the academia, and at a time when there was almost no systematic, organized abuses of the personal information. But today's different, we have big and little brothers who have "Collect Them All" as their slogan, and they are actively trying to exploit the information available to the maximum extents.
What is the response then? People (at least many in the hacking community) start to prefer private, semi-private, or in-group communities over public communication, often protected by cryptography. Some people also actively erases/purges their footprints, for example, some would delete every single post when they left a community, no matter how insightful they are, others may even deliberately insert misleading or false information. And we have something roughly similar to Vernor Vinge's True Name (describes an underground hacking community in the cyberspace). Good, now personal privacy and security is more or less protected by using the cryptographic barrier.
But what is it doing then? We are now creating a unprecedented, HUGE GAP of information in history, within our life time, we are now entering a new digital Dark Age where no one has seen before.
Centralized and/or proprietary services often delete information when they go out-of-service, too, so we need to archive them, desperately. You can't imagine how many resources/memories that are extremely valuable to members of some communities exist solely on a single web server/service provider. I remember reading a post from Schneier's blog that says a website contains numerous posts of wine culture were gone forever when the hard drive failed, and one commenter said that he uses w3m/lynx CLI browsers, and records everything he reads to his hard drive so he would never lost a single piece of information he has seen.
Is it an act of little brother surveillance? Arguably, it can be seen as one. But is it justified? I would say yes, and even say we need more people doing this, systematically. Naturally, archive.org was born in this way.
But then it faces the same issue. On one hand, many archived information can be abused, on the other hand, the more archive-refusing people we have, the more damage to historical records is made.
I don't know how to solve this problem.
The only way I can think of, is (1) Cypherpunks were correct. Anonymity is crucial in the information age, and we should have more of it: never use True Name and reveal personal information unless absolutely necessary, use an anonymous network (e.g Tor) if possible, discard identities...
Only if one thinks that Usenet is all that there was. It wasn't.
There are people who spend their lives trying to read between the lines of what survived, in order attempt to answer some question that could have been easily answered from some of the lost material.
The "modern fetish" of archiving everything is an attempt to avoid culling material that may later turn out to have been valuable. All but 1% of what's saved will always be worthless, it's just impossible to know for certain which 1% that will turn out to be.
Consider the 1800s, where much of our understanding of the attitudes of the day comes from newspapers and archived letters. Then consider how many more were discarded once they had served their purpose.
Today it's possible to archive all of that ephemeral information, but it has never been necessary.
Usenet, for example, was thought to be ephemeral because at best you had a few months worth of posts archived on your server and maybe your local machine, so if you said something boneheaded, it was going to naturally fall off the internet sooner rather than later. As it turns out, that was an incorrect view of the world.
Most forums are treated by their users the same way; a place for people to meet and talk about things in quasi-realtime, but not to archive those discussions for all time. Of course, as it turns out, those discussions are archived for all time, or until the forum closes or has a catastrophic data loss (e.g. this one we're on now).
Also consider the class implications here. Letter-writing (and archiving) was something more often practiced by folks who were "elite" in terms of wealth and/or education. The thoughts and opinions of these classes have disproportionate representation in our understanding, and those of people in lower classes are underrepresented or erased entirely.
So, the burning of the library at Alexandria was a catastrophe for our understanding of ancient civilization, but enough was preserved that we still have what is essentially a Greco-Roman understanding and practice of government, philosophy, history, etc.
Closer to our day, I remember things that were told to me about events that happened over 100 years ago by people who lived through them or who knew people who lived through them. For example, my mom has told me about how her Uncle Henry wheezed when she was a kid in the 30s because he was exposed to chemical warfare in the trenches of France during the Great War. My dad's stories of growing up on a farm during the Depression and World War II likewise tie in nicely with things I have read about the larger events at the time; e.g., he once mentioned how they received extra gas ration stamps once they bought a tractor because they were farmers producing food needed for the war effort. (They farmed with horses until 1942.)
So I feel like I have a pretty good understanding of what life was like for my parents and grandparents because of what they told me. Do I have anything like a complete record of it? No, but I don't need it. Do my children need it? Sure, but they don't really understand what actual Nazis were because that word has been misused and misapplied ever since Gore Vidal used that tactic to "win" an argument with William F. Buckley 50 years ago. And I can tell them about that and why it's important, but it doesn't seem to register because they were never taught any details of what happened in the 30s in Europe, and don't really have a sense of what that generation suffered through as a consequence of the toxic ideologies that flourished during that era.
So please forgive me, but while long posts like this on the internet might be interesting to a tiny percentage of us, for most of the people who live in the future they will at the very best be reduced to a one-sentence quote by some future Ken Burns-like documentarian read by some future Morgan Freeman-like actor.
I think that's a bold statement. This is circular reasoning: since we know resources that we have, obviously we know what they do contain. I remember university lectures on ancient Roman history, conducted by a serious researcher in the field, where he said how our general understanding of Roman political system could be changed if we had a couple inches less (or more) of papyrus of Festus: an author who lived way after the interesting stuff but managed to survive. In fact historical writings that are preserved for us seem to have seriously warped our understanding (in a pro-Senate & optimates, conservative way). Only relatively recently we are trying to correct that by reaching for some more obscure sources and reading more closely. Many important things we will never know.
Obviously people caring about history will be always a small portion of the population, just as with many other pursuits.
Obviously there was a period where much of that was irrelevant anyway, after the barbarian hordes overran the weak and degenerate remnants of the Roman empire. When the revival came over half a millenium later, did they get it all right? Probably not, but it was a definite change from what existed in the interim. And some of that culture was preserved through the Roman church, though in a muted way.
More to the point of the original discussion, are you really concerned that anything of lasting value would be lost if the entire internet were deleted? Maybe technical information, sure, but culturally not so much imo.
I have to admit, imagining nuking the entire internet is funny intellectually. It would leave us with a low connectivity, low-res "slow" version of history, something more resembling "long 1990s" or what a stubborn pre-milennial person may still be experiencing now. Would it be warped beyond recognition? Probably not, not yet. You could still reconstruct Western societies in broad strokes. But I think the cultural history at least would suffer significantly.
A global sense of US. That is what it was. People becoming aware, the world smaller...
I do not know the answer either, other than I too oppose Real Names type efforts.
From my experience people are fascinated when they find some saved communication/memorabilia from years back, and the only downside for them is the fear of being judged. The best dream situation would be where you can have changable identities, but there is an incentive to build respect for them over time (like on HN or Reddit), and there is a strong social convention that you could not be in any way harassed for what you said with such identity. You can also change it, like a mask (maybe in some minimal intervals), and it will be respected. Maybe also you can't refer to anyone's extra-online identity, and it concerns also people from the outside unless they're public figures.
Or, that you can freely say anything under your real name under some online circumstances. It would be similar to carnival in traditional culture, with role reversal, rule suspension and all that. Although for this to work it should have an aura of unseriousness and inherent lack of consequence for the normal reality.
I know what total anonymity with no continual identity leads to (vide *chans), but it is not even what we are talking about here. We could do better with some better social conventions, I think. Ie. have good amounts of both freedom and historical preservation.
https://groups.google.com/forum/#!forum/comp.os.linux
Even if your pieces are still online and/or backed up on the Internet Archive, they can still be very hard to track down in a systematic way.
I have pretty much everything I care about from the past 20 years or so but a lot of it would be effectively gone had I not specifically saved it.
While you have it - for "us" it is "effectively gone" - unless you uploaded it somewhere.
In the future, after you die, it may become "absolutely gone" - as your relatives or whomever likely won't care, and will discard it.
Unless you make some effort to preserve it. Most people don't - they never did; it was always left up to future people to do so.
For instance, my mom and dad saved a ton of old photos and letters between them, many while he was in the service during the Vietnam War. My mom saved them - but not in a great way. She basically put them all in a box, then that inside a trash bag, and we found it in my old childhood home after both she and my had passed away. So I saved them.
But - unless I scan them - they will be gone forever. In a sense, though, that's ok - because they were never meant to be public anyhow.
But for stuff on the internet - it was public; it was most likely meant to be public. But if you save it personally, then the site or whatever goes away, what the public had is gone. Things like that didn't really happen in the past - I mean, they did - but there was always the possibility of recovery in the future.
More than a few times was something thought "lost forever" - only to be recovered; it found hidden away somewhere, or buried in some dusty archive in a back room of a basement storage area at a university or something. Sometimes found inside a wall.
But the digital? That is rare indeed. Even precious artifacts like tapes and manuals from times past get thrown out all the time, usually by relatives of some passed on engineer, often by their widows, as "so much of this old junk".
Occasionally, though, it gets saved (usually by bitsavers) - and if we are really lucky, it gets scanned or pulled from the tapes - old archives of code, sometimes of other stuff...
But we lose more than we save.
I'm just rambling now - this kind of thing, this loss of our computing history in general (it's more than just "the early internet" - we've lost and thrown away so much of our "early computing" history, it's not funny)...
Think about the inventions that have changed humanity in radical ways, that have altered our society greatly: The steam engine, the automobile, the airplane, etc. How many museums and such are there that celebrate, archive, and curate information about them?
The computer?
How many museums are there for it in comparison?
The difference is stark. A thought I just now had is that I wonder if it is like this, because the computer represents such a different "threat" to humans - that is, it's the closest we've come to a representation of "us" - our minds. And - as it becomes more tightly integrated into our society, if not our body collective (and literal) - if we unconsciously seek to marginalize it due to fear in some manner; fear of the other? fear of the artificial? fear of the usurper? Maybe we seek to forget what we have wrought?
The stuff I have that I considered still relevant and interesting, I put on my website and it is presumably mirrored in The Wayback Machine by now although I have not actually checked.
I agree with your basic points. I have a lot of stuff that's scanned and online but it's a job.
For example, I have a large format book chronicling a year in product development at Data General in the mid-80s. It's really a fascinating snapshot of the time. I'm guessing it's not the only copy still around but it's probably one of a relatively few and should really be online.
and other queries for my name return usenet posts in Google Groups: https://groups.google.com/d/msg/comp.os.linux.development.ap...
but yes, I agree the search results are not nearly as good as they once were. the search engineers who were keen on getting this running years ago have moved on to other things.
Is google just hiring so many incompetent engineers that they wreck any project made 5+ years ago?
>and traffic is what gets attention at google.
This is likely what resulted in the huge collapse in engineering quality. People that spew out shit with flashy features for product managers get rewarded and move up while people who stabilize and maintain products are cast aside.
The size of the engineering group at google is more than 10x what it was back in the glory days of stuff like gmail, reader, maps, etc. Google now just makes incremental worse changes to existing products (look how slow maps and gmail are) and kills off things that the terrible engineering org can’t keep up with.
There are definitely great engineers at google, but the ratio of garbage to them has become untenable and Google is well on its path to mediocrity. A great example of this is that you have to do an interview again if you want to work on something actually innovative like self driving cars.
Unfortunately priorities at Google have changed significantly, and the focus is on things that can help sell more targeted ads at scale. Actually providing value is not that important anymore I'm afraid.
https://en.wikipedia.org/wiki/Henry_Spencer
That's like saying seismo was a random nuclear warhead detonation monitoring facility somewhere in Northern Virginia.
https://en.wikipedia.org/wiki/Rick_Adams_(Internet_pioneer)
If utzoo was involved, then they probably mounted a scratch monkey, at least.
https://en.wikipedia.org/wiki/Scratch_monkey
https://edp.org/monkey.htm
One of the interesting points the story brings up is that you never know what future people will care about. In this case cultural and political discussions are typically more interesting than arcana about bug fixes in long ago systems.
This is a feature of development based on sprints vs. a bigger vision. That design/development model has alot going for it, but it only works well when the scope of the system is narrow.
archive.org has several significant Usenet collections. Henry Spencer's UTZOO collection, stuff from Giganews, and some sort of dump for Google Groups / Deja itself. I don't think anyone's built a useful interface for them yet.
(Ben Swartz made the donation from Google: https://www.bensw.com/blog/Aaron-5-Years-Later/ )
> it’s 50 or 60 petabytes on disk, and the only people who can see it are half a dozen engineers on the project who happen to have access because they’re the ones responsible for locking it up.
https://www.theatlantic.com/technology/archive/2017/04/the-t...
They invested non-trivial amounts of money in scanning those books, deleting them would throw that money away. As for Usenet, it's similar: storing the data is cheap, and acquiring it again is probably next to impossible, so unlikely they threw it away unless idk it contains proof that sergey and larry have stolen their ideas from some usenet post or something.
To me it's a lot simpler than that: on Usenet one could find people exchanging opinions about a product while on today's web searches all we find is companies selling that product. Just try searching for anything and see how deep you need to go until you get something that resembles a legit conversation about something rather than people selling it. It was about monetizing every search results page, and that goal became clear to me when they removed the discussion search option from the search engine; that was the final nail in the coffin for the Internet as we knew it.
Some background: https://www.seroundtable.com/google-search-filters-gone-1799...
There's also the problem of consuming old internet content. Do you still have a gopher client? Will old web pages look the same in your browser as they did in 1995? Flash will no longer work (hooray?).
On the other hand books, photographs and magazines stick around until you throw them away and we have public institutions that are dedicated to their preservation.
Flash also works if you are using Chrome. Sadly, Firefox does not support Flash plugin anymore (and i say sadly because just yesterday i was trying to find some files from an old site that was made using Flash and i had to use Chrome just for that).
This is a big problem. Firefox dropped Gopher and Chrome never supported it. Most Gopher users, presumably, use a Gopher web proxy.
For fun I recently built a Gopher web client, implementing the entire protocol in a single page app (literally a single HTML file, no frameworks or dependencies). It supports most documented file formats including "modern" extras like video and audio.
Gopher client: - https://gopher.commons.host
Source code: - https://gitlab.com/commonshost/gaufre
There's also the question of hardware, but so long as we have at least one up-to-date PC emulator, we have an answer to that. It might become more of a challenge going forward, as mainstream emulators drop BIOS support and other legacy parts of the platform; but even then there's stuff like PCem.
But it's just gone now. Can't find a trace of it.
[1] http://www.angelfire.com/ga3/ebhistory/archive.html
That's why we have stories, and care about legacy. All to avoid forgetting and being forgotten.
Well, at the time scale of forever, it's always sketchy, but the odds are decent.
The internet will need that ability as it becomes more intelligent. Having AI reasoning based on vast pools of fake news and other disinformation would not be a good thing.
I singlehandedly maintain a 14-year-old website that used to be a modestly popular web magazine. It's not very expensive, but it's a pain. DNS system is horrible and it's easy to loose domain names to some nonsense. (I lost one that used to be a free 2nd level domain when it was converted to a paid-for zone. Not a matter of money, just paperwork.) Server management is a time drain. Stuff like adding SSL certificate to a legacy VPS can lead to a cascade of updates and config changes that can take days to make and test.
BTW, everyone sings praises to archive.org (and it's well-deserved), but most people here do not seem to realize that they are also a centralized platform that can collapse and take everything down with them. Who archives the archives, etc. Fortunately, they are not the only one of the sort. Unfortunately, it's all very ad-hoc.
If W3C weren't a bunch of corporate shills, there would be a standard for creating versioned web archives, like, 10 years ago. It's obvious that we need one.
Another archive:
https://www.bibalex.org/en/project/details?documentid=283
There are also partial distributed backups by volunteers:
http://iabak.archiveteam.org/
(also there is a standard for web archives: WARC)
* The famous first web page: http://www.w3.org/History/19921103-hypertext/hypertext/WWW/T...
* The Global Network Navigator: http://oreilly.com/gnn/gnnhome.html
* Trincoll Journal: https://web.archive.org/web/20010413164311/http://www.trinco...
* BBC Networking Club: https://archive.org/details/bbcnc.org.uk-19950301
Anything pre 1994 is very hard to find.
Great list you've got.
http://www.milliondollarhomepage.com/
[Wow]
That's actually easy - it's literally what blockchains are for. The whole issue is how to manage the tradeoff between reliably archiving everything (and expecting participants in the network to do the same, so that the system doesn't end up relying on a single point of failure), and delivering/serving the stuff efficiently.
This is really interesting. If you think about it, for all of our print archives, the creator bore some of the cost of archiving it. When you print a book you have to send a copy to the library of congress, so the "storage" is paid for by the publisher. Same with newspaper archives -- usually the newspaper is sent by the publisher, who pays for the "storage" (the paper and ink). I think the UK and France and many other countries have similar systems.
But in the digital age, the entire cost is carried by the archiver.
As best I can tell, you only need to send two copies of a book to the Library of Congress if you want to get an LCCN. Otherwise, there's no requirement to do so, certainly not to receive copyright protection.
But the bigger point is that to be in the archive you have to bear some of the cost.
The cost of just keeping a book in a library is great enough that all but the richest academic libraries have an ongoing process of identifying and selling or destroying books with no circulation over a period of time. Usually they do so after checking whether they can borrow a copy from another library in their network should one of their patrons ask for it.
Now the internet does not seem as fun and wholesome as it did back in the 90's; I feel bad that my kids don't get to enjoy that. On the plus side, maybe they can make some side income as influencers?