In addition to this, on the blogs that were migrated to the new platform (such as Raymond Chen's Old New Thing) all of the comments have been purged. That's extremely unfortunate because those comments had a lot of useful technical commentary on the blog posts.
If you don't have a system in place to automatically cleanse user-submitted content, you have a liability. The chilling effect this has will only grow over time.
Not sure why this is being downvoted. "If you don't have a system in place to automatically cleanse user-submitted content, you have a liability" is indeed not part of GDPR as far as I know, nor does it seem to be the reason why the comments weren't migrated.
Probably referring to "right of erasure"? If you want to delete your comments, but there is no mechanism for that, then Microsoft may be in violation of that part of GDPR.
Yep, I said as much in another comment. The downvoted comment I replied to was still right -- the issue preventing MS from migrating comments doesn't appear to be liability due to a lack of filters, which isn't even part of GDPR.
It could be that I'm confusing it with something else, it's hard when everything is named with meaningless acronyms. But I really thought it was something to do with GDPR even if the connection isn't immediately obvious.
You are thinking of Article 13 (the upload filter), which hasn't been passed yet. Not to be a pendant, but an initialism cannot be meaningless almost by definition -- it has to mean something otherwise it wouldn't be an initialism. GDPR means the "General Data Privacy Regulation".
FWIW, I completely agree with you on the chilling effect of upload filters. But upload filters aren't part of GDPR by any stretch of the imagination.
If I write a comment and my name is bryanrasmussen and then later I want to delete any data that can be publicly linked to me then I suppose a comment written with my name on it might count.
It also might not.
It might be worth it to some company to determine which it is, but who would expect that company to be Microsoft?
The GDPR doesn't say anything about requiring automated systems do do any of this. The volume of erasure requests is so low that I doubt it would even be worth it for Microsoft to invest in such a system.
MSDN comments still exist and Microsoft operates several forums. Perhaps GDPR is a questionable excuse for some lazy sysadmin not wanting to migrate comments, but it doesn't fly here.
Yikes, that's false and not nice. We delete comments for people nearly every day. It's true that we don't allow wholesale deletion of account histories, because that would gut the threads the account had participated in. But we also don't want anyone to get in trouble from anything they posted to HN.
The idea is to balance the need to preserve history (community interest) and the need to protect users (individual interest). We don't get there perfectly, but I can tell you for sure that the vast majority of people who ask us about this end up happy with how we help them.
I vividly remember the meeting I went to in 2003 with a bunch of Microsoft bloggers and lawyers -- it was one of the few times Raymond Chen and I were in the same room. :-)
It was astonishing because of what the lawyers said: yes, it is a public relations risk to have ordinary employees communicating directly with the developer community, and we are willing to take on that risk if it dramatically improves our satisfaction metrics in the developer community.
We were basically told "don't be stupid, don't share corporate secrets, comport yourselves professionally, keep it on-topic, and if there are legal problems, legal will handle them appropriately".
I don't know that things would be the same today; it was a different time. But I'd hope so.
That's pretty great :) I mean, of course there are situations where legal is right and there are situations where a risk is taken because it's worth it, like in that situation. The newegg anti-patent-troll stance comes to mind as well. But being overly risk averse and having lawyers drive the company happens as well, and it might have happened with the comments here.
That's pretty cool. Imo, I think it really paid off and I'm thankful to get the kind of insight into Microsoft that these bloggers managed to give. I think it humanized Microsoft and their developers for a lot of people (including me).
Or more accurately, MS did not want to make the effort to carry it over.
GDPR made it more expensive to port the comments, because it insisted that the user have control over their content. It did not prevent it from being done.
Control doesn’t mean that you need a user interface or an account management. It merely means that MS needs to respond to a request like "I posted this comment, please delete it". Furthermore, under GDPR you don’t have to delete everything a user has written. Only personal information. Although it’s probably easier to just delete the whole comment.
The problem is the EU approach to GDPR enforcement. Fines are so huge that probably MS lawyer team decided not to take chances for something that does not bring revenue - some developers might cry on Twitter or Facebook for a week or two, but that would cost MS nothing.
Unfortunately "right to be forgotten" has been wrongly defined in GDPR, if someone published something in a publicly available place, this shouldn't be removable on demand. I am perfectly for forcing companies to be obliged to delete its customers data if customer does not want make business with a said company or does not want to allow company to use the data for its business (like advertising), but this should not cover every random forum in the Internet.
In fact this is the reason those independent forum will disappear soon or later and be moved to Facebook, which is rather sad and shows only how EU bureaucracy is inefficient - they wanted to restrict Facebook usage of private data, instead they only caused even more people to move to Facebook and give up their privacy.
It’s really frustrating to see large influential companies continue to perpetuate misinformation about GDPR – legislation for the benefit of ordinary users. It’s doubly frustrating to see Hacker News users fall for it (other posters arguing that GDPR is not a valid reason for removing comments have been heavily downvoted). As an influential group themselves, Hacker News users continue to perpetuate these misunderstandings. There’s no provision in GDPR stating that users have the right to retract or amend content they have published. The GDPR is only concerned with personal data so comments should only be amended/removed if they contain information relating to an identified or identifiable natural person – which could be the comment author themselves or an entirely different other person.
sigh ... yeah, it's Microsoft a big company with lots of money. But it's not unlimited and not every case can be handled. It was decided that there wasn't time to move comments and put in controls to be able to migrate accounts and self delete, there likely also wasn't time for the effort it would take to put other controls in place AND actually do the migration of comments.
In the end, they project is probably like any other, short on time, and not enough people to cover all the cases needed. As it is, it seems like it wasn't smooth, and comments would be a much larger complication. That doesn't mean it could never happen, just that it hasn't happened.
GDPR is the reason for deciding it's easier to dump them than to migrate them AND provide the controls they would likely need in place. It may be a small number of requests, relatively speaking, but judging from what YouTube, twitter and the like receive, I can't blame MS for taking the easy route.
In a similar vein, I had one particularly obscure winapi bug that was discussed in some old msdn articles and blogs, but with no solution. Only after a few days of ramming my head into my keyboard did I do a wayback search and discover that a now-purged comment on one of the articles had had my solution all along.
The is highly unfortunate and a bone-headed move by Microsoft. But it does highlight two points that often get missed now that we rely on the web.
1 - If you don't host your own content then you have no control over it. Microsoft deleted all their blogs. YouTube and Facebook will one day cease to exist. This comment and all of yours will disappear once news.ycombinator.com is defunct. If you want something to last, host it yourself or be prepared to recreate it somewhere else every few years.
2 - Once you publish something at a URL you control (see point 1), you have a responsibility to keep it available at the same URL. Nobody is going to hold you to it but you will break 3rd party sites that link to you. Microsoft really dropped the ball here.
I presume that laughing at idiots who actually wrote this sort of thing without realising the irony will be one of the periodic amusements of next century.
The only way to increase the information's chance to survive any significant length of time is copies. But making copies of digital artefacts is so trivial we made it totally routine, when did your backups last run? Whereas historically objects like tablets were only reproduced if there was extraordinary interest, so we have mostly Bibles, and random lists that survived entirely by chance due to just how many there were, often inventories or other boring records.
To be fair most of the books written in Herodotus’ time are also gone! Many of them we only know about from Byzantine library catalogs or summaries, but surely many more have just completely vanished from knowledge or existence.
With the existence of things like the Internet Archive, it’s quite possible the public web content of today will be well preserved for centuries to come.
Right now the Internet Archive and BitKeepers feel like Alexandria's library, a kind of single point of failure with a big loss to all of us if something happens.
And we also need to keep running the applications that know how to deal with specific file formats.
To be fair, if we do manage to preserve the Internet Archive to a significant degree, information on html and other filetype standards should be readily available for future recreation.
But it's much easier to mirror the Internet Archive than it was to mirror Alexandria's library. In fact, it already has one official mirror (which is hosted by the modern library of Alexandria!).
FWIW, with today's typical Internet speeds and storage prices, it's entirely feasible for individuals to mirror the entirety of Wikipedia and Project Gutenberg (done both myself, and a bunch of other stuff like TED talks). Throw in a mirror of the entire Debian source package repository, and there's your apps, too. How many copies are already around? Who knows, but I'd say we already have a lot more redundancy than at any point in the past.
Of course, IA is to any of these like these are to what you could store locally circa 90s. I don't think we're going to be at the point where an individual can easily store 40 petabytes of data anytime soon. But for large entities - governments and corporations - this is feasible. And I suspect there are some private full IA mirrors out there already, in places like NSA's Utah data center.
I doubt you're ready the original manuscript though. You're reading copies of copies, there is no reason why content on the Internet will not continue to be copied in the same way.
It's a matter of scale. You can only make a copy of the whole internet by committing to that purpose, and there's only one outfit that does it in a public manner. I can't imagine what it would take to duplicate that database right now, and it increases every day.
Within the same distance into the future, most of today's knowledge will be gone.
A bunch of it is already gone.
Think of all the music recordings that didn't make the transition from 78's → LP's → CD's → streaming.
Think of all the movies that didn't make the transition from film → videotape → DVD → streaming.
I'm always sad when I see the pile of books being discarded by the Chicago Public Library for 10¢ a piece, knowing that those that aren't picked up by the public end up in the trash. The information in them may be outdated, but it's an important window into what people felt/thought/believed in a particular period of time.
The world loses a metric assload of its history each day.
Are they? I mean, just recently there was a post on HN asking whether the G+ backups were necessary in the sense of creating a permanent record of everything people submit online.
Make no mistake, certain content is worth archiving, but imagine if we'd been archiving everything since our species first days.
Now we have the capability, we're obsessed that we must archive everything; how useful are the older IMDB comments really?
I dread to think how much storage is 'wasted' in these archives storing useless content, and how useful it really is to people in reality.
There are archives of every reddit comment ever made, but there are many hundreds of thousands of spam or useless comments....
I don't think the Internet needs an immutable record of every piece of content or every hyper link to preserved for the rest of time, we need to discern what's really important and throw away the rest, if not to make it easier to parse (but, since that's difficult, we just grab it all which whilst great presents its own issues in terms of required resources).
I think it would have been immensely valuable for anthropological reasons to see the mundane details of the lives of people who lived thousands of years ago. If we just keep only the most important works it doesn't really give a clear picture of how people actually lived at that time. That said, I definitely agree that there will be diminishing returns if you keep everything.
At the very least it will be useful to future history/sociology phd students who will be able to pick any moderately active forum or subcommunity from the past and write a dissertation on its idiosyncracies
"As far as the business goes, I make a lot more in royalties on my beginner C# videos that O’Reilly distributes than I do on all my book royalties and editing fees."
(E.L., in the comment section apropos wishes for a book)
By a huge margin! And putting together that beginner course only took me a couple of months working in my spare time, and recording it only took a couple of days in O'Reilly's studio.
I still edit my friend's programming books, but I sure don't do it for the money.
Hi Eric,
Thanks for all the hard work you did (and doing) putting high-quality technical content on the web. I always consult your blog (or, in all laziness, your SO repertoire) first when in doubt with C#. You're hands down the best explainer of cs-things ever.
Thank you, that's a kind thing to say. I find that if I cannot explain a topic clearly to someone else, I probably don't understand it very well myself. :-)
To be fair, Microsoft is doing a great job with their new docs site, and I can't blame them for wanting to centralize and refine the search ranking toward their new pages.
Some still have thin content, but I've been pleasantly surprised to keep finding the new docs as the top search results. Just a year or two ago, it was a crap shoot.
"UPDATE: The (awesome) Scott Hanselman informs me that there has been a “hiccup” during migration, and that the intention was to archive the MSDN blogs in a read-only format with the same links; they should be back soon."
One favorite trick of Microsoft IT is to delete old instances of SharePoint servers for projects. Where "old" is maybe 3-4 years.
Goodbye design documents. Goodbye test plans. Goodbye user studies. Goodbye meeting notes. Goodbye just about everything interesting except for the source code of the project. And backups? "What is this word you are using . . . ''backup''?"
Which is why I just checked everything into source control and told the PMs they could bloody well learn how to use Source Depot.
There's been a lot of MS content I've noticed disappearing lately. A lot of resources related to Internet Explorer for example (the Microsoft Connect bug tracker, and some MSDN blogs). Yeah IE is (almost) dead by now, but there has been a lot of useful technical knowledge there.
BTW. Today is a great day to donate to the Web Archive:
For years I'd comment code using links to reference material, like the API documentation for an API call being made. It was very handy. Links to Microsoft online documentation, however, tended to go stale very quickly. And it didn't just go stale, it would disappear (google could no longer find it). A lot of COM/ActiveX documentation has disappeared. I don't understand why Microsoft does this.
They don’t want people to keep writing COM/ActiveX code, and they don’t want search engines to point you to such pages, so why would a commercial entity spend money to keep those pages around?
Hosting costs may be peanuts for them, but there’s also maintenance. Even if they don’t bother updating the look of the site, there’s the fact that old sites eventually run on no longer supported technologies that may only run on unsupported OSes, both potentially riddled with security vulnerabilities.
I take your point, but VS twitter is this week running a series of short videos with Larry Osterman about how awesome COM is, which seems to argue against this theory.
That said, lots of times Microsoft's left hand does not know what the right hand is doing.
"They don’t want people to keep writing COM/ActiveX code, and they don’t want search engines to point you to such pages, so why would a commercial entity spend money to keep those pages around?"
That would just be nuts. There is a ton of legacy code that needs maintenance and therefore needs documentation. this is not the JS world where it's OK to break stuff twice a year.
That is a very good point. I should start saving snapshots of the resources I want to link to in documentation using the "save now" feature at https://archive.org/web/
This way I know that the link will not rot and I will have a record of the exact information that lead to the decision I am documenting.
Edit: I wonder how the wayback machine deals with GDPR compliance
Edit 2: After further research, there is an exemption for archiving for the public interest. It looks like this was either an overreaction of MS's legal staff or simply blaming migration issues on GDPR.
But at the time I left Microsoft, I was told that .NET management no longer considered my blog a valuable asset, and that I should stop working on it on company time, and that they had “community PMs” who would be owning the relationship with the programming community.
Around that time I was also told, funnily enough, that I should never mention Windows on my blog, because the Windows marketing team wanted to own all communications with developers about writing software for Windows.
I really enjoyed my time at Microsoft; the people I left behind were amazing, and I miss working with them. And none of the above is high on the list of reasons why I left Microsoft. But middle management was weird, man, weird.
Reposting this comment from Eric's blog for more context (PM on the Developer Relations team. We have been migrating blogs since 2017 and enabling employees to export / migrate their blog. To your specific issue, ex-employees that want their blog posts can email me dan(dot)fernandez(at)microsoft(dot)com and I’ll route to the support team. This does not include comments as the MSDN/TechNet blogs are not GDPR compliant (aka the right to be forgotten).
As for why, other community sites like TechCommunity are driving blogging at Microsoft for individuals, while product teams are looking to group official blogs (Windows, Office, and Visual Studio), and for blog posts that were really tutorials, moving them to open source documentation on https://docs.microsoft.com on GitHub. Customers ran into a number of issues where blogs provided out-of-date content/guidance that lead to support issues. For example, of the 16k+ blogs, 86% had not been used in 18 months and thousands were created with no posts or only a single blog post. Cheers,-Dan
89 comments
[ 2.4 ms ] story [ 161 ms ] threadMight be the copyright reform, but comments should still be fine with that.
FWIW, I completely agree with you on the chilling effect of upload filters. But upload filters aren't part of GDPR by any stretch of the imagination.
It also might not.
It might be worth it to some company to determine which it is, but who would expect that company to be Microsoft?
MSDN comments still exist and Microsoft operates several forums. Perhaps GDPR is a questionable excuse for some lazy sysadmin not wanting to migrate comments, but it doesn't fly here.
The idea is to balance the need to preserve history (community interest) and the need to protect users (individual interest). We don't get there perfectly, but I can tell you for sure that the vast majority of people who ask us about this end up happy with how we help them.
It was astonishing because of what the lawyers said: yes, it is a public relations risk to have ordinary employees communicating directly with the developer community, and we are willing to take on that risk if it dramatically improves our satisfaction metrics in the developer community.
We were basically told "don't be stupid, don't share corporate secrets, comport yourselves professionally, keep it on-topic, and if there are legal problems, legal will handle them appropriately".
I don't know that things would be the same today; it was a different time. But I'd hope so.
Basically the old accounts couldn't be carried over and so users wouldn't be able to log in anymore to delete their comments.
GDPR made it more expensive to port the comments, because it insisted that the user have control over their content. It did not prevent it from being done.
Unfortunately "right to be forgotten" has been wrongly defined in GDPR, if someone published something in a publicly available place, this shouldn't be removable on demand. I am perfectly for forcing companies to be obliged to delete its customers data if customer does not want make business with a said company or does not want to allow company to use the data for its business (like advertising), but this should not cover every random forum in the Internet.
In fact this is the reason those independent forum will disappear soon or later and be moved to Facebook, which is rather sad and shows only how EU bureaucracy is inefficient - they wanted to restrict Facebook usage of private data, instead they only caused even more people to move to Facebook and give up their privacy.
https://ico.org.uk/media/for-organisations/documents/1554/de...
In the end, they project is probably like any other, short on time, and not enough people to cover all the cases needed. As it is, it seems like it wasn't smooth, and comments would be a much larger complication. That doesn't mean it could never happen, just that it hasn't happened.
GDPR is the reason for deciding it's easier to dump them than to migrate them AND provide the controls they would likely need in place. It may be a small number of requests, relatively speaking, but judging from what YouTube, twitter and the like receive, I can't blame MS for taking the easy route.
Oh, and we can't forget xpclient wanting his old Explorer search style back.
http://web.archive.org/web/20190301040236/https://blogs.msdn...
1 - If you don't host your own content then you have no control over it. Microsoft deleted all their blogs. YouTube and Facebook will one day cease to exist. This comment and all of yours will disappear once news.ycombinator.com is defunct. If you want something to last, host it yourself or be prepared to recreate it somewhere else every few years.
2 - Once you publish something at a URL you control (see point 1), you have a responsibility to keep it available at the same URL. Nobody is going to hold you to it but you will break 3rd party sites that link to you. Microsoft really dropped the ball here.
Within the same distance into the future, most of today's knowledge will be gone.
The only way to increase the information's chance to survive any significant length of time is copies. But making copies of digital artefacts is so trivial we made it totally routine, when did your backups last run? Whereas historically objects like tablets were only reproduced if there was extraordinary interest, so we have mostly Bibles, and random lists that survived entirely by chance due to just how many there were, often inventories or other boring records.
With the existence of things like the Internet Archive, it’s quite possible the public web content of today will be well preserved for centuries to come.
And we also need to keep running the applications that know how to deal with specific file formats.
FWIW, with today's typical Internet speeds and storage prices, it's entirely feasible for individuals to mirror the entirety of Wikipedia and Project Gutenberg (done both myself, and a bunch of other stuff like TED talks). Throw in a mirror of the entire Debian source package repository, and there's your apps, too. How many copies are already around? Who knows, but I'd say we already have a lot more redundancy than at any point in the past.
Of course, IA is to any of these like these are to what you could store locally circa 90s. I don't think we're going to be at the point where an individual can easily store 40 petabytes of data anytime soon. But for large entities - governments and corporations - this is feasible. And I suspect there are some private full IA mirrors out there already, in places like NSA's Utah data center.
And then you can make distributed backups, too: http://iabak.archiveteam.org/
Yeah I guess we need to trust that some of us will keep our recipe lists alive. :)
http://web.archive.bibalex.org/web/
A bunch of it is already gone.
Think of all the music recordings that didn't make the transition from 78's → LP's → CD's → streaming.
Think of all the movies that didn't make the transition from film → videotape → DVD → streaming.
I'm always sad when I see the pile of books being discarded by the Chicago Public Library for 10¢ a piece, knowing that those that aren't picked up by the public end up in the trash. The information in them may be outdated, but it's an important window into what people felt/thought/believed in a particular period of time.
The world loses a metric assload of its history each day.
Make no mistake, certain content is worth archiving, but imagine if we'd been archiving everything since our species first days.
Now we have the capability, we're obsessed that we must archive everything; how useful are the older IMDB comments really?
I dread to think how much storage is 'wasted' in these archives storing useless content, and how useful it really is to people in reality.
There are archives of every reddit comment ever made, but there are many hundreds of thousands of spam or useless comments....
I don't think the Internet needs an immutable record of every piece of content or every hyper link to preserved for the rest of time, we need to discern what's really important and throw away the rest, if not to make it easier to parse (but, since that's difficult, we just grab it all which whilst great presents its own issues in terms of required resources).
Then there would be no "Mysteries of...!" shows on basic cable.
(E.L., in the comment section apropos wishes for a book)
I still edit my friend's programming books, but I sure don't do it for the money.
http://web.archive.org/web/20190301145321/https://blogs.msdn...
Some still have thin content, but I've been pleasantly surprised to keep finding the new docs as the top search results. Just a year or two ago, it was a crap shoot.
As an example, here is the top result for "C# Synchronization Primitives" https://docs.microsoft.com/en-us/dotnet/standard/threading/o...
"UPDATE: The (awesome) Scott Hanselman informs me that there has been a “hiccup” during migration, and that the intention was to archive the MSDN blogs in a read-only format with the same links; they should be back soon."
Goodbye design documents. Goodbye test plans. Goodbye user studies. Goodbye meeting notes. Goodbye just about everything interesting except for the source code of the project. And backups? "What is this word you are using . . . ''backup''?"
Which is why I just checked everything into source control and told the PMs they could bloody well learn how to use Source Depot.
BTW. Today is a great day to donate to the Web Archive:
https://archive.org/donate/
Hosting costs may be peanuts for them, but there’s also maintenance. Even if they don’t bother updating the look of the site, there’s the fact that old sites eventually run on no longer supported technologies that may only run on unsupported OSes, both potentially riddled with security vulnerabilities.
That said, lots of times Microsoft's left hand does not know what the right hand is doing.
That would just be nuts. There is a ton of legacy code that needs maintenance and therefore needs documentation. this is not the JS world where it's OK to break stuff twice a year.
Win32 is mostly stuck in how Windows XP used to be.
This way I know that the link will not rot and I will have a record of the exact information that lead to the decision I am documenting.
Edit: I wonder how the wayback machine deals with GDPR compliance Edit 2: After further research, there is an exemption for archiving for the public interest. It looks like this was either an overreaction of MS's legal staff or simply blaming migration issues on GDPR.
But at the time I left Microsoft, I was told that .NET management no longer considered my blog a valuable asset, and that I should stop working on it on company time, and that they had “community PMs” who would be owning the relationship with the programming community.
Around that time I was also told, funnily enough, that I should never mention Windows on my blog, because the Windows marketing team wanted to own all communications with developers about writing software for Windows.
I really enjoyed my time at Microsoft; the people I left behind were amazing, and I miss working with them. And none of the above is high on the list of reasons why I left Microsoft. But middle management was weird, man, weird.
Doing a google search on the above comment leads to this HN comment...
As for why, other community sites like TechCommunity are driving blogging at Microsoft for individuals, while product teams are looking to group official blogs (Windows, Office, and Visual Studio), and for blog posts that were really tutorials, moving them to open source documentation on https://docs.microsoft.com on GitHub. Customers ran into a number of issues where blogs provided out-of-date content/guidance that lead to support issues. For example, of the 16k+ blogs, 86% had not been used in 18 months and thousands were created with no posts or only a single blog post. Cheers,-Dan