Don't forget emails.. I have everything I ever sent or received, and I have it backed up. I expect 90% of my inbox is the jpg signature logo they attach to the bottom of my clients email rather than hyperlink.
That's not even crap data - that's archived data that might be useful someday (though de-dupe is probably a great idea and email sigs are definitely wasteful trash) - most of the crap data is things that would never have been useful under any circumstances.
I have cleaned up dozens of product databases in cost management efforts and have found anywhere from 50-99% of data stored in product databases is crap, because they are not well managed and any single mistake can lead to a huge outsized impact on storage.
What to log all those http requests for just a day? Might as well turn that on for all time...
I ended up working on some software and I was deemed the email guy (it's a very small % of my job but it is the biggest pain).
"I need an email when this happens.. and when this happens."
The requests are endless and I'm convinced there are people who if they could would do their entire job from their inbox and get everything and anything an application can do via email.
The insidious problem is that it never solves anything. "I didn't get the email!" is a constant refrain (yes they did, they always did). "Oh someone didn't do the thing so can you add this to the email too." and so on.
> The requests are endless and I'm convinced there are people who if they could would do their entire job from their inbox and get everything and anything an application can do via email.
That sounds like a reasonable goal for a whole lot of job duties. And yes some entire office jobs. (Excluding some direct human communication but a lot of jobs already have too much of that in situations that could have been an email.)
> "I didn't get the email!" is a constant refrain (yes they did, they always did).
Well having to manually check wouldn't improve that, would it?
We had a similar request for a sales team once: "If this fails we want an email".... Okay, but we never seen this fail, so you'll never get the email. As expected, they never got an email, because no failures. So instead they wanted an email every time the job ran (once every night), so they'd know that the job had failed, if they DIDN'T get the email. Only time that happened was because the email got trapped in a spamfilter I think.
There must be thousands of copies of that email sitting in inboxes say: Job X ran successfully @ 04:30.
Well "the cloud" will generally store exactly one logical copy of your static jpg, one of the reasons why clouds are pretty good for efficiency.
There is really no sense at all in the article's claims that "we are destroying the environment" to do x y and z thing the author whines about. We are destroying the environment to drive a Dodge Ram to the Circle K to buy a 52oz Polar Pop. Information sector doesn't even show up on top ten lists of things that are destroying the environment.
It's probably deduplicated on the server though, so the millions and millions of messages with that logo likely share the same piece of disk space. Probably one reason why free providers don't tend to offer End-to-End encryption. It prevents deduplication (and otherwise compressing redundant information).
Back in the day Exchange offered SIS but in 2010 they ditched it. It's plainly not effective any more. Even regarding the OP' "the jpg signature logo" - it's a part of multipart in the message, not a separate file.
And one more thing - you can't just turn the dedup and be dandy, now you need to check against the hashes to determine if this chunk is unique or you already have it. And with TBs of data you need TBs of hashes to check. Until you have like 99% dedup efficiency, ie 99% of your incoming data is literally the same data you already have - it doesn't worth it.
Not sure about Microsoft solutions, but modern file systems like zfs and btrfs can do this on the filesystem level, no support from the mail server needed.
Also, I can imagine such schemes getting ditched for the same reasons of "now people can detect whether we already have a copy of the file, which is a privacy hole!" that killed cross-origin caching in browsers.
Or if everything in the year 20XX gets pushed into using E2E encryption, since that's pretty much antithetical to deduplication.
> Probably one reason why free providers don't tend to offer End-to-End encryption.
I would think that email providers tend to not offer E2EE just because it fundamentally isn't practical with email. Providers like Proton try to do it, but it works only if you talk to someone else on Proton (at which point you may as well do it on Signal, which has better encryption).
Fortunately with LLM's we can generate data and not store it. If you think about it that means articles could always be relevant with additional details added as time goes on.
No, the reality of this situation is that people need to save a snapshot of what the LLM said when they read it, just in case they need to substantiate it or blame someone for what they read at the time.
> One organization I knew of had 1,500 terabytes of data, with less than 2% ever having been accessed after it was first stored.
On a related note, probably a similar percentage of people claim on their car insurance. If only the rest realised they had "crap insurance" and were paying for nothing, they could save so much money!
This is obviously sarcasm, but I think it's important to remember that much of the data is stored because we don't know what we will need later. Photos of kids? Maybe that one will be The One that we end up framing? Miscellaneous business records? Maybe those will be the ones we have to dig out for a tax audit? Web pages on government sites? Maybe there will suddenly be an interest in obscure pages on public health policy if a global pandemic happens.
Complaining that data is mostly junk is not a particularly interesting conclusion without acknowledging this. Is there wastage? Yeah sure, but accuracy on what needs storing is directly traded off with time spent figuring that out, and often it's cheaper to store the data.
I have ~2.5TB of photos in iCloud, via Apple Photos. Excluding the various sized previews, I doubt many originals have been accessed in quite a while. I also have about 1TB Lightroom library archived to a different service; representing countless hours of photo processing work spanning over a decade. Haven’t touched that one in years. Neither are crap. (Yes, both have other backups and, yes, I’ve probably forgotten to sync one of them).
Sometimes I wish there was a feasible system to audit and reduce redundant photos in my iCloud. I have an embarassing ton of pictures that I never look at just like everyone else. I wouldn’t mind a tool that said “here’s 10^x photos that are the most similar to all your other photos would you like to put them in the trash?”
If you’re using Apple Photos, the feature is there. It will detect both exact copies as well as (nearly) identical visual duplicates. Look under utilities.
Maybe it's different on the desktop, but on the iPhone it only detects the same photo that may have been saved at different resolutions or things like that. That's useful, but I really want something that can say use an embedding or something to group all 10 photos I shot of friends around the campfire and help me select one keeper and delete the other 9.
The feature works the same on iPhone. Looking through my duplicates just now, the detected duplicates that had a dozen or so photos to merge, are either high speed sports photography bursts or astrophotography—the rest were just mostly two photo duplicates. I suspect the threshold for determining a the photo as duplicate is too high for successive* photos to be considered such, as with your campfire example.
With that said, I’m surprised Apple hasn’t implemented a feature to group/bundle successive photos beneath what is determined to be “best.”
[*] Note: successive photos being separate shutter taps/actuations (potentially several seconds apart), where bursts are continuous (generally as fast as hardware allows, ms apart).
I have about the same in iCloud and then dozens of drives of video footage. Beyond the "keep it in case you need it" aspect, another angle is that it's often cheaper to just keep everything than spend all the time it would take to do a thorough cull. Video is much slower to review, but also chews up far more space, so maybe that balances out.
This is exactly the problem. Consider an order processing system that processes a million orders a day, and retains them for ninety days. What percentage of those 89-day-old orders are actually needed at 89 days old? It could be quite low, maybe a couple thousand out of a million.
But if those orders aren't there, shit hits the fan. PCI compliance audits fail. The ability for customers to reconcile their charges with their purchases breaks. In that 0.01% of cases where the order was fraudulent, placed by mistake, or just didn't have what the customer thought it had in it, not having that data makes the order processor read as, if not malicious, at least incompetent.
The real question is, how much data do we need to store inefficiently, in a way that uses a lot of power and space?
I was just pondering this today, in terms of how much data and objects I create in a Google or Microsoft account, for example, and then they create a burden of cost and maintenance for me down the road. Especially deleting old emails and photos. That's arduous and sometimes poignant as I flush away my personal life story.
Cloud services make difficult and sometimes even byzantine processes for deleting stuff, and it's often impossible to operate en masse in order to clean up swaths of stuff quickly and efficiently. It's in their interest to retain everything at all costs, because more used storage can mean more profits. Cloud services also profit from unused storage, because if they're charging $20/year to 100,000 users who use 2% of their storage space, ka-ching!
It irks me to this day that standard or even advanced filesystems don't include "expiration dates" or "purge dates" on file objects. Wouldn't it be logical, if an organization has a "data-rentention policy" that mandates destruction after X date, that the filesystem simply purges it automatically? Why does this always get delegated to handmade userland cron jobs? Moreover, to my knowledge, nobody is really interested in devising a way to comb through backup media in order to [selectively] destroy data that shouldn't exist anymore. Not even the read-write media!
Google is now auto-deleting stuff like OTP SMS messages. I'd love it if auto-delete could be configurable account-wide, for more than just our web histories and Maps Timeline and stuff. Unfortunately, to "delete" cloud data means it still exists on backups anyway. But without deleting data in your cloud account, it becomes a juicier hacker target as it ages and accumulates personal stuff that shouldn't fall into the wrong hands. Likewise for any business, it behooves them to delete and destroy data that shouldn't be stolen. At least move it offline so that only physical access can restore it?
I will say that modern encryption techniques can make it easy to "destroy" data, simply by destroying the encryption keys. You can quickly render entire SSDs unreadable in the firmware itself with such a command. Bonus: sometimes it's even done on purpose!
But even deleting data presents a maintenance cost. So if 90% of an org's data is indeed crap, then 90% or more of your processing resources are going to be wasted on sifting through it at some later date. Imagine when your file formats and storage devices are obsolete, and some grunt needs to retrieve some record that's 30 years old, and 90% of your data was always crap. That grunt is hopefully paid by the hour. We really had this happen at a few of my jobs, where we had old reel-to-reel backup tapes and it was difficult enough to load the data into a modern SunOS machine.
I suspect for most cloud providers you deleting data is cheaper because the data is not charged by the byte. But then they like having data, maybe just to train their AI models or for bragging rights to their investors.
For the expiration dates most modern file systems have the concept of arbitrary extended attributes per file. It's quite easy to add meta data like this yourself.
Unused services are always pure profit, and storage space is no exception. Providers can offer 100GB for like $24/year, because only like 2% of the subscribers will ever approach the limit, and so the extra space is never wasted but can be allocated to someone or something else.
It's like gym memberships, ISP/telco service bundles, and amenities at your apartment complex. Anyone not using every possible service is wasting money, but it's impossible to purchase a bespoke service, so essentially everyone will waste money because they're chipping in money for services that someone else uses more than they do.
Here at home I don't ever use the gym, the racquetball courts, the doggy-doo supplies, the laundry room, or a parking space, and yet my rent (everyone's rent) includes upkeep for all of those things. I'm subsidizing all my neighbors and all the wear-and-tear damage they put on those common amenities. Likewise, everyone who's paying $24/year for storage, or any business that purchases big multi-terabyte storage media, they're paying for unused storage space and giving profit. It's practically impossible to rightsize your storage media, and you never want it undersized, and you can't simply shrink them and reclaim the resources you invested, you just keep adding on new ones and replacing the malfunctioning ones. So nearly everyone always owns or rents more space than they can realistically utilize.
Furthermore, you'll notice that I specified "automatic" destruction of data by expiration date. Of course it is trivial to tag any arbitrary file with arbitrary metadata, but the challenge is to create a filesystem that executes automatic data purges on schedule, rather than pushing it into a rickety old handmade cronjob in userland. I've never ever seen a filesystem with such a feature, nor does it seem that anyone's interested in doing so.
And here I thought that computers were useful for automating business logic and easing the burden on human effort. And this is me, manually sifting through emails and photos in order to manually delete each one with 3 dialog boxes intervening. It takes hours, days, weeks.
There’s a whole concept of records management in enterprises that manages the disposal of data. It’s far more complex than just purge dates as there’s often regulatory requirements and legal discovery issues so <2% of data is actually disposed due to perceived risk.
For personal data, the concept would be simpler but still has requirements like say tax records need to be kept 7 years.
I would love a feature like retention policies at the filesystem level. If it could somehow take data provenance into account, that would be even better, i.e. data that I've created directly is irreplaceable and should never be deleted unless I say so and should always be backed up (photographs, writing, project source code, art, game save files, my list of installed applications, etc.); data that I've created indirectly is high priority but may be deleted under certain circumstances or may be omitted from backups (browsing history, shell history, recently opened file lists, frequently used app lists); data that can be easily replaced is low priority and may be cleaned up at any time and need not be backed up at all, contingent on how replaceable it is (application files, cached data).
Not at the filesystem level, but Android does something that feels like that: an app can write files in its own space, and when you delete the app, it deletes its files.
Except when the app writes files outside of its own space, which is meant for stuff that should stay (like pictures).
Of course, some apps store the pictures in their private space, and you lose the pictures when you remove the app. And some apps write crap in the shared space. But that seems like a fundamental limitation (even if it was done at the filesystem): how do you make sure that the apps/software you use is doing it right?
See now, that's quite unrelated what I'm envisioning, or the topic of the OP's article, but that's definitely the way we're headed in terms of file handling.
"File-type strictly associated with app" is good for security and it's good for people's mental organization. Unfortunately you're referring strictly to local file storage, I assume, and practically nothing on my phone is in local storage but "in the cloud". The only local storage I use is for ringtones and certain audiobooks/prayers which I enjoy listening whether or not I'm connected to a network.
It would seem that data centers and cloud services are a long way from "file types strictly associated with app" and also the sort of behavior you describe would be really horrible and undesirable in a typical data-center setting. I mean, I can't even imagine how it'd be implemented, given that data storage is often remote or at least decoupled from the "application server", and it's definitely never a 1:1 relationship between those abstracted components.
Can you imagine needing to "delete the /bin/tar application" in order to purge every file ending in ".tar" or ".tgz"?
But it's true that the new mobile OS paradigm has advantages for cleanup. If you've got a 3rd-party app and you've created some big crap files with it, and you uninstall that app, then it's a clean sweep when it takes along all the crap and you don't need to worry about lingering loose ends, or files you simply can't open, because their handler's gone.
Scoped storage is way more secure; it's also wonderful on my Androids that I can selectively grant access to files and media, rather than giving "the keys to the kingdom" to all these Social Media apps. I definitely never want to post my bank screenshots to Facebook and the possibility no longer looms!
These days I am sort of nonplussed when a web app demands an all-access pass to my Google Drive, because permissions can be appropriately scoped now, and the app can play in its own Drive "Sandbox" without having access to my spreadsheets and medical records. I just wish that there were some really good "cleanup" apps. Remember those "cleanup" programs that would come with antivirus for your PC, or Regclean, or whatever freeware utility would purport to liberate your 10MB HDD? Those were good times.
It's the same with social media posts--I often wish to really scrub my account, delete all posts and all comments and basically all activity, and while that's possible on Facebook it's arduous and it takes me weeks to scrub the major stuff after 4-5 years. But I only think of it on Shrovetide Monday or Fat Tuesday, and then I spend 3 weeks in Lent doing scrub, scrub, scrub. Sometimes the mouse/keyboard gestures are more repetetive & physically demanding than cleaning the toilet. Way too difficult, and that's by design.
Google or apple can put a big button delete everything in their phones/accounts but then some prankster will do it to a family member and this gives bad PR. Let's be pragmatic.
> It's in their interest to retain everything at all costs, because more used storage can mean more profits.
As an user, I get the reverse. When I had local NAS then I dumped anything and everything - assuming.. this costs not much. Will clean up later. Once I moved to cloud that changed to If I put crap then it will cost me money! Keep it clean
Once upon a time Google had given generous unlimited to all education and workspace. They stopped it in the last 2 years and you can see most educational and companies are running a tight ship.
Due to backup costs in our organisation we are forcing people to use max of 100GB for emails.
> advanced filesystems don't include "expiration dates" or "purge dates" on file objects. Wouldn't it be logical, if an organization
Totally agree.
Outlook email service has some kind of keep only the last newsletter from this service etc.
> auto-delete could be configurable account-wide, for more than just our web histories
- features are designed with majority in mind
- most users have some sort of nostalgia for reading or keeping old emails SMS etc
Assuming your somewhat methodical about labeling email in your non-primary tabs in Gmail, you can pretty much do periodic purges while not worrying much about your primary which, in my case at least, gets relative little traffic.
Unfortunately a lot of data retention policies aren’t so mechanical, but are of the form “delete after 7 years unless there is a legal hold in force”, which is usually just rare enough of an edge case that orgs evaluate it manually and hence only do periodic manual purges. But probably the main reason auto-delete isn’t popular is because a process that can delete your old data is one bug/misconfig away from deleting your new data too.
> The real question is, how much data do we need to store inefficiently, in a way that uses a lot of power and space?
This is indeed the critical question, and it's far from being trivial.
One issue we all hit is moving the data from the higher tier storage to the cheaper and more efficient one, which requires sync and paying for the transfer most of the time, but also handling two separate access and authorization process, backup and recovery system for data that absolutely needs to be accessible for the few years of legal retention, and can/must completely disappear afterwards.
In most orgs I've seen the cost of going through all that complexity is just not worth it, compared to "just" paying for the higher tier storage for the few years long lifetime of the data.
probably a similar percentage of people claim on their car insurance.
In that 0.01% of cases where the order was fraudulent, placed by mistake,
how much data do we need to store inefficiently, in a way that uses a lot of power and space?
I'm feeling, the real real question, as Sci-Fi as it gets, is, is the winning ticket data even data, OR is it more like a thumbnail of "the whole data" that is 98%+ worthless, than standalone piece of data?
The winning ticket ID, e.g. a "0x-636e-7461-4e7b", only makes sense in the context as one among the entire cohort of contestants; I can make one up like I did, but I can't walkout with the payout unless the rest of the lottery didn't exist.
Statistically, philosophically, technically, and all sorts of *-cally speaking, is the 2% data, the winning ticket datum, even data?
True, when it comes to data, there is a strong case to be a compulsive hoarder. It's cheap to store; often free and data is often marginally more useful and interesting than most generic mass-produced old junk which sits in garage gathering dust.
Technology for sorting through unordered piles of data also gets better over time. When I started storing digital photographs, face detection wasn't a thing, but it is now, so those tens of thousands of photos from the 90s suddenly became more useful to me.
If I knew the data i was going to need in the future i'd be in the future predicting business.
But there's an important piece there that is about data that should not have been stored in the first place. All the big data bullshit made people believe every single piece of data you could possibly store from a user is data that you should store and this is both a huge waste of resources and a huge liability, because now a data leak with useless to you PII could completely bankrupt your business.
I go back and look at old photos all the time (like when I think of a trip, and look back to it, and then just keep scrolling for a while). The pictures bring back more memories and that's to me a great joy - and on the other hand I find no value in deleting the photos that are not "the best" (I'm talking about good enough photos, I delete blurry, useless or duplicate ones immediately) - what would it gain? A few dollars less storage cost over the next decade? The satisfaction of digital cleaning?
I think you miss the point of the article, which precisely says that we store a ton of data we don't need to store in the first place.
Photos of kids are obviously not part of that "crap" (even if we have too many of them and it would be worth triaging for our own sake).
The question is: what makes for most of that data? Is that all business records, or is that storing stuff because we can? I've worked in multiple startups, all were tracking users as much as they could. Adding tools that collect data is easy, and storing that data is cheap. "We may need it later". Never needed any of that crap, and it was invading our users' privacy.
In any case, I think the article raises a good point: it's so cheap to store crap that we don't even think about it. And it's bad for the environment. Just like it's so cheap to take a plane that we don't think about it, even when taking the train takes almost the same time.
The article touches on this issue by pointing out that:
>The Cloud is what happens when the cost of storing data is less than the cost of figuring out what to do with the crap
But I think that's wrong. The actual issue is that you often can't figure out "what to do with the crap" because the difference between useful data and crap data is determined at the point in time when you need it, not when you store it.
I'm relatively careful with deleting data, but even so, there were countless instances where I thought something was no longer needed and deleted it, only to figure out that, a month later, I needed it after all.
Also, the article has a few anecdotes like this:
>Scottish Enterprise had 753 pages on its website, with 47 pages getting 80% of visits
But that is completely orthogonal to the question of what is "crap." The fact that some data is rarely needed does not mean it's crap. If anything, the fact that it is rarely needed might make it more valuable because it might make it less likely that the same data can be found elsewhere.
I do cull photos and docs but it takes effort and you will make mistakes. On the other hand, intelligent culling makes it easier to find things.
It’s a trade off especially with digital where there’s not that much cost associated with holding onto things relative to the physical world. My whole house is basically in storage because of a kitchen fire and I’m planning to be pretty selective about what I move back in.
>intelligent culling makes it easier to find things
That's true, but things get easier to find over time. I have thousands of digital photographs from the 1990s that I burned on CDs and then copied to a NAS later. Today, for the first time, they're actually searchable because they're now indexed in Immich. So they're sorted by the person in them and searchable by their contents.
If I had culled the photos back then, I would have lost many photos that have become searchable (and thus valuable) only recently.
"Photos of kids? Maybe that one will be The One that we end up framing? Miscellaneous business records? Maybe those will be the ones we have to dig out for a tax audit? Web pages on government sites? Maybe there will suddenly be an interest in obscure pages on public health policy if a global pandemic happens."
Perhaps I am missing something, but these examples all sound like candidates for _offline_ storage, for which no third party custodian or data center is required.
The price of large capacity NVMe SSDs continues to fall.
The amount of energy, the resource requirements, not to mention the environmental and community impact, of running a car insurance company are miniscule in comparison to that of running a data center.
Offline storage is only easy on the very small and the very large scale. On the very small scale you can get archival-grade CD/DVD/Blurays. On the very large scale you dedicate climate-controlled rooms to an automatic tape library.
In between is a vast gulf where your only good options are disks that you have to occasionally spin up to check integrity, have the disk trigger a data refresh if it's an SSD, and replace any disks that failed and rebuild their data from redundancy (raid or whatever you prefer). HDDs die even when off [1] and flash storage in SSDs is only rated to hold data for three years without power.
Sure, you can roll this yourself, but it can easily go wrong. Easier to either keep the disks spinning or pay someone with a tape archive (e.g. AWS deep glacier)
For your car note, insurance is only worth it if the thing being insured can ruin you if something were to happen to it. In the case of cars, you can potentially get ruined from the value of the other party's car. But if you live somewhere where most people drive normal cars, it might be worth it not having insurance. Our culture of insure everything, from your iphone to house is a market failure. If your house were to burn down, the value of the asset is still mostly concentrated in the land.
There's another dimension to this, that storage is so cheap that being wasteful with it isn't really disincentivized. I know for example at work of a portal that accepts uploads of large files from external clients that stores both the initial upload and every subsequent transformation of the file (of which there are 4-6) permanently. It's extremely useful for debugging, as one of the bits of metadata we shove on the zip archive is the git hash of the code that was running, so it's trivial to pull down any failed step and diagnose what happened.
We are using 4-6 times as much storage as we need to, and these are often not small files (on the order of 100 MB - 5 GB, several dozen times a day) but fixing this overuse is so far down the priority list that I don't think it survived the great Jira purge of mid-2024.
> storage is so cheap that being wasteful with it isn't really disincentivized
I think another way of phrasing that is usage is correctly incentivised. In the example you give, the value to debugging is more than the cost of storage — and even if that’s not the case it’s so low-priority that it might not even be on your list of priorities anymore!
That literally means that it’s worth your limited, valuable time to do something else.
About a decade and a half ago I worked on a large data migration project at a FAANG. Multi-exabyte scale, many clusters across many countries. Once everyone was moved the old storage platform wasn't completely empty, because the number of migrations was large and users were (naturally) more focused on ensuring their data was in place and available on the target platform rather than ensuring every last thing was deleted on the legacy platform. We weren't initially concerned about it because it would all get deleted when we turned down the old setup.
As we were gearing up to declare victory and start turning down the several dozen legacy storage clusters someone mused that given some users were subject to litigation holds -- not allowed to delete any data -- that at least some of the leftover data on the old system might be subject to litigation hold, and we'd need to figure that out before we could delete it or incur legal risk. IIRC the leftover 'junk' data amounted to a few dozen petabytes spread across multiple clusters around the world, in different jurisdictions. We spent several months talking with the lawyers figuring that out. It was an interesting dance, because on the one hand we were quite confident that there was unlikely to be anything in the leftovers which was both meaningful and not migrated to the new platform, while on the other hand explaining that it wasn't practical to just "go and look" through a few dozen PB of data. I recall we ended up somewhere in between, coming up with ways to distinguish categories of data like caches and working data from various pipelines. It added over six months to the project, but was quite an interesting problem to work through that hadn't occurred to any of us earlier on, as we were thinking entirely in technical terms about infrastructure migration.
That does sound very interesting. Any insights on what would you do differently if you had to do it again? Any way to accelerate things now that you know the pain or do you think it's quite unavoidable and "legal Time"?
Doing it in parallel, which of course is only an option if you know about it. And templating that into a general approach and having legal folks sign off on that.
A gross underestimation, IMO. When I was in big data, fewer than 5% of data written was ever touched again, and only a single digit number of our large customers (out of tens of thousands) actually made real use of their "big data", and created most of the load. That's the trouble with "checkbox driven development" - 10 years ago you were required to have a "big data strategy" for anyone to take you seriously, even if your strategy boiled down to just ETL-ing a bunch of crap you're never going to need into the cloud and never touching it again. Now I'm in AI, and the same thing is happening to AI. It's great if you're selling shovels, so to speak, but not so great if you plan on selling them for an extended period of time.
This, by the way, has implications on storage systems design. You want something that's cheap yet dense to encode, potentially at the slight expense of decode speed. Normally people really lose sleep about decode speed first and foremost, which, while important, does not minimize the overall resource bill.
It's a problem for us when it comes to the GDPR and rights to be forgotten; companies will say they store your data carefully, will say they have shown you everything they have, will say they delete it, but "the company" in aggregate has no idea there's a thousand SharePoint sites and ex-employees mailboxes and filestores, and copies of old filservers from before a migration, and test databases containing copies of real data from a half-abandoned project where new management fired the contractors and then never got around to hiring new ones.
The right to be forgotten was never a practically enforceable right to begin with. It’s a right that burdens others with perpetual effort and cost, unlike most rights, which only require others not to do things. I’m not sure why Europeans are so obsessed with it.
Storage being cheap enough that it's not worth policing doesn't seem very consistent with it being expensive enough to include much energy use (what I assume the "destroying the environment" hyperbole is referencing).
> Having to deal with senior managers has always been the most unsavory part of my job, because when it comes to technology, these managers exist on a whole other level of stupid vanity and narcissistic pursuit of their own selfish agendas.
Whoever sent this dude made a mistake. People who don't share your worldview need to be persuaded, not insulted! Some dude stomps in, thinks all the snaps in the cloud are crap, things the big bosses are stupid for not instantly deleting the pictures they saved into the cloud.... and then what? Download Lisp? Thought we got over this, pal.
Make the cost of operating data centers reflect the damage to the environment and see how quickly people optimize. I don’t know how damaging storage is, but that’s the only way to make a difference.
The cost to operate a data center is already, to a close approximation, the cost of its electricity. The data center operator already pays for every joule.
Data centers used ~4% of USA electricity last year, about .15 quads, while the chemical industry used 10 quads, virtually all in the form of fossil fuel. If you are going to start reflecting the externality of energy consumption in the price of goods, information sector will probably not need to adjust anything, while the chemical sector will be fundamentally impaired.
> We’re destroying our environment to store copies of copies of copies of stuff we have no intention of ever looking at again
While I agree that most of the stuff in data centers is probably crap, but that's because most of everything people do is crap. That's not for me to decide though, people save things because they find value in them. Most of what has value to another person won't have value to you. Most of what people treasure in their life is thrown away after they die because nobody wants it, even their closest family members. Who gets to tell everyone the bad news, that objectively their memories are trash and they don't have a right to keep them anymore? Gerry Fuckin' McGovern?
Secondly, we aren't destroying the environment for any of this. Data centers use like 5% or less of the overall electricity use. It's a lot, but we don't have to put datacenters in random locations, we can (and do) put them where electricity is cheap. That generally means that the 5% of electricity used for data centers, kwh for kwh isn't as impactful as an average kwh of end use. Large companies like Meta and Google claim to have zero net carbon by obtaining offsets. So in general we aren't "destroying the environment" to store copies of photos.
I couldn't agree more - the data stored in data centers have "some" value, otherwise people wouldn't pay to host it. This author anecdotally comments about his experience while pearl clutching about the environment, typical.
> because when it comes to technology, these managers exist on a whole other level of stupid vanity and narcissistic pursuit of their own selfish agendas
Regardless of what you think about the article. This rings so true at many Fortune 500 companies.
The number of times I have seen teams work through pointless bullshit to push some meaningless objective for the company. Just so the middle manager (aka “Director of SVP of X product of Y branch”) can get a bullet point(s) in the quarterly “all hands”.
Oh and those 10 developers/off shore people that were just hired? It was all to pump his/her “head count” number to get to the promotion to next grade/level.
Then when that person gets promoted, those people get scattered throughout the firm or just let go.
> We’re destroying our environment to create and store trillions of blurred images, half-baked videos, rip-off AI ‘songs’, rip-off AI animations, videos and images, emails with mega attachments, never-to-be-watched-again presentations, never-to-be-read-again reports, files and drawings from cancelled projects, drafts of drafts of drafts, out of date, inaccurate and plain wrong information, and gigabytes and gigabytes of poorly written, meandering content.
Storing the files for Mr. McGovern’s website requires plastics, metals, power and physical space, yet I assume he believes that environmental effect is worthwhile. Who is he to decide for others that their choice to pay for the storage of data is not equally worthwhile to them?
That’s the beauty of a price system: each of us gets to decide what we will buy, and what we will not buy.
Now, perhaps his argument should be that the price of storing digital data does not adequately reflect the true cost. Perhaps there are unaccounted-for externalities. If so, then he should make that argument, perhaps arguing for a tax to align prices with costs.
Someone else might argue that data is a liability as well as an asset. That’s another argument he could make.
But haranguing folks for spending their money in ways he doesn’t like doesn’t seem likely to produce the outcome he appears to wish.
It's pretty simple: sorting through the data to determine which 90% is crap is more expensive and uses more of our scarce resources than just storing it all.
I agree with the general sentiment here, but don't like the examples. 200 photos per person per year isn't very much! That is all fine.
What really bloats things out is surveillance (video and online behavioral) and logging/tracking/tracing data. Some of this ends up cold, but a lot of it is also warm, for analytics. It bloats CPU/RAM/network, which is pretty resource intensive.
The cost is justified because the margins of big tech companies are so wildly large. I'd argue those profits are mostly because of network effects and rentier behavior, not the actual value in the data being stored. If there was more competition pressure, these systems could be orders of magnitude more efficient without any significant different in value/quality/outcome, or really even productivity.
When traveling, a funny thought I always have is watching other tourists take the same photos, from the same exact location, knowing it will be backed up to iCloud. I can't help but imagine how much disk space is taken up by duplicates of the exact same photo.
For a fun example, every time I'm on the Las Vegas strip I see dozens of people taking videos of the Bellagio Water Show.
There are 30 shows per night, if 50 people take videos in 4K 60fps (default on new iPhones), that's around 60 GB of data per show or ~600 TB per year of just videos of the Bellagio Fountain Show!
145 comments
[ 3.2 ms ] story [ 247 ms ] threadI have cleaned up dozens of product databases in cost management efforts and have found anywhere from 50-99% of data stored in product databases is crap, because they are not well managed and any single mistake can lead to a huge outsized impact on storage.
What to log all those http requests for just a day? Might as well turn that on for all time...
"I need an email when this happens.. and when this happens."
The requests are endless and I'm convinced there are people who if they could would do their entire job from their inbox and get everything and anything an application can do via email.
The insidious problem is that it never solves anything. "I didn't get the email!" is a constant refrain (yes they did, they always did). "Oh someone didn't do the thing so can you add this to the email too." and so on.
It is such an abused tool.
That sounds like a reasonable goal for a whole lot of job duties. And yes some entire office jobs. (Excluding some direct human communication but a lot of jobs already have too much of that in situations that could have been an email.)
> "I didn't get the email!" is a constant refrain (yes they did, they always did).
Well having to manually check wouldn't improve that, would it?
There must be thousands of copies of that email sitting in inboxes say: Job X ran successfully @ 04:30.
There is really no sense at all in the article's claims that "we are destroying the environment" to do x y and z thing the author whines about. We are destroying the environment to drive a Dodge Ram to the Circle K to buy a 52oz Polar Pop. Information sector doesn't even show up on top ten lists of things that are destroying the environment.
Back in the day Exchange offered SIS but in 2010 they ditched it. It's plainly not effective any more. Even regarding the OP' "the jpg signature logo" - it's a part of multipart in the message, not a separate file.
And one more thing - you can't just turn the dedup and be dandy, now you need to check against the hashes to determine if this chunk is unique or you already have it. And with TBs of data you need TBs of hashes to check. Until you have like 99% dedup efficiency, ie 99% of your incoming data is literally the same data you already have - it doesn't worth it.
https://techcommunity.microsoft.com/blog/exchange/dude-where...
Or if everything in the year 20XX gets pushed into using E2E encryption, since that's pretty much antithetical to deduplication.
I would think that email providers tend to not offer E2EE just because it fundamentally isn't practical with email. Providers like Proton try to do it, but it works only if you talk to someone else on Proton (at which point you may as well do it on Signal, which has better encryption).
not saying that it's infallible but yeah we need some chain of audible at least 1 or 2 layers deep.
On a related note, probably a similar percentage of people claim on their car insurance. If only the rest realised they had "crap insurance" and were paying for nothing, they could save so much money!
This is obviously sarcasm, but I think it's important to remember that much of the data is stored because we don't know what we will need later. Photos of kids? Maybe that one will be The One that we end up framing? Miscellaneous business records? Maybe those will be the ones we have to dig out for a tax audit? Web pages on government sites? Maybe there will suddenly be an interest in obscure pages on public health policy if a global pandemic happens.
Complaining that data is mostly junk is not a particularly interesting conclusion without acknowledging this. Is there wastage? Yeah sure, but accuracy on what needs storing is directly traded off with time spent figuring that out, and often it's cheaper to store the data.
Removing duplicates isn't that complex already, and these tools exist so anyone can try it. It's just a truely grueling process.
With that said, I’m surprised Apple hasn’t implemented a feature to group/bundle successive photos beneath what is determined to be “best.”
[*] Note: successive photos being separate shutter taps/actuations (potentially several seconds apart), where bursts are continuous (generally as fast as hardware allows, ms apart).
But if those orders aren't there, shit hits the fan. PCI compliance audits fail. The ability for customers to reconcile their charges with their purchases breaks. In that 0.01% of cases where the order was fraudulent, placed by mistake, or just didn't have what the customer thought it had in it, not having that data makes the order processor read as, if not malicious, at least incompetent.
The real question is, how much data do we need to store inefficiently, in a way that uses a lot of power and space?
Cloud services make difficult and sometimes even byzantine processes for deleting stuff, and it's often impossible to operate en masse in order to clean up swaths of stuff quickly and efficiently. It's in their interest to retain everything at all costs, because more used storage can mean more profits. Cloud services also profit from unused storage, because if they're charging $20/year to 100,000 users who use 2% of their storage space, ka-ching!
It irks me to this day that standard or even advanced filesystems don't include "expiration dates" or "purge dates" on file objects. Wouldn't it be logical, if an organization has a "data-rentention policy" that mandates destruction after X date, that the filesystem simply purges it automatically? Why does this always get delegated to handmade userland cron jobs? Moreover, to my knowledge, nobody is really interested in devising a way to comb through backup media in order to [selectively] destroy data that shouldn't exist anymore. Not even the read-write media!
Google is now auto-deleting stuff like OTP SMS messages. I'd love it if auto-delete could be configurable account-wide, for more than just our web histories and Maps Timeline and stuff. Unfortunately, to "delete" cloud data means it still exists on backups anyway. But without deleting data in your cloud account, it becomes a juicier hacker target as it ages and accumulates personal stuff that shouldn't fall into the wrong hands. Likewise for any business, it behooves them to delete and destroy data that shouldn't be stolen. At least move it offline so that only physical access can restore it?
I will say that modern encryption techniques can make it easy to "destroy" data, simply by destroying the encryption keys. You can quickly render entire SSDs unreadable in the firmware itself with such a command. Bonus: sometimes it's even done on purpose!
But even deleting data presents a maintenance cost. So if 90% of an org's data is indeed crap, then 90% or more of your processing resources are going to be wasted on sifting through it at some later date. Imagine when your file formats and storage devices are obsolete, and some grunt needs to retrieve some record that's 30 years old, and 90% of your data was always crap. That grunt is hopefully paid by the hour. We really had this happen at a few of my jobs, where we had old reel-to-reel backup tapes and it was difficult enough to load the data into a modern SunOS machine.
https://m.xkcd.com/1683/
For the expiration dates most modern file systems have the concept of arbitrary extended attributes per file. It's quite easy to add meta data like this yourself.
It's like gym memberships, ISP/telco service bundles, and amenities at your apartment complex. Anyone not using every possible service is wasting money, but it's impossible to purchase a bespoke service, so essentially everyone will waste money because they're chipping in money for services that someone else uses more than they do.
Here at home I don't ever use the gym, the racquetball courts, the doggy-doo supplies, the laundry room, or a parking space, and yet my rent (everyone's rent) includes upkeep for all of those things. I'm subsidizing all my neighbors and all the wear-and-tear damage they put on those common amenities. Likewise, everyone who's paying $24/year for storage, or any business that purchases big multi-terabyte storage media, they're paying for unused storage space and giving profit. It's practically impossible to rightsize your storage media, and you never want it undersized, and you can't simply shrink them and reclaim the resources you invested, you just keep adding on new ones and replacing the malfunctioning ones. So nearly everyone always owns or rents more space than they can realistically utilize.
Furthermore, you'll notice that I specified "automatic" destruction of data by expiration date. Of course it is trivial to tag any arbitrary file with arbitrary metadata, but the challenge is to create a filesystem that executes automatic data purges on schedule, rather than pushing it into a rickety old handmade cronjob in userland. I've never ever seen a filesystem with such a feature, nor does it seem that anyone's interested in doing so.
And here I thought that computers were useful for automating business logic and easing the burden on human effort. And this is me, manually sifting through emails and photos in order to manually delete each one with 3 dialog boxes intervening. It takes hours, days, weeks.
For personal data, the concept would be simpler but still has requirements like say tax records need to be kept 7 years.
There is no clear path forwards. Perhaps regulation could be a solution but I doubt it would turn out nice.
Except when the app writes files outside of its own space, which is meant for stuff that should stay (like pictures).
Of course, some apps store the pictures in their private space, and you lose the pictures when you remove the app. And some apps write crap in the shared space. But that seems like a fundamental limitation (even if it was done at the filesystem): how do you make sure that the apps/software you use is doing it right?
"File-type strictly associated with app" is good for security and it's good for people's mental organization. Unfortunately you're referring strictly to local file storage, I assume, and practically nothing on my phone is in local storage but "in the cloud". The only local storage I use is for ringtones and certain audiobooks/prayers which I enjoy listening whether or not I'm connected to a network.
It would seem that data centers and cloud services are a long way from "file types strictly associated with app" and also the sort of behavior you describe would be really horrible and undesirable in a typical data-center setting. I mean, I can't even imagine how it'd be implemented, given that data storage is often remote or at least decoupled from the "application server", and it's definitely never a 1:1 relationship between those abstracted components.
Can you imagine needing to "delete the /bin/tar application" in order to purge every file ending in ".tar" or ".tgz"?
But it's true that the new mobile OS paradigm has advantages for cleanup. If you've got a 3rd-party app and you've created some big crap files with it, and you uninstall that app, then it's a clean sweep when it takes along all the crap and you don't need to worry about lingering loose ends, or files you simply can't open, because their handler's gone.
Scoped storage is way more secure; it's also wonderful on my Androids that I can selectively grant access to files and media, rather than giving "the keys to the kingdom" to all these Social Media apps. I definitely never want to post my bank screenshots to Facebook and the possibility no longer looms!
These days I am sort of nonplussed when a web app demands an all-access pass to my Google Drive, because permissions can be appropriately scoped now, and the app can play in its own Drive "Sandbox" without having access to my spreadsheets and medical records. I just wish that there were some really good "cleanup" apps. Remember those "cleanup" programs that would come with antivirus for your PC, or Regclean, or whatever freeware utility would purport to liberate your 10MB HDD? Those were good times.
It's the same with social media posts--I often wish to really scrub my account, delete all posts and all comments and basically all activity, and while that's possible on Facebook it's arduous and it takes me weeks to scrub the major stuff after 4-5 years. But I only think of it on Shrovetide Monday or Fat Tuesday, and then I spend 3 weeks in Lent doing scrub, scrub, scrub. Sometimes the mouse/keyboard gestures are more repetetive & physically demanding than cleaning the toilet. Way too difficult, and that's by design.
Google or apple can put a big button delete everything in their phones/accounts but then some prankster will do it to a family member and this gives bad PR. Let's be pragmatic.
> It's in their interest to retain everything at all costs, because more used storage can mean more profits.
As an user, I get the reverse. When I had local NAS then I dumped anything and everything - assuming.. this costs not much. Will clean up later. Once I moved to cloud that changed to If I put crap then it will cost me money! Keep it clean
Once upon a time Google had given generous unlimited to all education and workspace. They stopped it in the last 2 years and you can see most educational and companies are running a tight ship.
Due to backup costs in our organisation we are forcing people to use max of 100GB for emails.
> advanced filesystems don't include "expiration dates" or "purge dates" on file objects. Wouldn't it be logical, if an organization
Totally agree.
Outlook email service has some kind of keep only the last newsletter from this service etc.
> auto-delete could be configurable account-wide, for more than just our web histories
- features are designed with majority in mind - most users have some sort of nostalgia for reading or keeping old emails SMS etc
This is indeed the critical question, and it's far from being trivial.
One issue we all hit is moving the data from the higher tier storage to the cheaper and more efficient one, which requires sync and paying for the transfer most of the time, but also handling two separate access and authorization process, backup and recovery system for data that absolutely needs to be accessible for the few years of legal retention, and can/must completely disappear afterwards.
In most orgs I've seen the cost of going through all that complexity is just not worth it, compared to "just" paying for the higher tier storage for the few years long lifetime of the data.
The winning ticket ID, e.g. a "0x-636e-7461-4e7b", only makes sense in the context as one among the entire cohort of contestants; I can make one up like I did, but I can't walkout with the payout unless the rest of the lottery didn't exist.
Statistically, philosophically, technically, and all sorts of *-cally speaking, is the 2% data, the winning ticket datum, even data?
But there's an important piece there that is about data that should not have been stored in the first place. All the big data bullshit made people believe every single piece of data you could possibly store from a user is data that you should store and this is both a huge waste of resources and a huge liability, because now a data leak with useless to you PII could completely bankrupt your business.
Look through the pictures with your kids (and with your kids) and pick out the best ones. Delete the rest.
1. Bulk favorite photos in Google Photos for long term archiving
2. Set a retention policy of months/years for photos based on metadata.
3. Have a UX to quickly sort the week or months photos via a swipe left/right UX.
That random photo of your messy living room might be trash right now, but beloved in 20 years when you want to remember your old home.
Storage is cheap, very cheap.
Photos of kids are obviously not part of that "crap" (even if we have too many of them and it would be worth triaging for our own sake).
The question is: what makes for most of that data? Is that all business records, or is that storing stuff because we can? I've worked in multiple startups, all were tracking users as much as they could. Adding tools that collect data is easy, and storing that data is cheap. "We may need it later". Never needed any of that crap, and it was invading our users' privacy.
In any case, I think the article raises a good point: it's so cheap to store crap that we don't even think about it. And it's bad for the environment. Just like it's so cheap to take a plane that we don't think about it, even when taking the train takes almost the same time.
>The Cloud is what happens when the cost of storing data is less than the cost of figuring out what to do with the crap
But I think that's wrong. The actual issue is that you often can't figure out "what to do with the crap" because the difference between useful data and crap data is determined at the point in time when you need it, not when you store it.
I'm relatively careful with deleting data, but even so, there were countless instances where I thought something was no longer needed and deleted it, only to figure out that, a month later, I needed it after all.
Also, the article has a few anecdotes like this:
>Scottish Enterprise had 753 pages on its website, with 47 pages getting 80% of visits
But that is completely orthogonal to the question of what is "crap." The fact that some data is rarely needed does not mean it's crap. If anything, the fact that it is rarely needed might make it more valuable because it might make it less likely that the same data can be found elsewhere.
It’s a trade off especially with digital where there’s not that much cost associated with holding onto things relative to the physical world. My whole house is basically in storage because of a kitchen fire and I’m planning to be pretty selective about what I move back in.
That's true, but things get easier to find over time. I have thousands of digital photographs from the 1990s that I burned on CDs and then copied to a NAS later. Today, for the first time, they're actually searchable because they're now indexed in Immich. So they're sorted by the person in them and searchable by their contents.
If I had culled the photos back then, I would have lost many photos that have become searchable (and thus valuable) only recently.
Data that ends up in that sea of crap is very often poorly labeled.
Data that you cannot find again is useless.
If you take more than 3 minutes to find a picture you wanted to show me, it doesn't deserve to be showed anymore.
This is the basic argument that every hoarder uses to justify their hoarding.
Perhaps I am missing something, but these examples all sound like candidates for _offline_ storage, for which no third party custodian or data center is required.
The price of large capacity NVMe SSDs continues to fall.
The amount of energy, the resource requirements, not to mention the environmental and community impact, of running a car insurance company are miniscule in comparison to that of running a data center.
In between is a vast gulf where your only good options are disks that you have to occasionally spin up to check integrity, have the disk trigger a data refresh if it's an SSD, and replace any disks that failed and rebuild their data from redundancy (raid or whatever you prefer). HDDs die even when off [1] and flash storage in SSDs is only rated to hold data for three years without power.
Sure, you can roll this yourself, but it can easily go wrong. Easier to either keep the disks spinning or pay someone with a tape archive (e.g. AWS deep glacier)
1 https://www.tomshardware.com/pc-components/storage/twenty-pe...
We are using 4-6 times as much storage as we need to, and these are often not small files (on the order of 100 MB - 5 GB, several dozen times a day) but fixing this overuse is so far down the priority list that I don't think it survived the great Jira purge of mid-2024.
I think another way of phrasing that is usage is correctly incentivised. In the example you give, the value to debugging is more than the cost of storage — and even if that’s not the case it’s so low-priority that it might not even be on your list of priorities anymore!
That literally means that it’s worth your limited, valuable time to do something else.
As we were gearing up to declare victory and start turning down the several dozen legacy storage clusters someone mused that given some users were subject to litigation holds -- not allowed to delete any data -- that at least some of the leftover data on the old system might be subject to litigation hold, and we'd need to figure that out before we could delete it or incur legal risk. IIRC the leftover 'junk' data amounted to a few dozen petabytes spread across multiple clusters around the world, in different jurisdictions. We spent several months talking with the lawyers figuring that out. It was an interesting dance, because on the one hand we were quite confident that there was unlikely to be anything in the leftovers which was both meaningful and not migrated to the new platform, while on the other hand explaining that it wasn't practical to just "go and look" through a few dozen PB of data. I recall we ended up somewhere in between, coming up with ways to distinguish categories of data like caches and working data from various pipelines. It added over six months to the project, but was quite an interesting problem to work through that hadn't occurred to any of us earlier on, as we were thinking entirely in technical terms about infrastructure migration.
https://en.wikipedia.org/wiki/Sturgeon%27s_law
This, by the way, has implications on storage systems design. You want something that's cheap yet dense to encode, potentially at the slight expense of decode speed. Normally people really lose sleep about decode speed first and foremost, which, while important, does not minimize the overall resource bill.
Didn't Facebook start to move most of their least-used data onto optical arrays a long time ago?
Whoever sent this dude made a mistake. People who don't share your worldview need to be persuaded, not insulted! Some dude stomps in, thinks all the snaps in the cloud are crap, things the big bosses are stupid for not instantly deleting the pictures they saved into the cloud.... and then what? Download Lisp? Thought we got over this, pal.
WORSE IS BETTER.
P.S. do not erase our porn. WORSE IS BETTER.
Data centers used ~4% of USA electricity last year, about .15 quads, while the chemical industry used 10 quads, virtually all in the form of fossil fuel. If you are going to start reflecting the externality of energy consumption in the price of goods, information sector will probably not need to adjust anything, while the chemical sector will be fundamentally impaired.
While I agree that most of the stuff in data centers is probably crap, but that's because most of everything people do is crap. That's not for me to decide though, people save things because they find value in them. Most of what has value to another person won't have value to you. Most of what people treasure in their life is thrown away after they die because nobody wants it, even their closest family members. Who gets to tell everyone the bad news, that objectively their memories are trash and they don't have a right to keep them anymore? Gerry Fuckin' McGovern?
Secondly, we aren't destroying the environment for any of this. Data centers use like 5% or less of the overall electricity use. It's a lot, but we don't have to put datacenters in random locations, we can (and do) put them where electricity is cheap. That generally means that the 5% of electricity used for data centers, kwh for kwh isn't as impactful as an average kwh of end use. Large companies like Meta and Google claim to have zero net carbon by obtaining offsets. So in general we aren't "destroying the environment" to store copies of photos.
Regardless of what you think about the article. This rings so true at many Fortune 500 companies.
The number of times I have seen teams work through pointless bullshit to push some meaningless objective for the company. Just so the middle manager (aka “Director of SVP of X product of Y branch”) can get a bullet point(s) in the quarterly “all hands”.
Oh and those 10 developers/off shore people that were just hired? It was all to pump his/her “head count” number to get to the promotion to next grade/level.
Then when that person gets promoted, those people get scattered throughout the firm or just let go.
It’s truly just weaponized incompetence.
Storing the files for Mr. McGovern’s website requires plastics, metals, power and physical space, yet I assume he believes that environmental effect is worthwhile. Who is he to decide for others that their choice to pay for the storage of data is not equally worthwhile to them?
That’s the beauty of a price system: each of us gets to decide what we will buy, and what we will not buy.
Now, perhaps his argument should be that the price of storing digital data does not adequately reflect the true cost. Perhaps there are unaccounted-for externalities. If so, then he should make that argument, perhaps arguing for a tax to align prices with costs.
Someone else might argue that data is a liability as well as an asset. That’s another argument he could make.
But haranguing folks for spending their money in ways he doesn’t like doesn’t seem likely to produce the outcome he appears to wish.
What really bloats things out is surveillance (video and online behavioral) and logging/tracking/tracing data. Some of this ends up cold, but a lot of it is also warm, for analytics. It bloats CPU/RAM/network, which is pretty resource intensive.
The cost is justified because the margins of big tech companies are so wildly large. I'd argue those profits are mostly because of network effects and rentier behavior, not the actual value in the data being stored. If there was more competition pressure, these systems could be orders of magnitude more efficient without any significant different in value/quality/outcome, or really even productivity.
For a fun example, every time I'm on the Las Vegas strip I see dozens of people taking videos of the Bellagio Water Show.
There are 30 shows per night, if 50 people take videos in 4K 60fps (default on new iPhones), that's around 60 GB of data per show or ~600 TB per year of just videos of the Bellagio Fountain Show!