I think about this problem often in life because of my tendency to procrastinate. Only the long term effects that cause me serious paranoia get any real attention (failing to meet social expectations, major health issues, large-figure finance problems). It’s absolutely necessary to define the constraints before they define you. This requires a lot of forward thinking and effort but considering how little time we get on this earth… worth it
I recently ran `df` and was surprised to see that I'd used more than half of the two terabyte SSD in my laptop. So, being apparently atypically proactive, my first action was delete all of the target directories in my Rust projects and was equally surprised to see that it freed up nearly 10% of my disk usage (~200 GB)!
On a SanDisk portable SSD I own, the cluster size is even 4MiB. If you put a node modules directory in there with thousands of tiny files, it can "take up" several GiB of space.
And i am wondering what do people even fill their storage with, and why do they need so much. In my case it is ~200-300 out of 1TB on 2 laptops, 37 out of 128gb on phone. And i don't even actively manage it.
Yes, I always use `du . -h 2>/dev/null | sort -k1hr | head -nN` where that last N is however many directories I want to show. I run this regularly and have for a very long time precisely because of the issue that the post talks about.
If you're on MacOS and suspect a lot of space is taken by multiple copies of the same files, use fclones to turn them into APFS clones. (You can also use this on Linux, as ext4 supports hard links, but only if you're sure the duplicates won't diverge.)
I have less than one terabyte of disk space, and less than 10% of the full capacity is in use, so it is closer to being empty. (The hard drives they sell have more capacity than I require.) And, there are files that I do not use (some of which come with the operating system, such as PulseAudio).
It's not just hard drive problem. If you buy a new storage rack at your home, it will be full in no time. No matter how many racks you buy, you will always find things fill those racks. This is like rain water filling up the lakes. if there is a lake, it gets filled. Or maybe it is called a lake because it gets filled. So, one can say that the disks create or cause the data. If there is no disk, there is no data.
Mine is mostly full and it's not stuff I've saved but stuff software has stored there without asking me. It's quite hard to figure what's going on. I once got a message from Chrome saying it couldn't go on because it had stored like 200gb for websites, would I like to delete it all, and I said yes and noticed no difference.
It's annoying because I'd rather store say family photos than have it full of random junk that means nothing to me. Dunno the answer - better os control? Partition the drive?
25 comments
[ 0.24 ms ] story [ 8.0 ms ] threadOn a SanDisk portable SSD I own, the cluster size is even 4MiB. If you put a node modules directory in there with thousands of tiny files, it can "take up" several GiB of space.
On Linux/macOS you can use `du` from the command line (or ask an AI to make a convenience wrapper for you).
Thats a pretty bold claim, but is it really that much faster?
https://www.portablefreeware.com/index.php?id=150
https://github.com/Byron/dua-cli
If you're on MacOS and suspect a lot of space is taken by multiple copies of the same files, use fclones to turn them into APFS clones. (You can also use this on Linux, as ext4 supports hard links, but only if you're sure the duplicates won't diverge.)
Delete is expensive. "Am I positive I'll never need this again?"
As a consequence, "cheap" actions (saves) accumulate, "expensive" ones (deletes) accrue until cheap becomes expensive as space becomes tight.
Ultimately, options become:
- Find criteria for rapidly assessing what data can be deleted.
- Purchase cheap archival storage and migrate old data to that.
- A media or device crash induces loss of arbitrarily-selected data.
It found 114GB of Hugging Face models, 17GB worth of models in an obscure app I tested once. In places I would've never thought to look.
I'd never have never found these myself.
Problem solved. For now!
Filling an ssd creates the one situation where nand wear out is a problem.
I bought 16TB disk because I needed that much storage.
If I were using, say, only 10TB, I would be wasting 6TB.
I can expand my LVM as much as I can. But I definitely want my disk to be full. Same goes for my RAM.
And I want my CPU ide time to stay as close as possible to 0% and my load average as close as possible to my no. of CPUs.
Anything else would mean oversizing and overspending.
It's annoying because I'd rather store say family photos than have it full of random junk that means nothing to me. Dunno the answer - better os control? Partition the drive?