Sounds ideal for large arrays for video post use. We used to string 16 drives together to get speeds of what a modern m.2 SSD can do as a single drive. Only, the arrays gave you the capacity to do useful things rather than the toy-like capacity of 2-4TB SSDs. Doubling the speed of spining rust might be interesting when building RAID-6 volumes for edit bays
But even if in "normal server workloads" it's comparable to older drive you still get the benefit during things like RAID recovery where it very much will be sequential operations dominating. So it's an improvement which can really matter for unlikely but very critical server operations.
Possibly up to double (480MB/sec) that soon for large sequential read workloads. Seagate has their multi-actuator Exos 2X14 Mach.2 drives which should start hitting some level of channel availability within the coming weeks.
For video editing, ETL and other read-heavy workloads, they have a point.
You've been out of the loop for a while. Certain HDDs have been able to achieve such speeds for about a decade, if not more. Most of their (slow) gains over the last 10 years have been in capacity.
I actually ran into this last year. We ordered about 2 thousand 8TB HDDs last year for a Ceph cluster and had an agreement with our server vendor that they are not supposed to substitute the HDD model. Well, seems like they ran out and a small fraction of the drives were of a different, older model. The performance difference was 2.5x for our workload:
Older model: 670 IO/s and 2.9 ms average latency
Newer model: 1680 IO/s and 1.1 ms average latency
We got the vendor to send us new drives and shipped the older model back to them.
No. Both models were PMR. I tried to find out more about what the differences are from the manufacturer, but didn't get far. Perhaps someone here can shed some light.
Older drive:
model number: MB8000GFECR
SATA Version: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
ATA Version: ACS-3 T13/2161-D revision 5
Newer drive:
model number: MB008000GWWQU
SATA Version: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
ATA Version: ACS-4
My guess at the time was that the newer drive had a deeper command queue, but I'm not actually sure, and I think both drives had the same amount of cache.
I'm not seeing any especially obvious differences between those models (aside from the ones you listed). Were the drive versions mixed in alongside each other behind the same storage controller? If so, I would wonder if perhaps the controller's handling of different SATA versions played a role.
It's possible. Yes, the drives were sprinkled around in the inventory, and we had some servers where there was a mix of old and new drives in the same box, behind the same controller.
We use the controller in HBA mode, so the controller shouldn't be doing anything fancy, but you might be right.
I've seen controllers in HBA mode pull some oddball performance shenanigans when presented with differences as small as running different drive firmware versions across connected disks, so it really wouldn't surprise me. SATA drives have a pretty bad reputation for firmware releases introducing unexpected and undocumented behaviors. I used to be on the customer success team for a hyperconverged distributed storage product, and such things were the bane of our existence. Over the years the inventory of product-supported SATA devices shrunk considerably as a.) more and more quirks were discovered in products and b.) flash prices continued to drop to the point that spinning disks were in less and less demand, putting downward pricing pressure on SAS devices.
These are both Seagate Exos drives, one is a generation newer than the other. I'm pretty sure both have the same number of platters, thus same number of heads.
And yes, HP changes the firmware and model numbers... super annoying.
230MB, 100MB, or 50MB per second doesn't make that big a difference for anything I'd use a HDD for. What I want is lower pricing. The 12TB drive I bought two years ago hasn't changed in price. The 8TB drive I bought four years ago is $30 more expensive now than it was then.
All kinds of "legacy" products go up in price-- RAM a couple generations back, etc.
SSDs have won. Spinning hard drives are going to ship fewer and fewer units. The economies of scale are not going to improve ever again and are already getting worse. Further, the market just doesn't care about paying for capacity for the most part.
We peaked in 2010 with 650 million drives/year. Current unit counts are about one third of that. Compare to SSDs with about 400 million units/year.
If you go used on LTO you can get a drive a few generations back for not insane prices, but it's still not entirely competitive with additional spinning rust.
This is my pain point. For those of us who are aware the cloud exists and would rather do tape anyway, why on earth is a recent vintage tape drive so expensive?
It's been a year since prices recovered from the earlier madness & got back to where they were. It's not totally stable, but prices as I see them have been going slowly down since.
I personally have some Exos 16TB's. There's a decent price history available[1] reaching back to mid-2019. Make sure to tick on "3rd party new". These aren't going to be the best prices available in many cases, but it's reasonably indicative.
I always buy enterprise drives - much higher reliability and lifetime and usually I have found they benchmark higher. They are also way heavier and almost always use better bearings on all surfaces. Some people complain they are super loud but the three types I have randomly purchased over the years have been just as quiet as any other spinning disk (as in I do not notice the sound at all once its inside my desktop)
For 1 drive, the differences are perhaps not much. When you have a cluster with 2k HDD drives, the difference in handling both squential and random workloads (see my other comment) can add up.
A 2.5x difference in IO/s adds up quickly.
As for HDD prices, neither 12TB nor 8TB were top of the line at the timeframes you mention. If you buy largest available drives, or a model lower, the prices have been going down. Not as much as we all would like, but overall the trend has been moving in the downward direction.
The helium filled drives also appear to be much, much more reliable. Only thing that worries me about them is that if you need to do disaster recovery you are pretty much out of luck unless you can get the original manufacturer to do it or find a highly specialized place.
> because of course disk vendors quote everything in the smaller SI units).
SI prefixes should be (and are) the rule rather than the exception. It only made sense to measure ram in base-2 units because they are manufactured according to base-2 addressing.
HDD’s have never used base 2 addressing.
Another example is network media. Serialized data transfer speed is measured in {K,M,G,T}b/s (bits) or B/s (Bytes) where a Gigabit is exactly 1000000000 bits and Gigabyte is exactly 1000000000 bytes.
In the early days of computers ram was extremely scarce and expensive. It was such a limitation that it was usually the very first question you asked about a computer’s capabilities.
I remember in 5th grade asking a fellow student who’s parents had just bought an Atari 800 “How much ‘K’ does it have?”
This intense focus on ram may have led people to assume that it’s odd nomenclature would apply to anything computer related.
Base 2 is used in the world's most common desktop operating system(s) so it's no surprise that people get confused. They buy a hard drive that's supposed to be 1TB and when they install it their computer says it's 0.9TB, that's surprising to those not in the know.
Despite hard drive manufacturers using correct units, people still blame hard drive manufacturers for their disappointment.
Android and desktop Linux also use base-10. It's a Windowsism at this point. Though some Android vendors specifically use the wrong unit in their software.
No, many things related to 'computers' are determined by physics and chemistry. In any case, 'computer' is being used loosely (and poorly) here, which is why I have been putting it in single quotes. 'Computer' can mean many things.
If you look at register size, you'll believe that different processors use different 'bases', depending on whether they're 8-bit, 16-bit, 32-bit, or 64-bit...
Computers access things in powers of 2 because we settled on a byte being 8 bits. We then proceeded to doing things in even multiple of bytes. A cache line, row in DRAM, registers, and a page of memory in your MMU are all some even multiple of bytes. And back in the day when we couldn't put 300 million transistors on a chip, because the systems work in binary, it was also quite useful to be able to do things like calculate the address of a pixel in the frame buffer by simply shifting the row index and adding the column index, meaning that something like 1024 pixels wide was much simpler to implement than 999 pixels wide.
There's lots of reasons why computers so often use powers of two and thus K being 2^10 instead of 10^3. But the word measure is probably not the best.
HDDs, RAM, network speed were base-2 (e.g. 1TB = 1024GB) for most of my life until SSDs became common and even Google now says "1 Gigabyte = 1000 Megabytes" to my exasperation.
For hard drives it was complicated, since their size is a function of the number of cylinders, heads, and sectors. Since the sector size is normally 512 bytes, they were a multiple of that. See e.g. this table of old hard drives:
The gap between SI prefixes and their base-2 approximations grows as sizes get larger.
Before 1GB HDD’s the file system overhead was a much more dramatic effect so it didn’t really matter to consumers which prefix convention they used or how they rounded.
> HDDs, RAM, network speed were base-2 (e.g. 1TB = 1024GB) for most of my life
No, network speeds are the exception in that they have always used base-10 for a long time. For example, the basic ISDN data rate of 64kbit/s is exactly 64000 bits/s (8000 samples of 8-bit PCM per second); a 9.6k modem is 9600 bits/s; a 10 megabit Ethernet link is 10000000 bits/s; and so on.
It's absurd to claim they're doing this for any other reason beyond deceptive marketing. It's like an advertising version of a dark pattern: a decision which has a sheen of reasonableness, but is clearly designed to deceive and exploit your customers.
Nowhere else in the computer is the base-10 version standard, and it shouldn't be. The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is clearly reasonable, the sentence "a 32-bit CPU register can address up to 4.29497... gigabytes of memory" is not. It's not just ram though: all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2, having a storage unit that is not is insanity. Or, to bring it back to hard drives: what's the max file size of a file stored on a FAT32 filesystem again?
It's just bad and deceptive, and hard drive manufactures should not be allowed to play these kinds of games. You should stop carrying the water for their shitty practices.
The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is sloppy, the sentence "a 32-bit CPU register can address up to 4 GiB of memory" is not.
> The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is sloppy, the sentence "a 32-bit CPU register can address up to 4 GiB of memory" is not.
"Up to" is marketing. From my experience "up to" starts from 0.
Back then, a "byte" was not necessarily "8 bits", but whatever the smallest unit of addressable memory was. This was the same as or an even fraction of the word length of the computer. Common word lengths back then included 12, 18, 24, and even 30 bits. 8-bit bytes would only fit evenly into one of those.
Storage devices like hard drives are not addressed like memory, and the logical segmentation of bits into bytes/words/clusters was a function of the software, not the hardware. One storage device could be built for use in an 8-bit or 12-bit computer, resulting in very different numbers if you tried counting its size in bytes on these different machines.
Nowhere else in the computer is the base-10 version standard
As the parent said, networking. (Which also suffers from an unfortunate bits/bytes mismatch.)
The real problem with storage IMO is that software used base-2 measurements and vendors used base-10 and eventually the discrepancy got big enough to be noticeable.
For me, the egregious part isn't even the difference in base 2 and base 10 but rather that windows uses base 2 but displays it as base 10 units. (Internally uses Gibibytes but displays the unit as Gigabytes, for example)
> Nowhere else in the computer is the base-10 version standard
Well, you're simply wrong. CPU frequency is measured in GHz, that is 1,000,000,000 Hz, not 1,073,741,824 Hz. Consequently, my CPU has a nominal frequency of 3.0 GHz, not 2.793967724 GHz. Since frequencies are denominated in powers of 10, bandwidths are as well: The L1 cache in my CPU can read 64 bytes/cycle (I think), so it has a read-bandwidth of 192 GB/s, that is 192,000,000,000 B/s. My Ethernet device can transfer 1 Gbit/s, that is 1,000,000,000 bit/s, because it's clocked at 125 MHz (that's 125,000,000 Hz) and transfers 8 bit per clock cycle.
> all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2
Yeah, and it pretty much ends with basic data types. Most of my structs aren't powers of two in size, even though their individual elements are 1, 2, 4, or 8 bytes in size. Almost none of my files have sizes related to powers of two. My SSD has a size of 931.51339 GiB – so tell me again how powers of two are more correct or more useful here?
> Almost none of my files have sizes related to powers of two.
They are all rounded up to 512 or 4096 bytes by your filesystem.
> My SSD
SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS. [1] HDDs a similarly low-level-formatted with 512 or 4096 B sector sizes.
> They are all rounded up to 512 or 4096 bytes by your filesystem.
That's an implementation detail, and neither I nor my programs care about it. We do care about the actual length of the file, which isn't related to powers of two, except in niche cases.
> SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS.
While true, this is also an implementation detail, and I don't think about the size of my SSD in multiples of sectors or pages or kibibytes – "976762584 kiB" just isn't a useful number for me, but "1000.2 GB" is manageable. Specifying it as "931.51 GiB" isn't more correct, so it makes sense to default to powers of 10 instead of powers of 2.
The situation would be different if HDDs and SSDs always or typically had exact sizes like 512*2^30 (= 512 GiB) or 3*2^40 (= 3 TiB) – you know, like RAM – in which case using powers of 2 would make sense. But that's not the world we're living in.
> The situation would be different if HDDs and SSDs always or typically had exact sizes
I just checked two of three SSDs available to me. Two were exact multiples of power-of-two gigabytes -- 200 GiB, 120 GiB. One was 500.3 GB.
SI units might be more useful to a consumer (and I am not arguing they aren't). But the argument that storage systems have "nothing to do" with powers of two is not valid.
For disks with sectors of 4096 bytes, the formula is the same except the constants are accordingly 8 times smaller.
Regarding sectors of 520 bytes, usable bytes are still 512. Quote from the above referenced document:
> HDDs formatted with T10 PI (Protection Information) have 8 bytes added to the end of each sector. However, the user usable sector size will remain at 512 bytes or 4096 bytes. The 8 bytes of PI will increase the overhead of the disk sector density similar to overhead of ECC bytes in a given sector.
> They are all rounded up to 512 or 4096 bytes by your filesystem.
Not always. Some filesystems allow small files to be compacted into other structures: the MTF can hold small in NTFS, ext* and btrfs support similar inlining of data in inodes for small files, other filesystems (ZFS, Reiser, btrfs again) support more general sub-block allocation (with techniques such as tail packing or block sub-division). Sometimes this is only for small files not “oddly” sized larger files, but things like tail packing can apply to the latter too.
I may not be the biggest fan of it ever but it is a fact that this is how hard drives have been measured since the days when a 5.25" full height MFM 10MB or 20MB hard drive in an IBM XT or clone was state of the art technology.
your comment seems to insinuate that this is some new sneaky marketing practice of HDD manufacturers to trick the consumer.
at this point I think it's something we will have to deal with, like, I'd prefer to buy carpet by the square meter, but fact is in canada it's still sold by the square foot just like in the USA, with weird non metric units.
Give it a little more time and it'll be like 2x4s. At one point they were actually 2" by 4", but now they're 3.5" by 1.5" and that's the standard size and we're stuck with this dumb little detail where the name doesn't match reality and everybody just accepts it - but it confuses virtually every single person on the planet at LEAST once or twice in their life.
if you really want to see something that's irretrievably stuck in american measurements, take a look at the national engineering standards and vendor sources for telecom tower components... for anything from a light duty 40' aluminum self support you might put in your yard for ham radio all the way up to gargantuan 2000' guyed broadcast towers.
canada has no hope of doing anything in metric here, everything is in US units, and probably will be until the year 2200
2x4s are 2" by 4" before they're finished. If you search S4S (sanded 4 sides) 2x4s, they will actually be 2" by 4". If you get the wood with rounded corners, they take a quarter inch off each side to "finish" it. It's been that way for 100 years.
> The limited availability of lumber and the rapid pace of housing construction made other methods like concrete-block housing viable. This pressured further compromise because thinner 2x4s were a way to compete in price with wood alternatives. Size standards, maximum moisture content, and nomenclature were agreed upon only as recently as 1964. The nominal 2x4 thus became the actual 1½ x 3½, imperceptibly, a fraction of an inch at a time.
So it seems the reduction was was driven by cost saving, not a product of the finishing process, and it's actually only been standardized that way for 58 years.
Except, per my original point - it’s called a 2x4 - but it’s not actually 2 inches by 4 inches, which would be as you say 50ish x 100ish mm. In reality it’s actually standardized to be 38 x 89 mm
What I tried to convey, in a stupid way, is that there wouldnt be an exact translation of the measurements into metric, but that the dimensions qould slightly change in switching over.
and now I realize that that point was irrelevant anyways. Sorry for derailing the thread with useless replies.
> For example, a "2×4" board historically started out as a green, rough board actually 2 by 4 inches (51 mm × 102 mm). After drying and planing, it would be smaller by a nonstandard amount. Today, a "2×4" board starts out as something smaller than 2 inches by 4 inches and not specified by standards, and after drying and planing is minimally 1+1⁄2 by 3+1⁄2 inches (38 mm × 89 mm).[8]
Your quote just means when you're buying green wood and it says it's 2"x4" it will be exactly 2"x4". It doesn't mean a 2x4 started out as a 2"x4" green board.
> Historically, the nominal dimensions were the size of the green (not dried), rough (unfinished) boards that eventually became smaller finished lumber through drying and planing (to smooth the wood). Today, the standards specify the final finished dimensions and the mill cuts the logs to whatever size it needs to achieve those final dimensions.
I was originally pointing out that the naming is simply a standard and convention, 'dimensional lumber'. All that matters is that if I buy a '2 by' almost anywhere in the United States, it will be close to the same size and characteristics.
It wasn't sneaky way back then. A "byte" has not always been defined as "exactly 8 bits"; it used to be hardware-dependent. 6-bit bytes were common in systems with word lengths that were multiples of 6 instead of the now-common 8.
You have to be very familiar with tech to have any idea about base-2 vs base-10. Universal standards to make computing units understandable would very clearly go with base-10, which is the status quo in units elsewhere (metric). Base-2 is just an implementation detail, and one that the vast majority of the world simply does not care about.
Only slightly familiar. Binary number systems are explained in introductory first books on computers. I knew binary nearly 10 years before I got my hands on a computer.
You need to have some domain knowledge to know -- did someone think the power of 2 was important here?
For communications they mostly haven't (we tend to deal with decimal clock rates, bandwidths, etc-- and even when we have 128-QAM or whatever that's going to not result in powers of 2). For memories and caches we generally like them to be powers of 2 to use all the bits of our addressing and decoders well. For secondary storage, ... we tend to have a non-round number of lumps that are a small power of 2 sized, and we might as well use decimal.
This is subtle and has been woefully inconsistent. It would be better to just use powers of 10.
(Me of 20 years ago would be very disappointed in myself for writing this comment).
> Or, to bring it back to hard drives: what's the max file size of a file stored on a FAT32 filesystem again?
That is not about hard drives, that is about the abstraction layer of FAT32, which uses a 32-bit quantity for size.
FAT32 could be used for a storage device that stores things based on powers-of-10 sizes. It's just an implementation detail if anything happens to do it with powers of 2 underneath.
> It's absurd to claim they're doing this for any other reason beyond deceptive marketing.
Your comment is a good example of "I so want this narrative to be true!" without any backing.
Back in the 90's, my computer studies textbook had 1 KB as 1024 bytes, and 1 MB as 1,000,000 bytes. It explicitly stated that it is not 1024 KB. What's their angle?
> It's absurd to claim they're doing this for any other reason beyond deceptive marketing. […] It's just bad and deceptive, and hard drive manufactures should not be allowed to play these kinds of games. You should stop carrying the water for their shitty practices.
The confusion of binary and decimal prefixes has been around for decades:
Anecdata: I have a lovely old Canon Cat in working condition that I want to sell to someone who is interested in this piece of computing history. But I have somehow misplaced the box of floppy disks that came with it!
I know they are here somewhere. Looks like I have some housekeeping to do.
When I find them, that number will become nonzero...
SI metric prefix standards really really matter to an engineer. The creation of a new indistinguishable unit which varies depending on what was measured (so that 1k != 1k) is mind-bogglingly idiotic in this old engineer’s opinion.
As a child I too felt the pain of feeling “shortchanged” by hardware manufacturers’ marketing practices with hard drive sizes, and I deeply understand the glamour of using 1024 based decimals for declaring total memory. However the cost of differing values for metric prefixes (k, M, G) is confusion, mistakes, and arguments. I also think that any card carrying geek knows their measurements, and cannot be easily deceived. Another argument for why software engineers are not. Disclaimer: last gig was JavaScript.
No, it is absurd to claim that something which predates using base 2 based counts being common is designed to con people who work in that sort of unit.
Communications theory used 1,000s and such to state transfer rates as with any other scientific unit long before modern drives. That stuck when things were recorded to mediums such as tape, and every disk ever has been measured on 1,000s or 1,000,000s or …
There have been some oddities along the way: punched cards were all over the place and were measured in characters not bits/bytes, many comms methods too were measured in characters/symbols or modulations rather than bits & bytes, many comms and storage (tape, disk, and RAM/ROM) formats are not nice round powers of 2 in raw form due to check bits, parity bits, and stop bits.
The one and only occurrence I can think of where 2^10 was used officially for a physical storage measure is the 1.44Mb floppy format which was a strange bastard of multiple divisions: it was 1440KiB, so 1.4410001024, so really was either ~1.38MiB or ~1.47MB.
> Nowhere else in the computer is the base-10 version standard, and it shouldn't be.
Nowhere else in science nor enginerring is 2^10 the standard instead of 10^3, and it shouldn't be. In computing we've adopted 2^10 that because some things are easier for us to think about that way, and 2^10 and 10^3 are close enough for many estimating or quick calc purposes, but why should science that predates that practise being common be forced to change simply for our convenience?
Yes I support moving from imperial to metric where that hasn't already been done, but not because I encountered metric first (I didn't) so the other is unfamiliar but because it makes sense. You might argue that to programmers or less technical tech users 2^10 makes more sense, but they are less important in terms of science and engineering then, well, the rest of science and engineering.
> what's the max file size of a file stored on a FAT32 filesystem again?
2GiB, or approximately 2.147GB. Many people incorrectly calling it 2GB does not change that G as a prefix means 10^9 and not 2^31 - otherwise we have to accept other things like “u” being a correct spelling of “you”!
It isn't a grand conspiracy to defraud us. We chose to give those prefixes a double meaning, it isn't the rest of science & engineering's fault that we sometimes get confused by that choice that we made and they didn't.
The two most common sector sizes on HDDs are 512 bytes (a byte being 8 bits) and 4096 bytes. Base 2 units make a lot more sense with these power-of-two sector sizes than base 10 units.
Funny enough RAM is still used to segment products in price classes even though its cheap. Want to buy a entry level mac? 8GB here you go, now you can only have a couple of tabs up and god forbid you open up photoshop while browsing.
It wont run out of memory but it will page to disk making switching programs slower. Next you have an OS update which conveniently consumes more RAM.
Hard drives (both rotary and SSD; floppies [1] and CDs [2] also) are also manufactured according to base-2 addressing. They are low-level formatted at the factory in sectors which are power-of-2 sizes. [3] This has been true for at least 40 years. [4]
> Another example is network media. Serialized data transfer speed is measured in {K,M,G,T}b/s (bits) or B/s (Bytes) where a Gigabit is exactly 1000000000 bits and Gigabyte is exactly 1000000000 bytes.
interesting network trivia: a LANPHY and a WANPHY 10 gigabit ethernet connection are not quite the same thing. The latter is designed to fit within legacy DWDM transport systems designed to carry OC-192 so its data rate will fit within 9953.28 Mbit/s or slightly under that.
As files have gotten bigger though, I have found myself in situations where I've had to explain why, if it's taking longer than you expect for a file to transfer, about 7% of that is unit conversion errors.
So while there is a bit of waste and we could do some work to speed up the transfer, it's still going to be 10% less than you think is attainable.
But even so, the gigabyte sins of the hard drive makers are still less than half of those of the network card makers who used to advertise baud instead of bytes, so the fact that the data is going across as 5:4 encoding means that you divide the Kbps by 10 - not 8 - to get the actual throughput.
> But even so, the gigabyte sins of the hard drive makers are still less than half of those of the network card makers who used to advertise baud instead of bytes,
Actually, they generally specified bits per second, even if they called it "baud".
> so the fact that the data is going across as 5:4 encoding means that you divide the Kbps by 10 - not 8 - to get the actual throughput.
Stop bits and start bits. E.g. a 9600 bit per second v.32 MODEM operated at 2400 baud (2400 4 bit symbols per second), and had an end-user throughput of 960 characters per second.
Clock speed is base10. Line speed is base10. Storage is base10. But ram is base2, because it's almost always constructed as a grid.
Tapes are essentially unitless, how long is a piece of string. The earliest magnetic harddrive I can find is the IBM 350 storing 5 million 6-bit characters. Where 5 million is 5 and a bunch of zeros. it's not 4.75 .. mibilion?
Where things get weird, is when we start dividing tracks into sectors. For convenience we went for sectors that were convenient to page into memory. Every harddrive since that has had a base10 number of base2 sectors. From that point onwards, you can be team base10 or team base2, they both end up wrong.
And this is how we end up with the 1.44MB floppy, where 1.44MB is 2880 * 512kb. Or 1.44 * 1000 * 1024.
Why does the storage size unit of measure have to do with addressing? That drive addressing uses the LBA-48 (bit) addressing has nothing to do with the fact the minimum addressible unit size, the physical sector is a base-2 value, i.e. 512 bytes or 4096 bytes.
Still seems the storage unit size should be IEC based, not SI.
Fun fact: The dimensions of a magnet (a bit of data) on an HDD are actually rectangular. It's about 3-5x wider than it is long. This favors read/write speed, since you cover more bits per rotational length. Aside from the obvious speed boost you get from it, it's also done to make it easier to stay on track.
One of the huge limitations with bit width is the ability to keep the actuator arm stable within the 15 or so nanometers because of the air turbulence. That's why they fill the drives with helium now, less aerodynamic drag/turbulence means you can either cram more platters in (which increases the turbulence back to where it originally was) or make narrower tracks since you have more precision.
Is it too hard to maintain vacuum? The comparison I can think of is a thermos, which I'm told uses a vacuum layer between the eg coffee, and the outside.
Vacuum won't work with current technology. The heads use air currents to fly a few nanometers over the platter. If you put a running drive in a vacuum chamber the heads will touch down and everything will be destroyed.
Is the status quo on drive firmware lying about what's actually been flushed to media still as terrible as it was, oh, twenty years ago?
A friend used to work for Apple and said that one of the reasons they had apple-specific versions of various mass market SCSI and IDE drives was because Apple's firmware actually flushed data to disk when told to do so, and didn't lie about whether data was flushed or not.
If there is un-written cached data there is a capacitor bank that allows this data to be quickly written to non-volatile flash and rw heads to be retracted in the event of power loss. They test this feature pretty heavily, esp. in enterprise drives. Source: used to work for an HDD manufacturer. So, they might be lying to you but it is pretty useless information on for modern drives.
Basically, "flushed" means safe. Whether it's written to internal DRAM/SRAM (with power-loss prevention), NV flash, or the actual platter is irrelevant. When in history did the lie start to not matter?
The other interesting thing about modern data centre HDDs is they have a media cache which significantly accelerates synchronous random write IOPS - OOTB these drives get 75 IOPS with fio fsync=1 bs=4k randwrite, but with a WCE=0 they can do 400 IOPS.
So if your app is fsync heavy (such as Ceph) then you can switch on this media cache by setting the drive to write through mode (WCE=0).
This is the result of "more bits-per-inch", which is the natural result of going from 4TB-per-drive to 20TB-per-drive.
Yes, I'm skipping a few steps, but that's the fundamental issue at play here. There's more platters, more bits per inch, more read / write speed at the same 7200 RPM.
Its not a lot, but its a steady increase as hard drives keep getting denser and denser.
Using a single Prometheus node with redundant drives rather than redundant nodes with single drives is an odd choice to me. Why replicate at block level when layer 7 has support?
A multi-node Prometheus setup is significantly more complex to design and operate than a Linux software RAID mirror, especially once you throw in Grafana, Alertmanager, and so on. Distributed anything makes it harder. The layer 7 may 'support' this, but it's not a plug and play setup.
I'm never buying another HDD in my life. There are a few tasks that take weeks on an HDD but only hours on an SSD. And the price difference is often negligible.
152 comments
[ 2.9 ms ] story [ 163 ms ] threadFor video editing, ETL and other read-heavy workloads, they have a point.
Older model: 670 IO/s and 2.9 ms average latency
Newer model: 1680 IO/s and 1.1 ms average latency
We got the vendor to send us new drives and shipped the older model back to them.
Older drive:
Newer drive: My guess at the time was that the newer drive had a deeper command queue, but I'm not actually sure, and I think both drives had the same amount of cache.We use the controller in HBA mode, so the controller shouldn't be doing anything fancy, but you might be right.
If so, then it would probably be useful to see if the vendor drive model numbers are available.
For the dramatic increase in iops, I wonder if there are more head+actuator assemblies on the newer drives or similar? Along the lines of these:
https://www.seagate.com/innovation/multi-actuator-hard-drive...
And yes, HP changes the firmware and model numbers... super annoying.
>We got the vendor to send us new drives and shipped the older model back to them
not much of an agreement then, shows where the power lies
>MB8000GFECR
Seagate ST8000NM0055 at 2-4x the price
>MB008000GWWQU
another seagate
SSDs have won. Spinning hard drives are going to ship fewer and fewer units. The economies of scale are not going to improve ever again and are already getting worse. Further, the market just doesn't care about paying for capacity for the most part.
We peaked in 2010 with 650 million drives/year. Current unit counts are about one third of that. Compare to SSDs with about 400 million units/year.
I personally have some Exos 16TB's. There's a decent price history available[1] reaching back to mid-2019. Make sure to tick on "3rd party new". These aren't going to be the best prices available in many cases, but it's reasonably indicative.
[1] https://camelcamelcamel.com/product/B07SPFPKF4
A 2.5x difference in IO/s adds up quickly.
As for HDD prices, neither 12TB nor 8TB were top of the line at the timeframes you mention. If you buy largest available drives, or a model lower, the prices have been going down. Not as much as we all would like, but overall the trend has been moving in the downward direction.
- did you buy those drives in the "sweet spot" of capacity per dollar? 8 and 12 TB drives aren't the sweet spot anymore
- there's also this covid disruption thing
- when was the factory fire in ?vietnam?
- and how much competition is there? Are we down to two companies basically?
The pair of 8TB I had previously are within $5 of what I paid in 2017.
SI prefixes should be (and are) the rule rather than the exception. It only made sense to measure ram in base-2 units because they are manufactured according to base-2 addressing.
HDD’s have never used base 2 addressing.
Another example is network media. Serialized data transfer speed is measured in {K,M,G,T}b/s (bits) or B/s (Bytes) where a Gigabit is exactly 1000000000 bits and Gigabyte is exactly 1000000000 bytes.
In the early days of computers ram was extremely scarce and expensive. It was such a limitation that it was usually the very first question you asked about a computer’s capabilities.
I remember in 5th grade asking a fellow student who’s parents had just bought an Atari 800 “How much ‘K’ does it have?”
This intense focus on ram may have led people to assume that it’s odd nomenclature would apply to anything computer related.
Despite hard drive manufacturers using correct units, people still blame hard drive manufacturers for their disappointment.
Marketers measure in base 10 because it lets them inflate the stated capacity in a way that isn't legally fraudulent.
What do you mean? 'Computers' don't measure storage, they execute programs. Humans (often programmers) are the ones who want to know about storage.
Base 10 is something that is primarily used for presentation to humans.
We all know this, this is HN....
There's lots of reasons why computers so often use powers of two and thus K being 2^10 instead of 10^3. But the word measure is probably not the best.
HDDs, RAM, network speed were base-2 (e.g. 1TB = 1024GB) for most of my life until SSDs became common and even Google now says "1 Gigabyte = 1000 Megabytes" to my exasperation.
https://www.computerhope.com/hdquantu.htm
E.g., if you take the Quantum Bigfoot 1.2 GB model:
2492 cylinders * 16 heads * 63 sectors * 512 bytes per sector = 1286111232 bytes
The size it was sold as, is definitely closer GiB (as in power of two):
1.2 * 2*30 = 1288490189
With rounding, they could sell it as 1.3 GiB with the current marketing trick of using SI units.
So the old branding strategy seems to have been: compute the size in bytes, then find a reasonably MiB or GiB value (powers of two).
Before 1GB HDD’s the file system overhead was a much more dramatic effect so it didn’t really matter to consumers which prefix convention they used or how they rounded.
2492 * 16 * 63 * 512 / 1000 ^ 3 = 1.286111232
4092 * 16 * 63 * 512 / 1000 ^ 3 = 2.111864832
4994 * 16 * 63 * 512 / 1000 ^ 3 = 2.577383424
Using GiB you get different numbers if you look past the first element in the table:
2492 * 16 * 63 * 512 / 1024 ^ 3 = 1.197784423828125
4092 * 16 * 63 * 512 / 1024 ^ 3 = 1.966827392578125
4994 * 16* 63 * 512 / 1024 ^ 3 = 2.4003753662109375
So I stand by my point, they have always used SI units. It's only ram (and software) that used incorrect definitions of k, M, G and T.
E.g. Seagate ST-506, first 5.25" hard drive.
"The total formatted capacity of the four heads and surfaces is 5/10 megabytes (32 sectors per track, 256 bytes per sector, 612/1224 tracks)."
612 * 32 * 256 = 5013504, or 5.01 megabytes or 4.78 mebibytes.
No, network speeds are the exception in that they have always used base-10 for a long time. For example, the basic ISDN data rate of 64kbit/s is exactly 64000 bits/s (8000 samples of 8-bit PCM per second); a 9.6k modem is 9600 bits/s; a 10 megabit Ethernet link is 10000000 bits/s; and so on.
Nowhere else in the computer is the base-10 version standard, and it shouldn't be. The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is clearly reasonable, the sentence "a 32-bit CPU register can address up to 4.29497... gigabytes of memory" is not. It's not just ram though: all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2, having a storage unit that is not is insanity. Or, to bring it back to hard drives: what's the max file size of a file stored on a FAT32 filesystem again?
It's just bad and deceptive, and hard drive manufactures should not be allowed to play these kinds of games. You should stop carrying the water for their shitty practices.
"Up to" is marketing. From my experience "up to" starts from 0.
Like saying a car can go up to 200mph, no one thinks the car is unable to go 50mph.
Back then, a "byte" was not necessarily "8 bits", but whatever the smallest unit of addressable memory was. This was the same as or an even fraction of the word length of the computer. Common word lengths back then included 12, 18, 24, and even 30 bits. 8-bit bytes would only fit evenly into one of those.
Storage devices like hard drives are not addressed like memory, and the logical segmentation of bits into bytes/words/clusters was a function of the software, not the hardware. One storage device could be built for use in an 8-bit or 12-bit computer, resulting in very different numbers if you tried counting its size in bytes on these different machines.
As the parent said, networking. (Which also suffers from an unfortunate bits/bytes mismatch.)
The real problem with storage IMO is that software used base-2 measurements and vendors used base-10 and eventually the discrepancy got big enough to be noticeable.
Well, you're simply wrong. CPU frequency is measured in GHz, that is 1,000,000,000 Hz, not 1,073,741,824 Hz. Consequently, my CPU has a nominal frequency of 3.0 GHz, not 2.793967724 GHz. Since frequencies are denominated in powers of 10, bandwidths are as well: The L1 cache in my CPU can read 64 bytes/cycle (I think), so it has a read-bandwidth of 192 GB/s, that is 192,000,000,000 B/s. My Ethernet device can transfer 1 Gbit/s, that is 1,000,000,000 bit/s, because it's clocked at 125 MHz (that's 125,000,000 Hz) and transfers 8 bit per clock cycle.
> all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2
Yeah, and it pretty much ends with basic data types. Most of my structs aren't powers of two in size, even though their individual elements are 1, 2, 4, or 8 bytes in size. Almost none of my files have sizes related to powers of two. My SSD has a size of 931.51339 GiB – so tell me again how powers of two are more correct or more useful here?
They are all rounded up to 512 or 4096 bytes by your filesystem.
> My SSD
SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS. [1] HDDs a similarly low-level-formatted with 512 or 4096 B sector sizes.
[1] https://spdk.io/doc/ssd_internals.html
That's an implementation detail, and neither I nor my programs care about it. We do care about the actual length of the file, which isn't related to powers of two, except in niche cases.
> SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS.
While true, this is also an implementation detail, and I don't think about the size of my SSD in multiples of sectors or pages or kibibytes – "976762584 kiB" just isn't a useful number for me, but "1000.2 GB" is manageable. Specifying it as "931.51 GiB" isn't more correct, so it makes sense to default to powers of 10 instead of powers of 2.
The situation would be different if HDDs and SSDs always or typically had exact sizes like 512*2^30 (= 512 GiB) or 3*2^40 (= 3 TiB) – you know, like RAM – in which case using powers of 2 would make sense. But that's not the world we're living in.
I just checked two of three SSDs available to me. Two were exact multiples of power-of-two gigabytes -- 200 GiB, 120 GiB. One was 500.3 GB.
SI units might be more useful to a consumer (and I am not arguing they aren't). But the argument that storage systems have "nothing to do" with powers of two is not valid.
[0] http://www.idema.org/wp-content/downloads/2169.pdf
Regarding sectors of 520 bytes, usable bytes are still 512. Quote from the above referenced document:
> HDDs formatted with T10 PI (Protection Information) have 8 bytes added to the end of each sector. However, the user usable sector size will remain at 512 bytes or 4096 bytes. The 8 bytes of PI will increase the overhead of the disk sector density similar to overhead of ECC bytes in a given sector.
Not always. Some filesystems allow small files to be compacted into other structures: the MTF can hold small in NTFS, ext* and btrfs support similar inlining of data in inodes for small files, other filesystems (ZFS, Reiser, btrfs again) support more general sub-block allocation (with techniques such as tail packing or block sub-division). Sometimes this is only for small files not “oddly” sized larger files, but things like tail packing can apply to the latter too.
your comment seems to insinuate that this is some new sneaky marketing practice of HDD manufacturers to trick the consumer.
canada has no hope of doing anything in metric here, everything is in US units, and probably will be until the year 2200
> The limited availability of lumber and the rapid pace of housing construction made other methods like concrete-block housing viable. This pressured further compromise because thinner 2x4s were a way to compete in price with wood alternatives. Size standards, maximum moisture content, and nomenclature were agreed upon only as recently as 1964. The nominal 2x4 thus became the actual 1½ x 3½, imperceptibly, a fraction of an inch at a time.
http://www.harvarddesignmagazine.org/issues/45/nominal-versu...
So it seems the reduction was was driven by cost saving, not a product of the finishing process, and it's actually only been standardized that way for 58 years.
(I rounded those values because if the US were to go metric, the dimension would slightly change, rather than being exactly 50.8 by 101.6 mm)
What I tried to convey, in a stupid way, is that there wouldnt be an exact translation of the measurements into metric, but that the dimensions qould slightly change in switching over.
and now I realize that that point was irrelevant anyways. Sorry for derailing the thread with useless replies.
"Dimensional lumber is available in green, unfinished state, and for that kind of lumber, the nominal dimensions are the actual dimensions."
Your quote just means when you're buying green wood and it says it's 2"x4" it will be exactly 2"x4". It doesn't mean a 2x4 started out as a 2"x4" green board.
I was originally pointing out that the naming is simply a standard and convention, 'dimensional lumber'. All that matters is that if I buy a '2 by' almost anywhere in the United States, it will be close to the same size and characteristics.
Only slightly familiar. Binary number systems are explained in introductory first books on computers. I knew binary nearly 10 years before I got my hands on a computer.
If you want base-2 then use kibiBytes (kiB) and move on..
For communications they mostly haven't (we tend to deal with decimal clock rates, bandwidths, etc-- and even when we have 128-QAM or whatever that's going to not result in powers of 2). For memories and caches we generally like them to be powers of 2 to use all the bits of our addressing and decoders well. For secondary storage, ... we tend to have a non-round number of lumps that are a small power of 2 sized, and we might as well use decimal.
This is subtle and has been woefully inconsistent. It would be better to just use powers of 10.
(Me of 20 years ago would be very disappointed in myself for writing this comment).
That is not about hard drives, that is about the abstraction layer of FAT32, which uses a 32-bit quantity for size.
FAT32 could be used for a storage device that stores things based on powers-of-10 sizes. It's just an implementation detail if anything happens to do it with powers of 2 underneath.
Not without wasting lots of space. FAT32 only allows sector sizes of 512, 1024, 2048, or 4096 bytes.
[1] https://www.cs.fsu.edu/~cop4610t/assignments/project3/spec/f...
Your comment is a good example of "I so want this narrative to be true!" without any backing.
Back in the 90's, my computer studies textbook had 1 KB as 1024 bytes, and 1 MB as 1,000,000 bytes. It explicitly stated that it is not 1024 KB. What's their angle?
The confusion of binary and decimal prefixes has been around for decades:
* https://en.wikipedia.org/wiki/Timeline_of_binary_prefixes
This is not some recent conspiracy to boost profits or whatever.
It holds 1,474,560 bytes.
That is 1.44 * 1000 * 1024.
I know they are here somewhere. Looks like I have some housekeeping to do.
When I find them, that number will become nonzero...
SI metric prefix standards really really matter to an engineer. The creation of a new indistinguishable unit which varies depending on what was measured (so that 1k != 1k) is mind-bogglingly idiotic in this old engineer’s opinion.
As a child I too felt the pain of feeling “shortchanged” by hardware manufacturers’ marketing practices with hard drive sizes, and I deeply understand the glamour of using 1024 based decimals for declaring total memory. However the cost of differing values for metric prefixes (k, M, G) is confusion, mistakes, and arguments. I also think that any card carrying geek knows their measurements, and cannot be easily deceived. Another argument for why software engineers are not. Disclaimer: last gig was JavaScript.
Communications theory used 1,000s and such to state transfer rates as with any other scientific unit long before modern drives. That stuck when things were recorded to mediums such as tape, and every disk ever has been measured on 1,000s or 1,000,000s or …
There have been some oddities along the way: punched cards were all over the place and were measured in characters not bits/bytes, many comms methods too were measured in characters/symbols or modulations rather than bits & bytes, many comms and storage (tape, disk, and RAM/ROM) formats are not nice round powers of 2 in raw form due to check bits, parity bits, and stop bits.
The one and only occurrence I can think of where 2^10 was used officially for a physical storage measure is the 1.44Mb floppy format which was a strange bastard of multiple divisions: it was 1440KiB, so 1.4410001024, so really was either ~1.38MiB or ~1.47MB.
> Nowhere else in the computer is the base-10 version standard, and it shouldn't be.
Nowhere else in science nor enginerring is 2^10 the standard instead of 10^3, and it shouldn't be. In computing we've adopted 2^10 that because some things are easier for us to think about that way, and 2^10 and 10^3 are close enough for many estimating or quick calc purposes, but why should science that predates that practise being common be forced to change simply for our convenience?
Yes I support moving from imperial to metric where that hasn't already been done, but not because I encountered metric first (I didn't) so the other is unfamiliar but because it makes sense. You might argue that to programmers or less technical tech users 2^10 makes more sense, but they are less important in terms of science and engineering then, well, the rest of science and engineering.
> what's the max file size of a file stored on a FAT32 filesystem again?
2GiB, or approximately 2.147GB. Many people incorrectly calling it 2GB does not change that G as a prefix means 10^9 and not 2^31 - otherwise we have to accept other things like “u” being a correct spelling of “you”!
It isn't a grand conspiracy to defraud us. We chose to give those prefixes a double meaning, it isn't the rest of science & engineering's fault that we sometimes get confused by that choice that we made and they didn't.
The two most common sector sizes on HDDs are 512 bytes (a byte being 8 bits) and 4096 bytes. Base 2 units make a lot more sense with these power-of-two sector sizes than base 10 units.
No honestly: Kilo should always mean 1'000, Mega should always mean 1'000'000 and not 1'048'576.
And that's why we have "kibi", "mebi", etc.
So Kilo means times 1,000. Kibi means times 1,024 and so on.
Still the "minimum" unit, the sector or "block" is 512 or 4096 bytes, so the multiple/power of 2 is "ingrained" in the media.
[1] https://en.wikipedia.org/wiki/Disk_formatting#Low-level_form...
[2] https://en.wikipedia.org/wiki/CD-ROM#Sector_structure
[3] https://en.wikipedia.org/wiki/Disk_formatting#Low-level_form...
[4] https://en.wikipedia.org/wiki/Disk_sector
interesting network trivia: a LANPHY and a WANPHY 10 gigabit ethernet connection are not quite the same thing. The latter is designed to fit within legacy DWDM transport systems designed to carry OC-192 so its data rate will fit within 9953.28 Mbit/s or slightly under that.
So while there is a bit of waste and we could do some work to speed up the transfer, it's still going to be 10% less than you think is attainable.
But even so, the gigabyte sins of the hard drive makers are still less than half of those of the network card makers who used to advertise baud instead of bytes, so the fact that the data is going across as 5:4 encoding means that you divide the Kbps by 10 - not 8 - to get the actual throughput.
Fuck those guys.
Actually, they generally specified bits per second, even if they called it "baud".
> so the fact that the data is going across as 5:4 encoding means that you divide the Kbps by 10 - not 8 - to get the actual throughput.
Stop bits and start bits. E.g. a 9600 bit per second v.32 MODEM operated at 2400 baud (2400 4 bit symbols per second), and had an end-user throughput of 960 characters per second.
Clock speed is base10. Line speed is base10. Storage is base10. But ram is base2, because it's almost always constructed as a grid.
Tapes are essentially unitless, how long is a piece of string. The earliest magnetic harddrive I can find is the IBM 350 storing 5 million 6-bit characters. Where 5 million is 5 and a bunch of zeros. it's not 4.75 .. mibilion?
Where things get weird, is when we start dividing tracks into sectors. For convenience we went for sectors that were convenient to page into memory. Every harddrive since that has had a base10 number of base2 sectors. From that point onwards, you can be team base10 or team base2, they both end up wrong.
And this is how we end up with the 1.44MB floppy, where 1.44MB is 2880 * 512kb. Or 1.44 * 1000 * 1024.
Still seems the storage unit size should be IEC based, not SI.
One of the huge limitations with bit width is the ability to keep the actuator arm stable within the 15 or so nanometers because of the air turbulence. That's why they fill the drives with helium now, less aerodynamic drag/turbulence means you can either cram more platters in (which increases the turbulence back to where it originally was) or make narrower tracks since you have more precision.
https://en.wikipedia.org/wiki/Flying_height
A friend used to work for Apple and said that one of the reasons they had apple-specific versions of various mass market SCSI and IDE drives was because Apple's firmware actually flushed data to disk when told to do so, and didn't lie about whether data was flushed or not.
So if your app is fsync heavy (such as Ceph) then you can switch on this media cache by setting the drive to write through mode (WCE=0).
SATA SSDs have a similar quirk.
Yes, I'm skipping a few steps, but that's the fundamental issue at play here. There's more platters, more bits per inch, more read / write speed at the same 7200 RPM.
Its not a lot, but its a steady increase as hard drives keep getting denser and denser.
(It also requires more hardware.)
(I'm the author of the linked-to entry.)