But still, you can go to Amazon and get something like this: https://amzn.com/B01IAGSD68 at 22 cents/GB, 510/530 read/write speeds, 92K/83K random read/writes.
Endurance of consumer SSDs may be derated for marketing reasons, but there are also physical differences that give enterprise SSDs higher endurance. A consumer 240 GB SSD has 256 GB of flash, while an enterprise 240 GB SSD may have 512 GB of flash. More flash allows more writes and the effect is nonlinear because FTLs are more efficient with more free space.
A SATA SSD with capacitors, high endurance, and a good warranty is not "consumer that pretends they are enterprise". It's actually better. Really the only advantage of SAS is dual port and some customers don't need that.
The benefit of 15k SAS drives is 250MBs per drive actual throughput, even more because they are RAID arrays - for 3c a GB, they are HIGHER performance than most SSD drives on the market.
Sticking "enterprise" on the box will not make enterprise pay 42c per gig for less performance.
and when the hardware they plug into is a sas interface, they arent going to be buying them either.
seriously, getting a bit tired of the immaturity and shilly nonsense on here.
check any rack machine from cisco, dell or any of the other big enterprise suppliers (of which intel is an oem, they dont really deal with the likes of hsbc or shell corporation) and they all come with sas interface for storage.
enterprise means "more than 10000 employees".
you do have dedicated support personel from your supplier.
intels "data center range" definately isnt enterprise imho. enterprises still not stupid enough to go anywhere near the cloud.
This seems a bit pissy and it doesn't really sound like you've got the experience to back up your claims.
Enterprises are absolutely using SSDs (find one who isn't), what on earth makes you think otherwise? At the most 'dramatic' I can muster, I've seen the same general thing in creaky smelly old 'enterprises' such as you describe. Almost all these places have the same story on how they got to the point where they had no other choice but to spank loads on storage, this is why SSDs became so pervasive there and it's why your post makes so little sense to me.
It goes something like this (ring the bell if you've lived this before); you'll have some ex-windows/mcse-types who fancied themselves as 'virtualization' experts back when vmware gave you ssh access and the words 'mezzanine connector', 'blade enclosure' and 'IO VIRT' were just whispers on the weirder parts of the internet.. Maybe they even spent a week at a vmware event and got that t-shirt they wear every friday which irritates you irrationally for some reason. They'll have taken an entire SAN's worth of disks and made one giant lun from it, presented that to their vmware 'cluster' and then can't understand why the hell the 2000 VMs all running across the same spindles (which are probably in 'training wheels' HW raid5 mode) then perform like an asthmatic camel carrying a few binbags of spanners under any load. What then? Well, unless they got lucky enough to hire some people who knew what they were actually doing who could help, they drop EMC a cool few mil and you get auto-tiered into the 8U of SSDs that got pushed in, someone talks at a conference and everyone pats themselves on the back and goes home to have a wank.
They have cool as fuck blue LEDs on the front of the racks though and it's all pre-tax spends so no one really loses.
The latest dell kit generally uses SAS for the 'os' drives even if you go cheapo and load it up with slow LFF sata's. I'm not really sure what your point is? If you're connecting to JBODs and building your own storage, then yep -- SAS isn't going anywhere.. You really shoudn't be doing that sort of thing though. It's almost always better and cheaper to BUY storage instead of building it unless you have the ops capacity to actually do yourself (hint: you probably don't, and even if you do you probably won't keep it for the lifetime of the solution). Same story for virtu; yeah sure you can spend 8 months deploying openstack but it's probably cheaper to just give vmware some money and work on your actual problems which probably aren't around provisioning compute anyway.
So; back to the point. Hooking up to JBODs and local disks in your boxes are probably using SAS. If you're actually dealing with a larger infra tho you're probably not lucky enough to have local storage across the estate, so you're rigging up sas to some build some SANs, or to some specific giant DB or other which should probably be left to a vendor anyway. If you're building out real shared storage anywhere which has more than a dozen racks per dc then it's almost certainly going to have to be fed out over the network.
Then it'll via be 10g fibre HBAs in your VM hosts, probably with a filesystem, then a file which'll show up as disks in the vms VM's. This is a really poor way to do it, but things are fast enough now that it mostly works, even if it gives us a bit of a scratchy feeling in our eyes. If you're not masochistic enough to deal with iSCSI (I'll never do it through choice; zfs+gluster+nfs is almost always the better choice) -- then you'll work out the simplicity of a NAS outweighs the whole SAN thing these days at almost any scale. It'll take a few massive outages until you really get there though. Before the claims otherwise, I've run multi-TB oracle DB's over NFS, we're not in...
It seems "a bit pissy and inexperienced" that I would want to see actual performance figures before making an enterprise purchasing decision?
42c per gig is VERY expensive, more than 10 times the price of 15k SAS at volume.
SSDs - while I didn't say "are not used" in enterprise - are "generally" not used. As in, for every $10mill enterprise spends on SAS drives, they spend maybe a few hundred thou on SSD.
"Dynamic Write acceleration" means its TLC nand emulating SLC nand. At some point your write saturates the buffer and you lose the speed advantage. The MX series have a dynamically expanding buffer (unlike samsung evos and other competition which have a static sized buffer,) but its not infinite.
The sustained random write speed of that drive is more like 6600 IOPS, even though its burst performance in ideal conditions is close to what's specified.
These drives have several enterprise-specific features. Like most enterprise hardware, you are paying a premium for these features. If you need these features, then you will pay the price premium. If you don't need these features, then the drive will seem overpriced.
This one has capacitors and probably guarantees that what the SSD has acknowledged was actually saved.
EDIT: also often these are guaranteed IOPS, so no stalling on the numbers due to garbage collection or such.
Also, any kind of workload that involves a sync for durability and correctness.
In enterprise drives the drive can return quickly even if it's in the buffer because there's enough charge in the capacitor to whiteout to NAND from RAM in the event of power failure (hard reboot or power loss).
The consumer drive absent the capacitor either has poor fsync performance or lies about syncs.
There are people who run consumer drives in data centers due to the low GB/$ cost of them or they have applications that are not sensitive for data loss or were engineered to deal with it (replication, journaling, etc..). You won't want to run your important database on such a drive.
In SSDs, the big difference between Enterprise and Consumer is endurance. Flash memory wears out over time and the firmware manages the wear and maps out bad sectors over time. Eventually, too much wears out and the drive can no longer be written to, or is too slow to usably write to. The endurance spec tells you how much data you can expect to write before the drive must be replaced.
Micron sells both consumer and enterprise drives. The consumer 1100 SSDs max out at 400TB endurance which is fantastic for a consumer drive. These enterprise 5100 drives max out at 17.6PB endurance -- that's 44X better!
If you have an application which writes a lot of data, you should be evaluating whether or not you need something like this.
There is some uncertainty in the numbers, so the manufacturers quote fairly conservative numbers. In recent years flash cells have become a bit more fragile and the process technology has changed multiple times, so good past results on endurance testing are not necessarily indicative of future results.
I've personally burnt out a set of SSDs installed in a RAID-6. They lost no data and failed no writes, but they had become unusably slow -- less than a tenth of their original write speed.
What kind of SSDs were these, and how much writing had they been subjected to? If the flash was really wearing out, I'd expect read performance to also suffer significantly as LDPC soft decode error correction was required to recover the data intact. If only write speed was degrading and if you weren't thoroughly hammering the drives with writes around the clock, then you might merely have filled up the drives with a RAID configuration that didn't pass through TRIM commands.
If the read speed had degraded, I didn't notice since the RAID controller is the bottleneck in this particular system.
We burned out the drives with a 800Mb/s sustained 24/7 write stream. In post-mortem I did the math and the consumer drives ran out of endurance in less than a month. We replaced the drives with enterprise drives which were expected to last at least 3 years.
Also, here's a really strange finding... TRIM doesn't matter as much as one would expect. Many TRIM implementations are buggy. If the SSD contains a reliable TRIM implementation, then you can 100% guarantee that it will also have internal "move stable bits around to level out the usage" code.
I mention it because the core storage technology is quite clearly phase change memory, it's the interconnect that is unique (especially the intergrated ovonic threshold switch). But for the longest time, Intel/Micron have not wanted to associated the technology with previous PCM technologies even going so far as to deny it is PCM in the initial release press conferences.
According to some recruiters from IMFlash, they've unfortunately run into several roadblocks while devising ways to actually manufacture the stuff. Pretty much everything is there besides that, and many engineers the were working on X-point went back to working on improved flash in the meantime.
I'm rather unsure why you would want to use TLC configured as 60% spare. Wouldn't the same amount of flash at two bits per cell be much faster and longer lasting?
When you program a flash cell, you first erase it, then you program it with a new value. Each erasure wears the cell out, so the more bits-per-cell, the longer the flash memory will last.
It may also be faster: to read a cell you have to address it and then measure its value. With more bits per cell, you read/write more bits per address cycle.
These cells are nanometer-scale. Having fractional charges is extremely subtle. It is significantly slower to read and write multiple bits, and it takes a lot fewer writes until it doesn't have enough precision to support that use.
36 comments
[ 4.9 ms ] story [ 87.5 ms ] threadI already can buy SSDs under 45 cents/GB, with more IOPS and faster sequential writes.
For example the MX300 quotes endurance of 200-300TB. The 5100 is several times that - up to 17PB.
If you're running a database server then you might need that endurance.
Be interesting to see how they compare with these guys:
http://techreport.com/review/27436/the-ssd-endurance-experim...
and, afaik, for enterprise, sas hdd still leads ssd in performance price and endurance.
Enterprise servers are mostly sas interfaces. These are sata, which is mostly home and soho plus a few startups running on a shoe string (afaik)
Sticking "enterprise" on the box will not make enterprise pay 42c per gig for less performance.
so you kinda missed the point.
and when the hardware they plug into is a sas interface, they arent going to be buying them either.
seriously, getting a bit tired of the immaturity and shilly nonsense on here.
check any rack machine from cisco, dell or any of the other big enterprise suppliers (of which intel is an oem, they dont really deal with the likes of hsbc or shell corporation) and they all come with sas interface for storage.
enterprise means "more than 10000 employees".
you do have dedicated support personel from your supplier.
intels "data center range" definately isnt enterprise imho. enterprises still not stupid enough to go anywhere near the cloud.
Enterprises are absolutely using SSDs (find one who isn't), what on earth makes you think otherwise? At the most 'dramatic' I can muster, I've seen the same general thing in creaky smelly old 'enterprises' such as you describe. Almost all these places have the same story on how they got to the point where they had no other choice but to spank loads on storage, this is why SSDs became so pervasive there and it's why your post makes so little sense to me.
It goes something like this (ring the bell if you've lived this before); you'll have some ex-windows/mcse-types who fancied themselves as 'virtualization' experts back when vmware gave you ssh access and the words 'mezzanine connector', 'blade enclosure' and 'IO VIRT' were just whispers on the weirder parts of the internet.. Maybe they even spent a week at a vmware event and got that t-shirt they wear every friday which irritates you irrationally for some reason. They'll have taken an entire SAN's worth of disks and made one giant lun from it, presented that to their vmware 'cluster' and then can't understand why the hell the 2000 VMs all running across the same spindles (which are probably in 'training wheels' HW raid5 mode) then perform like an asthmatic camel carrying a few binbags of spanners under any load. What then? Well, unless they got lucky enough to hire some people who knew what they were actually doing who could help, they drop EMC a cool few mil and you get auto-tiered into the 8U of SSDs that got pushed in, someone talks at a conference and everyone pats themselves on the back and goes home to have a wank.
They have cool as fuck blue LEDs on the front of the racks though and it's all pre-tax spends so no one really loses.
The latest dell kit generally uses SAS for the 'os' drives even if you go cheapo and load it up with slow LFF sata's. I'm not really sure what your point is? If you're connecting to JBODs and building your own storage, then yep -- SAS isn't going anywhere.. You really shoudn't be doing that sort of thing though. It's almost always better and cheaper to BUY storage instead of building it unless you have the ops capacity to actually do yourself (hint: you probably don't, and even if you do you probably won't keep it for the lifetime of the solution). Same story for virtu; yeah sure you can spend 8 months deploying openstack but it's probably cheaper to just give vmware some money and work on your actual problems which probably aren't around provisioning compute anyway.
So; back to the point. Hooking up to JBODs and local disks in your boxes are probably using SAS. If you're actually dealing with a larger infra tho you're probably not lucky enough to have local storage across the estate, so you're rigging up sas to some build some SANs, or to some specific giant DB or other which should probably be left to a vendor anyway. If you're building out real shared storage anywhere which has more than a dozen racks per dc then it's almost certainly going to have to be fed out over the network.
Then it'll via be 10g fibre HBAs in your VM hosts, probably with a filesystem, then a file which'll show up as disks in the vms VM's. This is a really poor way to do it, but things are fast enough now that it mostly works, even if it gives us a bit of a scratchy feeling in our eyes. If you're not masochistic enough to deal with iSCSI (I'll never do it through choice; zfs+gluster+nfs is almost always the better choice) -- then you'll work out the simplicity of a NAS outweighs the whole SAN thing these days at almost any scale. It'll take a few massive outages until you really get there though. Before the claims otherwise, I've run multi-TB oracle DB's over NFS, we're not in...
42c per gig is VERY expensive, more than 10 times the price of 15k SAS at volume. SSDs - while I didn't say "are not used" in enterprise - are "generally" not used. As in, for every $10mill enterprise spends on SAS drives, they spend maybe a few hundred thou on SSD.
The market for SSDs is elsewhere.
googles to check. YUP http://www.anandtech.com/show/10348/q1-2016-market-trends-ss... http://www.anandtech.com/show/10098/market-views-2015-hard-d... WD alone ships 3 times more enterprise HDD than the entire enterprise SSD market. roughly speaking Enterprise HDD is about the same size as the entire SSD market.
My pissy comes from HN becoming more and more "pimp my company/product" and downvote to hell anyone remotely critical, with less "news about hacking".
In enterprise drives the drive can return quickly even if it's in the buffer because there's enough charge in the capacitor to whiteout to NAND from RAM in the event of power failure (hard reboot or power loss).
The consumer drive absent the capacitor either has poor fsync performance or lies about syncs.
There are people who run consumer drives in data centers due to the low GB/$ cost of them or they have applications that are not sensitive for data loss or were engineered to deal with it (replication, journaling, etc..). You won't want to run your important database on such a drive.
On Ubuntu, if you have installed PostgreSQL this is available at /usr/lib/postgresql/9.5/bin/pg_test_fsync
I get 4000 IOPS on my old Intel 320, which beats the crap out of most "top" performers on the consumer side today.
In SSDs, the big difference between Enterprise and Consumer is endurance. Flash memory wears out over time and the firmware manages the wear and maps out bad sectors over time. Eventually, too much wears out and the drive can no longer be written to, or is too slow to usably write to. The endurance spec tells you how much data you can expect to write before the drive must be replaced.
Micron sells both consumer and enterprise drives. The consumer 1100 SSDs max out at 400TB endurance which is fantastic for a consumer drive. These enterprise 5100 drives max out at 17.6PB endurance -- that's 44X better!
If you have an application which writes a lot of data, you should be evaluating whether or not you need something like this.
I've personally burnt out a set of SSDs installed in a RAID-6. They lost no data and failed no writes, but they had become unusably slow -- less than a tenth of their original write speed.
We burned out the drives with a 800Mb/s sustained 24/7 write stream. In post-mortem I did the math and the consumer drives ran out of endurance in less than a month. We replaced the drives with enterprise drives which were expected to last at least 3 years.
Also, here's a really strange finding... TRIM doesn't matter as much as one would expect. Many TRIM implementations are buggy. If the SSD contains a reliable TRIM implementation, then you can 100% guarantee that it will also have internal "move stable bits around to level out the usage" code.
[1] https://newsroom.intel.com/news-releases/intel-and-micron-pr...
FWIW an anonymous source emailed me that early next year there would be more news on this front. But I'll believe it when I see it.
It may also be faster: to read a cell you have to address it and then measure its value. With more bits per cell, you read/write more bits per address cycle.