I think it's something like the user holds private key A, publicizes public key A, and requests private key B such that public key A+B has the desired characteristics and private key A+B is only known if you have private key A.
PCIE slots are limited so when you are talking about actual compute scale the 1070 can be more expensive considering you'll have to get another case, cpu, ram, and an PSU that can handle 4 cards at max load.
And of course the more components you have in a given system the more likely is that one of the will fail, granted if you are designing a system to be resilient against attrition a system with more lower powered components can be more resilient than a system with fewer more powerful ones based on the performance difference between them and the MTBF for each component.
But bang for buck charts don't mean much even for Gaming I never understood why people even use them, spend what you can and want to get the performance you want.
Spending 50% more to get only 20% more performance is a valid option because those 20% can mean the difference between a playable game at high resolution and a laggy mess.
Traditionally SLI setups for mid range cards could out-perform and be cheaper than high end cards but with a huge caveat - being dependant on the SLI profile for the game, if it come out to be shitty or non-existent you are just stuck with 2 cards that can't really run the game at the settings/resolution you want.
Also unless you are building a small datacenter power consumption can be ignored, the 50-75W difference in total system draw between a mid range and a high end card when the rest of the hardware is the same isn't going to have any financial impact on anyone even if we take the most expensive power in Europe that's going to come out to about 30 cents per KWh.
Eh, the most expensive power in Europe is more at 60ct per kWh.
Average in northern Germany is already 38ct per kWh, although this is expected to change with the northern and southern German electricity market being split in the next weeks (Northern Germany has a huge surplus, Southern Germany not nearly enough).
Even so we are usually talking about 50W max draw difference between the mid and high end cards is still 20 hours of gaming at max load before you have 60 cents difference between them :P
Interesting. That sounds expensive actually? $0.60US for 20 hours of gaming just for the machine? 60 and 38 cent kWh sounds extremely high. This was my last power bill. As you can see it might average out to $0.05/kWh once you add in the taxes and regulatory fines.
I was taking the 60cents per KWh price, 50W difference is 1 KWh @ 20h of gaming, 20h of gaming is what the average gamer plays a week so given the gamer with the higher end card will pay 60 cents more per week 30 about 30$ more per year than the one with the mid range card given that all the other parts are identical and they both play 20 hours a week :)
I agree that for gaming it makes sense getting the higher end cards rather than dealing with SLI, but my use case is password cracking that doesn't require five 9's uptime.
The GTX 1070 is 150W power consumption while the 1080 is 180W. Rather than get another case i'd just scale up the case size based on the number of cards - 2 cards ($100[0]) 4 cards ($200[1]), 6 cards ($400[2])
1200W PSU $300[3], riser cards $10[4] and then motherboard + cpu + SSD about $200-250 so it works out $700 base for 2 GPU system, $900 base for 4 GPU system and $1400 for 6 GPU (2 PSU's)
If you work that out with the card options the sweet spot is 4x 1070 GPU's for $1500. It really isn't worth spending $4k on cards to get equiv performance from Titan X's or 15-30% more performance with the 1080 (although power/performance is better).
What motherboard can you buy for under 50 with 6 PCIeX16 (power) slots?
The cheap "bitcoin" motherboard that go for about 50-75$(old intel express chipset) won't work in this case, not sure about the 1070 but the 980 (non-ti) and the Titan X do not want to work with any PCIe slot that isn't a true PCIeX16 power slot.
The PCIe x1-4 to x6 riser converter boards do not work, even the "good" ones with a SATA power connector and an actual power PCB for some reason, my guess is that there is some actual sensing going on in the card itself.
Unless you buy something like this http://www.amazon.com/ASRock-Motherboard-H81-PRO-BTC/dp/B00H... which which is i think the only non-server motherboard which is commonly available you aren't getting more than 4-4 PCIe slots, and even to get 4 (PCIe16) you need to spend about 200$ on a motherboard these days and the H81 chipset mobo's don't want to work with modern nVIDIA cards (or the other way around) with all but the main PCIe16 slot.
Pretty much to get more than 2 PCIx16 slots these days you have to buy a good gaming Z170/Z97/X99 motherboard, that ain't cheap, nor are the CPU's that go into them.
While the AMD CPU's might be a bit cheaper again the motherboards won't be.
P.S. my personal password cracking setup at home is 4 5970's you could get them for about 100$ even a couple of years ago (today they are much cheaper 2 for 130 or b/o: http://goo.gl/yplISa) and for most hashes they go 40-50% of the perofmrnace of a Titan X (I do have 2 Titan X's in my personal gaming rig which i can use in a pinch).
I got them setup on an ASROCK motherboard (h61 iirc) that was designed for bitcoin mining so it comes with 8 PCIE slots and powered converter ribbons connect the cards do it all of this sits in a al cheapo rackmount case of a 20 year old compaq server.
But if you are looking for more scalable compute instance AWS CUDA instances are probably a better solution and I've been using them more and more lately.
I have been wondering about the AWS GPU instances. They are something like $2.60 / hour. At what point do they make sense monetarily? You can build a machine like described here for ~$5000 total. Thats not that many hours of AWS time. So if anyone is doing any long running cracking, or machine learning, does it ever make sense? I am truly curious about what people's experience has been.
g2.8xlarge $0.7414 per Hour (linux pricing) it makes sense unless you are doing it for a "hobby", you just roll costs into each project which makes it much easier from an accounting POV :P
If you buy Spot instances you can get them for ~85% discount off normal pricing. Your spot instances can be pre-empted, of course, but for password cracking this shouldn't really matter much.
Insidepro is a great resource, but you usually aren't going to get a VAT invoice to either bill your client for or deduct in your taxes ;)
AWS is simple, it's not overly expensive for most password cracking ventures (e.g. password quality audit that certain organization like to do on their AD/LDAP) and more importantly you aren't giving away potentially very sensitive information to people that by default cannot be trusted.
Inside pro is ok if you are going mostly non-legit / off the books to begin with, it's also ok for basic research on non-real world data, but never use it for actual real world data not to mention B2B services.
Huh, interesting. I know Amazon, at least, has provisions for HIPAA, [1] which seems pretty strict. Having to notify newspapers in case of breach and some pretty stiff fines. I know a bunch of places do credit card processing on AWS. PCI is pretty strict as well.
Is the big concern physical access? Or is there a subtle hack to escape the VM you're running in? I'm just having a hard time seeing the difference between a vm connected to the internet at the office, a colo server and a aws instance. I guess you must be referring to minimizing surface area, not a proven attack.
It's tempting to trust your team, and you need to. If you're doing real, regular audits of access controls and have some pretty robust physical security, that could be a win. Seems like you'd have to be a pretty big shop to do security "right".
A sophisticated attacker with access to Amazon's infrastructure can read anything they like out of the memory of the machine your instance is on without you or anyone else ever knowing it.
Doesn't need to be airgapped, just defended, and not in the cloud.
If the client is someone like Juniper or Cisco (which, if you're a pentest firm buying 8x gpu servers for cracking hashes, would be a reasonable customer) nation state actors would absolutely love to have access to your dataset. And they're already in AWS's datacentres.
With your cracking setup I can definitely see the attraction of building multiple cheaper boxes rather than one larger one. This is definitely something I need to spend more time researching (which i'd intended)
I'm using AWS at the moment - I fire up spot instances but have noticed that there has been a lot of demand recently and prices spike a lot. I've written some basic scripts to bring instances back up, continue tasks, send jobs, shutdown on complete, etc. I'm spending a lot on AWS at the moment which is why i'm looking at putting a new rig together (my old rig broke down a year ago and is 6+ years old)
I don't know if the 970 will work, don't have one, IIRC the GTX770 doesn't want to work either, but I don't remember if i tested the 770/780ti's i had lying around from older gaming rigs.
The AM3+ mobo's might works, just make sure to get one that actually has been rated for 4-way xfire since I've noticed that quite a few of the mid range ones(regardless of being Intel or AMD) don't actually support more than 2 PCIEx16 power ones at the same time (none of them come with 48 PCIE lanes to begin with).
According to wikipedia:
990FX
Four physical PCIe 2.0 ×16 slots @ x8 electrical which can be combined to create two PCIe 2.0×16 slots @ x16 electrical, one PCIe 2.0×4 slot and two PCIe 2.0×1 slots, the chipset provides a total of 38 PCIe 2.0 lanes and 4 PCIe 2.0 for A-Link Express III solely in the Northbridge
Looks like you'll be getting either 2 x16 electrical with the rest disabled when a card is inserted (if the OEM decided to invest in some extra hardware) or and 4x8 electrical.
x8 electrical is usually enough for (most) Nvidia cards to work, not sure about the 1070/1080, but you probably want to confirm it.
You can ignore the PCIe land bandwidth since it's not a factor for (most) compute applications, but the electrical stuff is sadly a pitfall, this gen could even be potentially worse since allot of the OEM's dropped to a single power connector (6pin on 1070, and 6/8pin on 1080 non OC cards) which might indicate they are planning to draw more power from the PCIe bus.
The fact that Nvidia kinda also dropped support for 3 and 4 way SLI (which now requires an "enthusiast" code that is written to the bios of your card) could also introduce some issues if OEM's pretty much bet that you'll only plug the card into an X16 electrical slot even in SLI so the cards might either not work or not be stable if they are connected to an X8 one (x1/x4 shouldn't work at all unless nvidia did a complete 1080 from the previous gen).
P.S. if you are building everything from the ground up and not using old hardware that you already own I would look at old (like 2nd gen Core based Xeon, or old Opteron) servers with lots of PCie slots, you can usually find them for a couple 100's $ on ebay.
Thanks again - i've been digging into the specs of those boards and it does look like you have to go pretty far down that list to get to a board that can support 4x PCI-E v3 x16 simultaneously so you were right with your first comment that it is $250+
HP DL380's which can take 2 cards for $200 - pretty good.
I'm using these adapters that give me a PCIe x16 mechanical connection as well as the extra juice via a molex/sata power conector:
http://www.frozencpu.com/images/products/main/cab-1907.jpg
So i don't think I need short the detection contacts to ground i think because the PCIex16 mechanical slot is does it for me (the AMD cards work without any modification).
The Titan X doesn't want to work with these, I never paid it much thought since i wasn't going to fill the machine i have with Titan X's anyhow, I just wondered if i can use them when i retire them (i filled with 5970's since i got 4 of those for about 300$ a about a year ago).
I will investigate this further if at any point I'll retire the Titan X's and decide to replace my some of the AMD cards, atm I'm still using the Titan's for gaming, and I really don't want to build a mixed GPU rig since it won't give me any boost to performance unless i somehow manually batch my hashes since iirc hashcat doesn't know how to address multi vendor GPU's my itself.
In my experience many PCIe extenders—even those extending x1 to x16 with a separate power plug—do not short the presence detect pins.
Also, I don't think the fact AMD cards work does not mean shorting won't be required for other cards. The PCIe initialization process is complex enough that there could be a variety of reasons why AMD cards can be down-plugged but not your Titan X.
In short (pun intended) I would highly recommend you to try shorting if you feel confident in doing it.
The problem is then you are spending about 2000$ on your "chassis"(case, cpu, motherboard, memory, cooling etc.) if you are in that situation then getting the more expensive cards (1080/1080ti or even the Titan w/e they'll name it now) is usually the only way to go because your chassis is now a major factor of the final price so you'll need to squeeze a much performance from it as possible.
Yes, I can confirm this is the case (pun intended). I've been speccing out GPU machines for molecular simulation, and any way you slice it, the actual GPUs account for less than 50% of your total cost, as long as you're looking at GTX class cards.
The other issue with 8 GPU machines, which tend to use dual CPU motherboards to have enough PCIe lanes, is that you pay a serious performance penalty for communication between GPUs attached to different CPUs (over QPI). It's so bad that I've seen several standalone 8 GPU systems using FDR Infiniband (with GPUdirect RDMA) to communicate internally on the machine. Which is of course bloody expensive.
If it suits your workload, having two separate 4 GPU machines is better and cheaper than a single 8 GPU machine.
Can you actually get full performance out of that? I thought the bottleneck these days is the CPU which maxes out at 40 lanes at the high end. This only gets you 1x16 and 3x8 even if all the slots support 16 lanes.
Physical space on the motherboard IMO is the true limiting factor here. And this is what made Google's TPU interesting IMO. I don't think it demonstrated world-beating performance, but it likely fit a strict power and size envelope as an add-on board in the their datacenters, hence all the bragging about perf/W without a single absolute performance benchmark.
>But bang for buck charts don't mean much even for Gaming I never understood why people even use them
I've never bough the highest end card for gaming in my life. The price premium is close to double and you get maybe 20-30% the performance. Worse, a lot of that performance is for gaming bragging rights like playing absurdly high resolutions or having very high settings. I can barely tell the difference between high and ultra in most games anyway. Medium to High can also be a crapshoot if medium is at a decently high resolution.
Not to mention a lower tier card uses 60-100 watts less, which means I don't need to upgrade my mid-range power supply either. I find that high watt ps's are fairly delicate as well and blow frequently, but that may be only my experience. I have 400 and 500 watt ps's that run for nearly a decade with no issues but every 600+ I've had to buy blows by year two or three.
Worse, the cards don't age well. A 980ti isn't going to be good for VR sooner than later because the 1000 series will support a single viewport for both eyes (SMProjection) which means a performance boost of anywhere between 20-50% in VR. You're better off spending your dollars on a 960/970 range card and just pocketing that extra $200-$300 for an earlier upgrade to the next series instead of sitting on the higher version of an old series too long. With SMProjection, even a 1060 is going to outperform a 980ti bought recently for VR. That'll be a $199 card killing a $600 card in VR.
Just to play devil's advocate if I may. Of course the longer you wait the more you get for your money. The difference is you didn't get to enjoy a 980Ti for a long time and someone who bought one did. Money can't buy that, it's priceless. The type of people running a 980Ti probably already have a 1080 anyway. :) While everyone else is snickering about how these wealthy folks somehow lost.
People who have a Geforce 1060 budget, probably won't have a HTC Vive either to worry about beating 980Ti VR performance..
I think the PSU thing is anecdotal. Not enough information to explain that either way. I generally go for a decent wattage PSU but worry a lot more about the 12v rail. I've had my existing Corsair HX650 for many years now and among many reasons chose it for 52A single 12v rail. Rock solid PSU.
I do agree that just recently has buying less-than-top end has started to make sense. But I disagree that historically that move has been as sensible as it is now. I agree with the OP. I always bought top tier hardware until recently because we actually needed the performance. I always paid the price when stepping down, that 20-30% was needed for a very long time. I've been using 256bit memory bandwidth GPUs since at least 2002. There was a huge gap between them and the lesser tier 128bit stuff. Again, some things are priceless. Every cent I spent on my original 3dfx Voodoo card in 1996 was worth it. Even if I could've saved a few bucks going with a Rendition Verite.
Today, not as much as the market has matured if you discount VR. I'm probably done buying high end for desktop machines but I think for decades now, it's been well worth it. Example, I was dismayed at the slow FPS my Geforce 8800GT offered me. When I upgraded to the 8800GTX it was very much worth it.
Even so, my PC spending is actually going up though from historic numbers. I'm buying gaming laptops from here on out, something I've never done before. Waiting for a Razer Blade with Geforce 1080M so I can use it with a Vive in a wirefree backpack configuration. Similar story to the above, it's expensive: but can you put a price on completely wireless roomspace VR? Not in my opinion. While I'll likely use this in VR then also have a docking station for it, for my 2nd PC, I'm moving to the Skull Canyon NUC and successors.
The hashcat project could make a bunch of money with cracking as-a-service: you supply hashes, hashcat runs on a dedicated multi-GPU instance (SoftLayer etc provides these), and get outputs.
For users: no setting up hardware or software and much better price / performance than DIY
For hashcat developers: money to pay their rent and work on hashcat
Edit: looks like the company sells SW/HW but not as-a-service. Massive opportunity for them there: https://sagitta.pw/software/
Can you point me to some resources for learning how to do this? Starting with capturing my wi-fi traffic to running the right tools to crack it? I'd like to see how long it takes me to crack my own password.
There are quite a lot of guides out there on sites like hackforums I reckon. The search keywords are "aircrack-ng WPA".
In short terms, you need to sniff the 4-way handshake between a legitimate client and the AP it connects to. This can sometimes be forced by spoofing a "deauth" (disconnect) packet from the client, but requires support in the chipset for your wlan-card. (Injection and monitor mode).
Once you have the handshake, your options are either aircrack-ng, hashcat or other password cracking tools. Some of these have a crazy amount of options and possibilities for cracking, and getting to know them can increase your success rate by a lot.
However, the easiest way these days is to exploit WPS in the AP. Look up Reaver and Pixiewps.
This[0] guide goes over how to use WiFite in Kali Linux[1] to capture a wpa handshake and how to use hashcat to crack it. You have to have a wifi-card that is compatible with aircrack-ng[2].
Things like this really make me consider using WPA-Enterprise on my home AP, enabling RADIUS on my pfSense VM wouldn't be too hard. I don't particularly care about the guest network, it's on a separate VLAN and has no access to the rest of the network (I should probably throttle the traffic).
There was an earlier discussion about KeePassX, where I mentioned I used 10 million rounds of the AES encryption. Someone commented that it doesn't really add much additional strength.
I'm not familiar enough with the topic to understand why it would be so. I wonder if there is some basis for that claim?
[Disclaimer: This is assuming you aren't talking about key schedule, since you use odd terminology, and even then how do you get to 10M rounds is beyond me]
Because you use the same key, not to mention the same IV most likely, unless you are storing 10M IV's some where...
Doing 10M rounds of AES is just stupid(ly) expensive.
Also verifying the decryption with 10M rounds, means, 10M CRC checks, and if you are using it on a non-ECC system there is a good chance that many attempt to decrypt anything stored like that will fail due to single bit errors.
He's talking about KeePass's key derivation algorithm - SHA256 the password, make a random 256 bit key, AES-encrypt the hash N times using the key, and SHA256 the result.
10 million iterations is about a second of work on a typical desktop.
I don't have expertise on the subject as was probably already evident. For me it's a setting in KeePassX, and that's it.
With the 10 million config, opening the password database takes just a bit over a second on i7-4800MQ which should come with AES-NI. I can't say if the software is using that or not.
On my Android it takes 4 seconds, so very acceptable still.
Interesting that the GOST-R 512bit seems no slower than the 256bit hashing.
Does this mean the 256bit is underperforming or is the 512bit flavour just as compute intensive as its lesser counterpart with half the number of bits. Least for me seems to stick out a bit(sic).
There's not even any mention of training neuralnets, rendering things or doing science. Are there many non-shady reasons for purchasing the above device based on the purpose described?
FWIW password cracking is not necessarily a shady use-case. At my old job the infosec team had a GPU cluster continuously trying to crack employees' passwords and would force users to change passwords if they succeeded.
A number of molecular dynamics, astrophysics, and similar simulation applications are scalable enough to use 8 GPUs per node. Especially when multiple nodes are involved, these scientific applications are in general more mature compared to deep-learning frameworks, which have just started supporting multiple nodes.
It just goes to show that in 2016, you really can't be using SHA1/256/512 for password hashing anymore, even salted. Bcrypt/Scrypt or similar is a must.
From what I can tell with reasonable password length SHA512 still seems relatively secure. I'm not saying there is any reason you should use it - but if it does 8624.7Mh/s and your password is JUST numbers, lowercase and uppercase English letters and it's only 10 characters it would still take it (26+26+10)^10 / 8624.7Mh/s ~= 83929936 seconds which is still about 3 years for a single hash.
If it's 12 characters - then you're up to 10,000 years.
True - reasonable password length being the important factor. I was basing my calculations on 8 character passwords.
At my university we were forced to use passwords that are exactly 8 characters long, for some ridiculous reasons. And this was meant to be one of the top universities in Australia...
> Originally, some developer, somewhere, was working on an old Unix system from the previous century, which used the old DES-based "crypt", actually a password hashing function derived from the DES block cipher. In that hashing function, only the first eight characters of the password are used (and only the low 7 bits of each character, as well). Subsequent characters are ignored. That's the banana.
Just a hash or a single salted hash iteration has been on the bad practice list for a very long time. Hashes are designed to be fast, key derivation functions are intended to be slow.
There are a number of KDFs that build on top of the SHA family, from PBKDF2-HMAC-SHA1 to sha512crypt, that are all fine in principle but are often used with iteration counts that are off by 1 or 2 orders of magnitude.
86 comments
[ 3.6 ms ] story [ 154 ms ] threadhttp://www.videocardbenchmark.net/high_end_gpus.html#value
The 1080 is $599 while the 1070 will be $379 while the Titan X is ~$1000.
$1,500 for a 4x 1070 setup could be very attractive considering the performance should be on par with 4x Titan X (at $4k)
edit: forgot to mention power consumption as well, which should be a component of price/performance
edit: https://www.oniongen.com/
But bang for buck charts don't mean much even for Gaming I never understood why people even use them, spend what you can and want to get the performance you want. Spending 50% more to get only 20% more performance is a valid option because those 20% can mean the difference between a playable game at high resolution and a laggy mess. Traditionally SLI setups for mid range cards could out-perform and be cheaper than high end cards but with a huge caveat - being dependant on the SLI profile for the game, if it come out to be shitty or non-existent you are just stuck with 2 cards that can't really run the game at the settings/resolution you want.
Also unless you are building a small datacenter power consumption can be ignored, the 50-75W difference in total system draw between a mid range and a high end card when the rest of the hardware is the same isn't going to have any financial impact on anyone even if we take the most expensive power in Europe that's going to come out to about 30 cents per KWh.
Average in northern Germany is already 38ct per kWh, although this is expected to change with the northern and southern German electricity market being split in the next weeks (Northern Germany has a huge surplus, Southern Germany not nearly enough).
Tier 1 first 500 kWh at $0.018 per kWh (winter) . . . . . . . . . . . . . . . . . . . . . . .$9.00
Tier 2 next 25 kWh at $0.056 per kWh (winter) . . . . . . . . . . . . . . . . . . . . . . . .$1.40
Regulatory Charges 525 kWh at $0.01414 per kWh . . . . . . . . . . . . . . . . . . . . .$7.42
Community Benefit Charges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .$2.91
Power Supply Adjustment 525 kWh at $0.02783 per kWh . . . . . . . . . . . . . . .$14.61
Residential Sales Tax
Taxable Amount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .$45.34
City Sales Tax 1% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .$0.45
TOTAL CURRENT CHARGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .$45.79
The GTX 1070 is 150W power consumption while the 1080 is 180W. Rather than get another case i'd just scale up the case size based on the number of cards - 2 cards ($100[0]) 4 cards ($200[1]), 6 cards ($400[2])
1200W PSU $300[3], riser cards $10[4] and then motherboard + cpu + SSD about $200-250 so it works out $700 base for 2 GPU system, $900 base for 4 GPU system and $1400 for 6 GPU (2 PSU's)
If you work that out with the card options the sweet spot is 4x 1070 GPU's for $1500. It really isn't worth spending $4k on cards to get equiv performance from Titan X's or 15-30% more performance with the 1080 (although power/performance is better).
[0] http://www.coolermaster.com/case/mid-tower-cm690-series/cm69... [1] http://www.corsair.com/en/obsidian-series-900d-super-tower-c... [2] http://www.miningrigs.net/?product=graymatter-6gpu-server-ca... [3] http://www.newegg.com/Product/Product.aspx?Item=N82E16817139... [4] http://www.amazon.com/Fixable-Adapter-Flexible-Extension-Con...
Unless you buy something like this http://www.amazon.com/ASRock-Motherboard-H81-PRO-BTC/dp/B00H... which which is i think the only non-server motherboard which is commonly available you aren't getting more than 4-4 PCIe slots, and even to get 4 (PCIe16) you need to spend about 200$ on a motherboard these days and the H81 chipset mobo's don't want to work with modern nVIDIA cards (or the other way around) with all but the main PCIe16 slot.
Pretty much to get more than 2 PCIx16 slots these days you have to buy a good gaming Z170/Z97/X99 motherboard, that ain't cheap, nor are the CPU's that go into them. While the AMD CPU's might be a bit cheaper again the motherboards won't be.
P.S. my personal password cracking setup at home is 4 5970's you could get them for about 100$ even a couple of years ago (today they are much cheaper 2 for 130 or b/o: http://goo.gl/yplISa) and for most hashes they go 40-50% of the perofmrnace of a Titan X (I do have 2 Titan X's in my personal gaming rig which i can use in a pinch). I got them setup on an ASROCK motherboard (h61 iirc) that was designed for bitcoin mining so it comes with 8 PCIE slots and powered converter ribbons connect the cards do it all of this sits in a al cheapo rackmount case of a 20 year old compaq server.
But if you are looking for more scalable compute instance AWS CUDA instances are probably a better solution and I've been using them more and more lately.
For password cracking? Never, especially considering you can just go on insidepro or something and rent access to someones cluster.
AWS is simple, it's not overly expensive for most password cracking ventures (e.g. password quality audit that certain organization like to do on their AD/LDAP) and more importantly you aren't giving away potentially very sensitive information to people that by default cannot be trusted.
Inside pro is ok if you are going mostly non-legit / off the books to begin with, it's also ok for basic research on non-real world data, but never use it for actual real world data not to mention B2B services.
Is the big concern physical access? Or is there a subtle hack to escape the VM you're running in? I'm just having a hard time seeing the difference between a vm connected to the internet at the office, a colo server and a aws instance. I guess you must be referring to minimizing surface area, not a proven attack.
It's tempting to trust your team, and you need to. If you're doing real, regular audits of access controls and have some pretty robust physical security, that could be a win. Seems like you'd have to be a pretty big shop to do security "right".
[1] https://aws.amazon.com/compliance/hipaa-compliance/
A sophisticated attacker with access to Amazon's infrastructure can read anything they like out of the memory of the machine your instance is on without you or anyone else ever knowing it.
If the client is someone like Juniper or Cisco (which, if you're a pentest firm buying 8x gpu servers for cracking hashes, would be a reasonable customer) nation state actors would absolutely love to have access to your dataset. And they're already in AWS's datacentres.
That is exactly the motherboard that I was thinking of - I just rounded down a little too aggressively (iirc you can get them for $69 on NewEgg).
> not sure about the 1070 but the 980 (non-ti) and the Titan X do not want to work with any PCIe slot
I'd completely forgotten about this - thanks. When I initially started setting out a new rig build I was looking at the 970 which would work.
Here is pcpartpicker filtered with at least 4x PCIex16:
http://pcpartpicker.com/products/motherboard/#h=4,7&sort=a8
It brings this board up:
http://pcpartpicker.com/product/3YH48d/gigabyte-motherboard-...
Think that works?
With your cracking setup I can definitely see the attraction of building multiple cheaper boxes rather than one larger one. This is definitely something I need to spend more time researching (which i'd intended)
I'm using AWS at the moment - I fire up spot instances but have noticed that there has been a lot of demand recently and prices spike a lot. I've written some basic scripts to bring instances back up, continue tasks, send jobs, shutdown on complete, etc. I'm spending a lot on AWS at the moment which is why i'm looking at putting a new rig together (my old rig broke down a year ago and is 6+ years old)
The AM3+ mobo's might works, just make sure to get one that actually has been rated for 4-way xfire since I've noticed that quite a few of the mid range ones(regardless of being Intel or AMD) don't actually support more than 2 PCIEx16 power ones at the same time (none of them come with 48 PCIE lanes to begin with).
According to wikipedia: 990FX Four physical PCIe 2.0 ×16 slots @ x8 electrical which can be combined to create two PCIe 2.0×16 slots @ x16 electrical, one PCIe 2.0×4 slot and two PCIe 2.0×1 slots, the chipset provides a total of 38 PCIe 2.0 lanes and 4 PCIe 2.0 for A-Link Express III solely in the Northbridge
Looks like you'll be getting either 2 x16 electrical with the rest disabled when a card is inserted (if the OEM decided to invest in some extra hardware) or and 4x8 electrical. x8 electrical is usually enough for (most) Nvidia cards to work, not sure about the 1070/1080, but you probably want to confirm it.
You can ignore the PCIe land bandwidth since it's not a factor for (most) compute applications, but the electrical stuff is sadly a pitfall, this gen could even be potentially worse since allot of the OEM's dropped to a single power connector (6pin on 1070, and 6/8pin on 1080 non OC cards) which might indicate they are planning to draw more power from the PCIe bus. The fact that Nvidia kinda also dropped support for 3 and 4 way SLI (which now requires an "enthusiast" code that is written to the bios of your card) could also introduce some issues if OEM's pretty much bet that you'll only plug the card into an X16 electrical slot even in SLI so the cards might either not work or not be stable if they are connected to an X8 one (x1/x4 shouldn't work at all unless nvidia did a complete 1080 from the previous gen).
P.S. if you are building everything from the ground up and not using old hardware that you already own I would look at old (like 2nd gen Core based Xeon, or old Opteron) servers with lots of PCie slots, you can usually find them for a couple 100's $ on ebay.
HP DL380's which can take 2 cards for $200 - pretty good.
You should try shorting the presence detection contacts, as described in the middle section of my post:
http://blog.zorinaq.com/?e=42
That was years ago I did it, but it never failed in making any x16 card work in an x1 PCIe slot.
The Titan X doesn't want to work with these, I never paid it much thought since i wasn't going to fill the machine i have with Titan X's anyhow, I just wondered if i can use them when i retire them (i filled with 5970's since i got 4 of those for about 300$ a about a year ago).
I will investigate this further if at any point I'll retire the Titan X's and decide to replace my some of the AMD cards, atm I'm still using the Titan's for gaming, and I really don't want to build a mixed GPU rig since it won't give me any boost to performance unless i somehow manually batch my hashes since iirc hashcat doesn't know how to address multi vendor GPU's my itself.
Also, I don't think the fact AMD cards work does not mean shorting won't be required for other cards. The PCIe initialization process is complex enough that there could be a variety of reasons why AMD cards can be down-plugged but not your Titan X.
In short (pun intended) I would highly recommend you to try shorting if you feel confident in doing it.
(tho you'd have to cut holes in the case to bolt radiator fans onto)
The other issue with 8 GPU machines, which tend to use dual CPU motherboards to have enough PCIe lanes, is that you pay a serious performance penalty for communication between GPUs attached to different CPUs (over QPI). It's so bad that I've seen several standalone 8 GPU systems using FDR Infiniband (with GPUdirect RDMA) to communicate internally on the machine. Which is of course bloody expensive.
If it suits your workload, having two separate 4 GPU machines is better and cheaper than a single 8 GPU machine.
I'm inclined towards the hypothesis of parent being a pentester and buying for their freelance business, or company.
I've never bough the highest end card for gaming in my life. The price premium is close to double and you get maybe 20-30% the performance. Worse, a lot of that performance is for gaming bragging rights like playing absurdly high resolutions or having very high settings. I can barely tell the difference between high and ultra in most games anyway. Medium to High can also be a crapshoot if medium is at a decently high resolution.
Not to mention a lower tier card uses 60-100 watts less, which means I don't need to upgrade my mid-range power supply either. I find that high watt ps's are fairly delicate as well and blow frequently, but that may be only my experience. I have 400 and 500 watt ps's that run for nearly a decade with no issues but every 600+ I've had to buy blows by year two or three.
Worse, the cards don't age well. A 980ti isn't going to be good for VR sooner than later because the 1000 series will support a single viewport for both eyes (SMProjection) which means a performance boost of anywhere between 20-50% in VR. You're better off spending your dollars on a 960/970 range card and just pocketing that extra $200-$300 for an earlier upgrade to the next series instead of sitting on the higher version of an old series too long. With SMProjection, even a 1060 is going to outperform a 980ti bought recently for VR. That'll be a $199 card killing a $600 card in VR.
People who have a Geforce 1060 budget, probably won't have a HTC Vive either to worry about beating 980Ti VR performance..
I think the PSU thing is anecdotal. Not enough information to explain that either way. I generally go for a decent wattage PSU but worry a lot more about the 12v rail. I've had my existing Corsair HX650 for many years now and among many reasons chose it for 52A single 12v rail. Rock solid PSU.
I do agree that just recently has buying less-than-top end has started to make sense. But I disagree that historically that move has been as sensible as it is now. I agree with the OP. I always bought top tier hardware until recently because we actually needed the performance. I always paid the price when stepping down, that 20-30% was needed for a very long time. I've been using 256bit memory bandwidth GPUs since at least 2002. There was a huge gap between them and the lesser tier 128bit stuff. Again, some things are priceless. Every cent I spent on my original 3dfx Voodoo card in 1996 was worth it. Even if I could've saved a few bucks going with a Rendition Verite.
Today, not as much as the market has matured if you discount VR. I'm probably done buying high end for desktop machines but I think for decades now, it's been well worth it. Example, I was dismayed at the slow FPS my Geforce 8800GT offered me. When I upgraded to the 8800GTX it was very much worth it.
Even so, my PC spending is actually going up though from historic numbers. I'm buying gaming laptops from here on out, something I've never done before. Waiting for a Razer Blade with Geforce 1080M so I can use it with a Vive in a wirefree backpack configuration. Similar story to the above, it's expensive: but can you put a price on completely wireless roomspace VR? Not in my opinion. While I'll likely use this in VR then also have a docking station for it, for my 2nd PC, I'm moving to the Skull Canyon NUC and successors.
1070: http://www.hardocp.com/article/2016/06/06/nvidia_geforce_gtx...
Other benchmarks: http://www.trustedreviews.com/nvidia-geforce-gtx-1080-review...
One question for the knowledgeable: Why are speeds for PostgreSQL hashes so high (~ 25k MH/s) compared with, say, MSSQL(2012) (~ 1k MH/s)?
The hashcat project could make a bunch of money with cracking as-a-service: you supply hashes, hashcat runs on a dedicated multi-GPU instance (SoftLayer etc provides these), and get outputs.
For users: no setting up hardware or software and much better price / performance than DIY
For hashcat developers: money to pay their rent and work on hashcat
Edit: looks like the company sells SW/HW but not as-a-service. Massive opportunity for them there: https://sagitta.pw/software/
Edit 2: they're doing it: Q1 2017 https://twitter.com/jmgosney/status/740146970254147584
22.5 minutes (11.25 on average) to crack a randomized 8 hex digit password that a good number of wifi modem/routers come with (mine certainly did)
In short terms, you need to sniff the 4-way handshake between a legitimate client and the AP it connects to. This can sometimes be forced by spoofing a "deauth" (disconnect) packet from the client, but requires support in the chipset for your wlan-card. (Injection and monitor mode).
Once you have the handshake, your options are either aircrack-ng, hashcat or other password cracking tools. Some of these have a crazy amount of options and possibilities for cracking, and getting to know them can increase your success rate by a lot.
However, the easiest way these days is to exploit WPS in the AP. Look up Reaver and Pixiewps.
[0]https://www.blackmoreops.com/2014/03/27/cracking-wpa-wpa2-wi... [1]https://www.kali.org/downloads/ [2]http://www.aircrack-ng.org/doku.php?id=compatible_cards
Tip: To require 65535 rounds of hashing on a Debian or Ubuntu system just run this:
(Note: Assumes you're still using the default hashing SHA-512 hashing algorithm)I'm not familiar enough with the topic to understand why it would be so. I wonder if there is some basis for that claim?
Also verifying the decryption with 10M rounds, means, 10M CRC checks, and if you are using it on a non-ECC system there is a good chance that many attempt to decrypt anything stored like that will fail due to single bit errors.
10 million iterations is about a second of work on a typical desktop.
With the 10 million config, opening the password database takes just a bit over a second on i7-4800MQ which should come with AES-NI. I can't say if the software is using that or not.
On my Android it takes 4 seconds, so very acceptable still.
* bcrypt: 2^5, when 2^10 or 2^11 is a much more typical figure
* scrypt: N=1024 r=1 -> 128KiB. For interactive logins the recommendation is N=16384 r=8 -> 16MiB.
LastPass is also off by a factor of 10 (500 rounds, when they default to 5000).
https://github.com/hashcat/oclHashcat/blob/master/include/sh...
Does this mean the 256bit is underperforming or is the 512bit flavour just as compute intensive as its lesser counterpart with half the number of bits. Least for me seems to stick out a bit(sic).
"Hashtype: GOST R 34.11-2012 (Streebog) 256-bit
Speed.Dev.#1.: 50018.8 kH/s (334.18ms) Speed.Dev.#2.: 49784.4 kH/s (332.43ms) Speed.Dev.#3.: 51323.1 kH/s (325.52ms) Speed.Dev.#4.: 50947.0 kH/s (327.87ms) Speed.Dev.#5.: 51510.6 kH/s (329.13ms) Speed.Dev.#6.: 50417.3 kH/s (331.39ms) Speed.Dev.#7.: 50825.4 kH/s (333.57ms) Speed.Dev.#8.: 50853.3 kH/s (333.38ms) Speed.Dev.#.: 405.7 MH/s
Hashtype: GOST R 34.11-2012 (Streebog) 512-bit
Speed.Dev.#1.: 49979.4 kH/s (329.40ms) Speed.Dev.#2.: 49849.7 kH/s (330.26ms) Speed.Dev.#3.: 50315.4 kH/s (336.13ms) Speed.Dev.#4.: 50305.3 kH/s (328.99ms) Speed.Dev.#5.: 51486.7 kH/s (326.69ms) Speed.Dev.#6.: 49709.2 kH/s (328.72ms) Speed.Dev.#7.: 51328.8 kH/s (330.30ms) Speed.Dev.#8.: 51530.7 kH/s (326.41ms) Speed.Dev.#.: 404.5 MH/s"
There's not even any mention of training neuralnets, rendering things or doing science. Are there many non-shady reasons for purchasing the above device based on the purpose described?
If it's 12 characters - then you're up to 10,000 years.
At my university we were forced to use passwords that are exactly 8 characters long, for some ridiculous reasons. And this was meant to be one of the top universities in Australia...
> Originally, some developer, somewhere, was working on an old Unix system from the previous century, which used the old DES-based "crypt", actually a password hashing function derived from the DES block cipher. In that hashing function, only the first eight characters of the password are used (and only the low 7 bits of each character, as well). Subsequent characters are ignored. That's the banana.
There are a number of KDFs that build on top of the SHA family, from PBKDF2-HMAC-SHA1 to sha512crypt, that are all fine in principle but are often used with iteration counts that are off by 1 or 2 orders of magnitude.
5000 iterations sounds like a lot. It is not.
However it is uncrippled on GP100 (ie. Tesla P100).