I've had this happen back in on a modular 1100 watt psu on my 8800 Ultra..
The system was unstable at high graphics load, and it took me a long time to find the fault, the PCI-E power connector was high-resistance at the PSU end, the connector was totally melted when I found it, the instability was due to the gfx card browning out from insufficient power supply due to the high connector resistance.
A loose connector on my RX580 did the same to me, could never explain the weird resets even with power draw diagnosis until I opened the case and just "wtf"
yes, by all means, add additional components, complexity and digital logic to detect the overheating of an underspecced component, that sound much more reasonable than using a connector that can carry the required current without heating up.
The color a black body emits at certain temperatures follow the planckian locus (https://en.wikipedia.org/wiki/Planckian_locus). The only saturated primary color on that curve is red.
Even easier to use the old standard instead of making a new solution to a problem that didn't exist in the first place (except for aesthetic reasons).
The old connectors were solid and had plenty of headroom if used within spec, with the new cards you'd still just need 3-4 to be well within spec - no monitoring circuitry needed.
The PCI-SIG maximum safe rating for the old and janky 8-pin connectors was 150 watt, despite how bulky it was. The 4090 would need 4 of those (5 if you account for when it exceeds 600 watt).
A new connector was absolutely necessary, even for more reasonable GPU power levels. That the current connector is broken is a different problem, and could likely be fixed without manufacturing changes without a new overall connector design.
Then when they redesign the spec they should have made the connector they used larger.
There's a problem with running 600 Watts through that connector, if this fire thing is regular issue it's going to burn someone's house down or kill someone
Bad contact shouldn't be an issue. You should have to be hilariously and obviously bad with the contact in order for it to be a problem. If slightly bent cables are the problem, it's clear that your cable is rated far too high for what it is and you need to mandate larger conductors or more conductors carrying less current so that there is no chance it burns unless the cable is very badly damaged or misused.
Bad contact shouldn't be an issue, but as the connector is overheating it is an issue. The only challenge in making a power connector is guaranteeing strong enough contact on every use. It could be caused by all sorts of things like manufacturing tolerance or exact springiness of the material of the pins.
The issue bending the cable near the connector is that it causes the connector pins to deflect, making the flimsy contact worse.
More conductor is unnecessary, it just needs to make a more reliable contact.
But the new connector is still only rated for 600W (if used within spec), if we directly compare that to using 4 of the old connectors (rated at 150W each) we're spreading the same load over many more pins.
A 12VHPWR connector is using 12 pins for GND/12V.
An 8pin PCIe connector has 6 pins for GND/12V, with 4 connectors that's 24 pins total.
Assuming that the pins are fairly similarly specced, the old connector should be safer, especially when exceeding the rated power. Regardless of how you frame it, it seems like aesthetics got prioritized over safety.
> But the new connector is still only rated for 600W (if used within spec),
I think that's the problem right there: running the connection at its rated limit. It implies that a damaged or slightly out-of-spec connector could fail, as we see happened here.
Defensive engineering involves safety margins for safety-relevant specifications. Running a digital signal at the limits of its specification is one thing; data corruption is a detectable problem and not inherently unsafe. Running power connections at rated limits is something else entirely because the failure modes are physical damage, fire, and electrical arcing.
600W of power delivery is no joke, and I'd like to see that power carried over cabling and connectors with absolute maximum ratings much greater than that, say 1kW. Per the specs, it seems that each of the 12 current-carrying conductors in the connector runs about 9A. A dodgy connection that results in just 1-ohm of resistance would then represent 80W lost as heat at that point.
Frankly, I think that running 600W over a single 12V multi-core cable is insane. You're talking 50A of current, or about the same current rating as the largest typical home circuits for an electric kitchen appliance or car charger. In the home, that calls for thick structural cabling, but it appears (https://www.cable-sleeving.com/pcie5-12vhpwr-16pin-connector) that the in-case recommendations are to use 16AWG wire or even a touch thinner.
The power pins are in the spec rated for 9.2A each, for a total of 662.4W@12V. To be compliant the connector must handle at least that. At the same time, the max advertised capacity is 600W, so a device must pull at most that.
So by the spec it must have at 10% headroom - and to live up to that, manufacturers must add their own margin on top. Rating for 1kW would be a 66% margin and a humongous waste of copper, and it wouldn't solve this contact issue in the first place.
Regarding wire gauge - unrelated to the problem at hand - 16AWG is already overkill. For house installation in EU, we rate that gauge for 13A to 18.5A depending on thermal conditons, with 13A being "very poor" thermal conditions (no cooling and surrounded by other hot things). With 6 pairs in parallel, that gives a conservative minimum of 78A capacity - and with the excellent thermal conditions of a case with fans ("very good" is normally just a dangling cable, not one with fans cooling it), the realistic capacity should exceed 120A. As you say, we need only 50A, so that's a 140% margin! Even 20AWG would be plenty here.
More pins is not safer, but just increases the risk of a single failed pin without mitigating the possible damage. Pins with good contact are safer than those without.
Pulling 50A through a connector of that size or smaller is in itself not an issue. As mentioned elsewhere, powerpole is smaller than an 8-pin connector and handles 55A at 10000 cycles without a hitch (even hot plug at 30A for hundreds of cycles). That this connector only does 40 cycles before breaking is caused only by it being poorly made or a poor design (e.g. relying on interference fit of contacts rather than true spring loading), and has nothing at all to do with the size.
The current plug is an ugly and cheap connector, so I don’t see aesthetics as a valid argument.
Another thing is that the remaining 2 pins on an 8 pin connector are sense pins. They are connected to +12V and GND on the load side, and to a voltmeter on the supply side. This allows the supply to measure the voltage drop which indicates a bad connection or overloaded wire. The new 12 pin connector does not support this.
This is not true. Both 6 and 8 pin connectors have "sense" pins (sense0 and sense1 - one in the 6-pin section, one in the extra two pins), but they are not a voltage feedback line. They serve to signal that 0W (nothing), 75W (6-pin) or 150W (8-pin) has been connected.
This is directly equivalent to the two sense pins on the new connector - except that rather than signalling 0, 75 or 150W, it signals 150W, 300W, 450W and 600W. Cable present has been given its own pin, together with a new "power stable" pin - all in the small 4-pin sub-connector, rather than wasting power pins for sensing.
No capabilities have been lost. Bad contact issues aside, it's a big improvement in every aspect.
Still don't know why a smaller cable was needed. This card is the size of a game console, why does it need a slim cable? I would much rather buy some AIB partner 4090 that has four 8-pin connectors instead of one new connector.
The old mini-fit connector had evolved to be kind of sketchy - a six-pin connector with two pins you hold on the side to turn it into an 8-pin connector? Not pretty.
And the new connector also adds 'sideband pins' so the power supply can tell the card how much power is available. That should reduce problems with random shutdowns due to drawing too much power.
With that said, a connector that melts clearly isn't an improvement.
"a six-pin connector with two pins you hold on the side to turn it into an 8-pin connector"
No, most modern PSUs come with solid 8 pin connectors (as well as 6+2 pin connectors that you mention). If you watch the YouTube PC building channels, you'll see no one uses the latter for cards with 8 pin connector.
The 8-pin connector is also just an outdated design. It's unnecessarily bulky, and rated at a measly 150 W. The idea wasn't for you to put 4 of these (32 pins and wires) together to power a single device.
For reference, an Anderson PP45 connector pair (from 1966!) is smaller than the original 8-pin connector and would handle 660 W at 12V reliably at up to 10000 cycles (and can even handle hot-unplug).
Besides aesthetics and ease of integration in the chassis, air flow / cooling is a non-trivial issue, particularly with the wattages systems are running now. More or wider cables increase chassis air flow impedance which requires higher pressure fans to overcome which translates to a louder overall system.
The Nvidia Founders Edition cards have tiny PCBs, due to their relatively small coolers and the flow through design. There isn't space for 3 8-pin connectors. And yes, that was definitely a choice but they're sticking to it.
Interesting that the connector (12VHPWR) in question seems to be safe for only 40 (forty) cycles. That's really not a lot, as these things go.
Of course it's an internal connector and not something you're supposed to plug/unplug all that often, but I still think it sounds like a really low number.
They are absolutely right and I have no idea why they are getting downvoted. Last time this was posted someone linked industry specsheets for all common PC connectors, from 8-pin GPU power plugs, to the good old MOLEX plug, and they all were rated for about ~30 connections in their lifetime.
Edit: there you go, found it - normal bog standard 8-pin PCI-e connector, rated for 30 connections:
"Update: We have confirmed with NVIDIA that the 30-cycle spec for the 16-pin connector is the same as it has been for the past 20+ years. The same 30-cycle spec exists for the standard PCIe/ATX 8-pin connector (aka mini-fit Molex). The same connector is used by AMD and all other GPU vendors too so all of those cards also share a 30-cycle life. So in short, nothing has changed for the RTX 40 GPU series."
I've never heard of other connectors melting though. They might be rated for only so many cycles, but how many real cycles are required for them to melt like these adapters?
Outside of the motherboard and GPU connectors, no other connector in a PC draws these levels of insane amount of current. Basically that's DVD/BD drives and spinning rust HDDs.
"Molex" connectors (the 4-pin ones used for fans and DVD drives etc.) are notorious for melting, and molex-to-sata adapters especially. It wasn't unheard-of for PCIe connectors to melt either.
As Mastax said, I've had multiple molex connectors melt over the decades across a fleet of PCs. I've seen one SATA power connector melt. I've had numerous of the 4 pin aux power connectors to motherboards melt. But note this is across fleets of thousands of PCs.
In my early days I saw a Pentium process melt the socket off the board. I don't understand how it didn't trip the power before it got that far.
The bigger callout is the safety margins that were assigned to each plug. The new connector runs pretty much right at rated margin minus 84W (6pins * 12V * 9.5A = 684W) for losses at full spec. The old connector was rated for 324W (3pins * 12V * 9A) but only specced for 150W or approx 4.2A per pin. So there was a massive safety margin there. Also the larger pin spacing means that the connections were better able to dissipate any heat that did show up for the most part anyway. The much tighter pin spacing on the new connection pretty much ensures that any dongle like this will be extremely hard to cool because of the fact the wires must come together in the actual connector.
Long story short: IMO the new connector makes me extremely nervous, running anything that can cause fire within ~14% of rated is not a wise idea unless you're doing Aerospace and have no other options. Even then they generally have extremely high QA and assembly verification, something I'm not expecting from computer OEMs.
Keep in mind that 40 cycles is the minimum. 99.9% of the plugs will be fine after 40 cycles and probably will do 400 without issue. But you're outside what the manufacturer guarantees it'll last.
And by experience, 40 is absolutely plenty. Almost all plugs in a computer are likely to only experience 10 cycles at most (and half of them only two; during setup and during EOL).
Sounds like the issue can happen due to high bend in the cable or too many cycles. Given the size of these video cards, placement of the connector and the size of computer cases I don't see why they didn't opt for right angle connectors so there is no bend needed.
edit: Looks like the nvidia designed version has the connector angled front at 45 degrees while the non-nvidia versions in the article don't.
Right angle connections pointing in which direction? Some PSUs are on the bottom and some are on the top. It would be worse having to connect a right angle and then routing over or around the card to get to the PSU.
I've been planning to use that case for my next build for months now, but I'm not sure anymore. These hot new components make it look like liquid cooling is mandatory if you value your ears, and the Torrent isn't the best choice for that.
If it's angled to the front of the case it'd cover both cases. Interestingly enough the founder's edition (ie: nvidia designed cards) seems to have the connector angled front at 45 degrees. However the non-founders cards in the article have it sticking horizontally to the side like previous cards. Seems something nvidia should have communicated to it's partners.
Under the typical use case (ie. you install it and leave it there for a few years), 16 cycles is more than enough. Now, if you're an enthusiast running benchmarks or whatever it might cause issues, but isn't really "insane" under the typical use case. Also, I'm not sure where the "16 cycles" number is from, but other commenters linked to an article that says it's rated for 40 cycles.
30–40 cycles is still very low for it to start failing. For comparison TE's popular Deutsch connectors are tested to 100 cycles without electrical or mechanical failure
This 'fact' gets batted around so often and its incredibly misunderstood. That is saying that in the worst case, where everything goes wrong, the cable can last no less than X cycles, ever. Also, it's not 16.
Also, these new connectors have the exact same rating as the prior 8-pin connectors.
Nvidia has historically used the power cable orientation as a form of market segmentation. Their high end product that goes into server cases have the power connections on one end of the card and they are parallel with the card. Consumer GPUs have the power connectors sticking out on the top of the card, and even in 4U server cases that can cause the power connectors to rub up against the top of the case or be blocked entirely.
I hadn’t even noticed that. And you’re completely right. My A5000 has pin holes on the board for a top mount but it’s power connector is on the side facing towards the front panel of the case.
I wonder what would be your suggestion for someone who wants to build his own first ML machine (and actually first computer as well). I would like to start small and add (graphics) cards and then maybe upgrade them later one by one. Should I go with high-end cards? latest? Thanks for any good pointers as well.
Is there a reason you can't use Colab? That would be the first suggestion. A pro license costs about 600 a year.
Next question is what are you training?
The latest cards are almost certainly faster, but you're normally concerned by VRAM than absolute speed. The 3090 (or the older Titan series) is unique in consumer cards in that you get almost double the RAM of the next cards down. I've had no problem training models on an 1080ti 8GB card but I wouldn't want to go smaller than that. System RAM is cheap in comparison, you can easily spec a 128GB machine, M.2 SSDs are a must for fast data loading, etc. Most CPUs are quads now, but more cores and lots of RAM means you can run more parallel dataloaders.
Otherwise there are plenty of guides that do a cost analysis of which cards to buy. If money no object then the 3090/Titan series, but maybe you don't need that much RAM. I saw a sibling comment mentions issues with heat - the stock 3090 has terrible VRAM cooling. I water cooled mine and it works well, but it's not cheap at all.
Thanks for the reply. Also, I have the tendency to become very conscious when I use a paid service and kind of messes my mind. I think in the long run it will be cheaper to own a machine.
Actually I am more into RL than DL. But DRL use DL.
I don't need a machine at the moment, I am in the learning state at the moment. But eventually I will need something and passively want to learn on building my own machine. So trying to avoid costly mistakes.
It will be NVIDIA cards for sure. It does not have to be the latest, but I would like to be able to upgrade the cards without need to change the rest of the machine, at least not more than 10% of initial cost. I am very ignorant on this that's why asked how to build a machine starting with cheaper graphics cards but then be able to upgrade to latest and the best if I need to. I guess motherboard is the most important component but which one? Also, size of the box. Any article that discuss these in depth? Not videos, I prefer reading.
Money is not an issue unless I don't make the mistake of buying wrong components that does not fit both physically and compatibility.
I would like to build a compact machine yet able to upgrade, for example 2 graphics card eventually.
The biggest issue you're going to run into is available RAM. Card speed is less important, you can always train a bit slower, but you can't get around memory limits. A recent *80 series would be fine to start, e.g. the 2080/3080 have 10-11GB. You have to go to the higher end cards like the 3090 to get 24GB. Generally working on a single card is easier than distributed, so one 3090 is better than 2x3080 IMO. Fortunately the mining crash has made stock a bit more available and prices have stabilised a bit.
If you want room to upgrade to 3 GPUs then it's going to be a reasonably sized case. These are 2-slot cards at minimum and you're going to need to either buy a big PSU up front (1kW at least), or upgrade later. Any high end motherboard with enough PCIe slots should be OK though.
I don't have a particular guide in mind, at this point I would say just post on Reddit /r/buildapc or /r/machinelearning and say you want to build a 3-GPU machine, or see what others have built on there. You can use something like PC Part Picker https://pcpartpicker.com/list/ to filter by compatible components.
GPU pricing is kind of all over the place at the moment, but, assuming you are on a budget, I would look for either a RTX 2070 (super if you can) or GTX 1080 Ti as great 'starter' cards for ML. Both should be available second hand for a fairly reasonable price. If you want a 'new' card, I would go for the 3070 or 3060. The 3060 is interesting since while it has few and slower cores than the 3070, it has more VRAM which is important for certain workload.
As for the rest of the components, don't sweat it too much. As to which is actually more important, it depends a lot on exactly what you are doing. 32 GB of RAM, basically any 'mid range' CPU from the past few years and an SSD an you're good to go.
One thing worth considering however is getting a latest gen CPU/motherboard so that you can upgrade the CPU in the future. For example, while you will probably find great deals on AMD AM4 CPUs and Motherboards today, you won't be able to upgrade to the latest AMD CPU in a couple of years since AM4 is end of life. Getting an AM5 CPU/motherboard will cost more up front but you will have the option to upgrade the CPU down the line.
Also as a final point, ask yourself do you really need an ML machine? If you're just 'playing around' consider sites like Colab and others that offer free or cheap GPUs for training ML models. I know lots of people who have done real ML research and publish actual ML papers using only the free tier Colab, so you can get quite far on it.
You have to pay more for the CPU, mobo and DDR5 ram, so it's quite a tax right now. It depends on your workload of course, but if it's mostly GPU-bound (ML, gaming) I think the AM4 5800X3D is a really good choice that is destroying benchmarks and should last 4-5 years.
For GPU it depends on your vram needs, the Stable Diffusion stuff seems fine with 8, but Textual Inversion (adding yourself to the Stable Diffusion model, etc.) and text/transformers work needs more. I wanted the latter so I bought a used 3090 which was relatively cheap (compared to before, anyway!)
True, I guess it really depends on your workload. If you're not CPU bound and don't really expect to be, then getting the best CPU/mobo combo you can afford today and not worrying about upgrading might be the best way to go.
With Intel's latest release, I'm not sure AM5 is worth investing in at this point. Raptor Lake seems to crush AM5 with support from existing sockets and DDR4 all while being cheaper. That massively cooled my desire to go all in on a big AM5 system.
I had to throw away many perfectly good power supplies when upgrading the graphics cards because they had not enough pins. So i would count out economic reasons.
Many 3d Printers started to convert to 24v recently because you have thin cables and overburdened plugs for high powered heaters and stepper motors. That caused no issue either it is amazing what modern voltage regulators can do.
The reasons were for better and more efficient power supplies, not having multiple voltage outputs with multiple current ratings, etc.
I'd be happy with a separate tier of PSU that has the standard 12v rails, but also includes a rail with 36 or 48 volts for dedicated GPUs. The less amperage running down those slim wires, the better...
I wonder if they could develop GPUs meant to run in series directly from higher voltage? Shunt regs could prevent imbalanced current long enough for software to rebalance the cores?
While we're at it, can we get a new motherboard standard that acknowledges the fact that video cards are at least as complex as the motherboard itself? With the strong preference for a ring configuration for central cooling, like the trash can Mac Pros. Or a star one.
If I recall correctly the only reason that configuration was abandoned was because the physical distance between the CPU and RAM needed to be as short as possible. I liked the Pentium 2 card as much as the next guy but I like faster system speeds more.
Would it be feasible to put one of these in its own case and use it as an external peripheral? I admit to losing track of what's the current standard for connector speeds, which seems to be changing every week.
106 comments
[ 3.7 ms ] story [ 190 ms ] threadSo happy it didn't burn up now!
A new connector was absolutely necessary, even for more reasonable GPU power levels. That the current connector is broken is a different problem, and could likely be fixed without manufacturing changes without a new overall connector design.
There's a problem with running 600 Watts through that connector, if this fire thing is regular issue it's going to burn someone's house down or kill someone
The issue bending the cable near the connector is that it causes the connector pins to deflect, making the flimsy contact worse.
More conductor is unnecessary, it just needs to make a more reliable contact.
A 12VHPWR connector is using 12 pins for GND/12V.
An 8pin PCIe connector has 6 pins for GND/12V, with 4 connectors that's 24 pins total.
Assuming that the pins are fairly similarly specced, the old connector should be safer, especially when exceeding the rated power. Regardless of how you frame it, it seems like aesthetics got prioritized over safety.
I think that's the problem right there: running the connection at its rated limit. It implies that a damaged or slightly out-of-spec connector could fail, as we see happened here.
Defensive engineering involves safety margins for safety-relevant specifications. Running a digital signal at the limits of its specification is one thing; data corruption is a detectable problem and not inherently unsafe. Running power connections at rated limits is something else entirely because the failure modes are physical damage, fire, and electrical arcing.
600W of power delivery is no joke, and I'd like to see that power carried over cabling and connectors with absolute maximum ratings much greater than that, say 1kW. Per the specs, it seems that each of the 12 current-carrying conductors in the connector runs about 9A. A dodgy connection that results in just 1-ohm of resistance would then represent 80W lost as heat at that point.
Frankly, I think that running 600W over a single 12V multi-core cable is insane. You're talking 50A of current, or about the same current rating as the largest typical home circuits for an electric kitchen appliance or car charger. In the home, that calls for thick structural cabling, but it appears (https://www.cable-sleeving.com/pcie5-12vhpwr-16pin-connector) that the in-case recommendations are to use 16AWG wire or even a touch thinner.
So by the spec it must have at 10% headroom - and to live up to that, manufacturers must add their own margin on top. Rating for 1kW would be a 66% margin and a humongous waste of copper, and it wouldn't solve this contact issue in the first place.
Regarding wire gauge - unrelated to the problem at hand - 16AWG is already overkill. For house installation in EU, we rate that gauge for 13A to 18.5A depending on thermal conditons, with 13A being "very poor" thermal conditions (no cooling and surrounded by other hot things). With 6 pairs in parallel, that gives a conservative minimum of 78A capacity - and with the excellent thermal conditions of a case with fans ("very good" is normally just a dangling cable, not one with fans cooling it), the realistic capacity should exceed 120A. As you say, we need only 50A, so that's a 140% margin! Even 20AWG would be plenty here.
Pulling 50A through a connector of that size or smaller is in itself not an issue. As mentioned elsewhere, powerpole is smaller than an 8-pin connector and handles 55A at 10000 cycles without a hitch (even hot plug at 30A for hundreds of cycles). That this connector only does 40 cycles before breaking is caused only by it being poorly made or a poor design (e.g. relying on interference fit of contacts rather than true spring loading), and has nothing at all to do with the size.
The current plug is an ugly and cheap connector, so I don’t see aesthetics as a valid argument.
This is directly equivalent to the two sense pins on the new connector - except that rather than signalling 0, 75 or 150W, it signals 150W, 300W, 450W and 600W. Cable present has been given its own pin, together with a new "power stable" pin - all in the small 4-pin sub-connector, rather than wasting power pins for sensing.
No capabilities have been lost. Bad contact issues aside, it's a big improvement in every aspect.
And the new connector also adds 'sideband pins' so the power supply can tell the card how much power is available. That should reduce problems with random shutdowns due to drawing too much power.
With that said, a connector that melts clearly isn't an improvement.
No, most modern PSUs come with solid 8 pin connectors (as well as 6+2 pin connectors that you mention). If you watch the YouTube PC building channels, you'll see no one uses the latter for cards with 8 pin connector.
For reference, an Anderson PP45 connector pair (from 1966!) is smaller than the original 8-pin connector and would handle 660 W at 12V reliably at up to 10000 cycles (and can even handle hot-unplug).
In those kinds of cases, cable management and space/size tolerances can be very difficult - swapping four connectors for one makes a big difference
Besides aesthetics and ease of integration in the chassis, air flow / cooling is a non-trivial issue, particularly with the wattages systems are running now. More or wider cables increase chassis air flow impedance which requires higher pressure fans to overcome which translates to a louder overall system.
Of course it's an internal connector and not something you're supposed to plug/unplug all that often, but I still think it sounds like a really low number.
Edit: there you go, found it - normal bog standard 8-pin PCI-e connector, rated for 30 connections:
https://www.molex.com/webdocs/datasheets/pdf/en-us/045586000...
Edit 2: found the quote from another article too:
WCCFTECH:
"Update: We have confirmed with NVIDIA that the 30-cycle spec for the 16-pin connector is the same as it has been for the past 20+ years. The same 30-cycle spec exists for the standard PCIe/ATX 8-pin connector (aka mini-fit Molex). The same connector is used by AMD and all other GPU vendors too so all of those cards also share a 30-cycle life. So in short, nothing has changed for the RTX 40 GPU series."
In my early days I saw a Pentium process melt the socket off the board. I don't understand how it didn't trip the power before it got that far.
Long story short: IMO the new connector makes me extremely nervous, running anything that can cause fire within ~14% of rated is not a wise idea unless you're doing Aerospace and have no other options. Even then they generally have extremely high QA and assembly verification, something I'm not expecting from computer OEMs.
And by experience, 40 is absolutely plenty. Almost all plugs in a computer are likely to only experience 10 cycles at most (and half of them only two; during setup and during EOL).
https://en.overclocking.com/12vhpwr-adaptator-is-a-consumabl...
edit: Looks like the nvidia designed version has the connector angled front at 45 degrees while the non-nvidia versions in the article don't.
[1] https://en.overclocking.com/12vhpwr-adaptator-is-a-consumabl...
Also, these new connectors have the exact same rating as the prior 8-pin connectors.
It looks like the 12VHPWR cables don't bend very much close to the connection itself.
There has been many well known cases of diluting alloy ratios to save cost until a problem is found.
Unfortunately, it can be quite expensive to check atomistic ratios of metal parts in mass production.
(sorry for the off topic question)
Get a 3080ti or 3060 with 12gb of ram. If you want to experiment with very high memory models then pay for colab pro.
Next question is what are you training?
The latest cards are almost certainly faster, but you're normally concerned by VRAM than absolute speed. The 3090 (or the older Titan series) is unique in consumer cards in that you get almost double the RAM of the next cards down. I've had no problem training models on an 1080ti 8GB card but I wouldn't want to go smaller than that. System RAM is cheap in comparison, you can easily spec a 128GB machine, M.2 SSDs are a must for fast data loading, etc. Most CPUs are quads now, but more cores and lots of RAM means you can run more parallel dataloaders.
Otherwise there are plenty of guides that do a cost analysis of which cards to buy. If money no object then the 3090/Titan series, but maybe you don't need that much RAM. I saw a sibling comment mentions issues with heat - the stock 3090 has terrible VRAM cooling. I water cooled mine and it works well, but it's not cheap at all.
Actually I am more into RL than DL. But DRL use DL.
I don't need a machine at the moment, I am in the learning state at the moment. But eventually I will need something and passively want to learn on building my own machine. So trying to avoid costly mistakes.
It will be NVIDIA cards for sure. It does not have to be the latest, but I would like to be able to upgrade the cards without need to change the rest of the machine, at least not more than 10% of initial cost. I am very ignorant on this that's why asked how to build a machine starting with cheaper graphics cards but then be able to upgrade to latest and the best if I need to. I guess motherboard is the most important component but which one? Also, size of the box. Any article that discuss these in depth? Not videos, I prefer reading.
Money is not an issue unless I don't make the mistake of buying wrong components that does not fit both physically and compatibility.
I would like to build a compact machine yet able to upgrade, for example 2 graphics card eventually.
The biggest issue you're going to run into is available RAM. Card speed is less important, you can always train a bit slower, but you can't get around memory limits. A recent *80 series would be fine to start, e.g. the 2080/3080 have 10-11GB. You have to go to the higher end cards like the 3090 to get 24GB. Generally working on a single card is easier than distributed, so one 3090 is better than 2x3080 IMO. Fortunately the mining crash has made stock a bit more available and prices have stabilised a bit.
If you want room to upgrade to 3 GPUs then it's going to be a reasonably sized case. These are 2-slot cards at minimum and you're going to need to either buy a big PSU up front (1kW at least), or upgrade later. Any high end motherboard with enough PCIe slots should be OK though.
I don't have a particular guide in mind, at this point I would say just post on Reddit /r/buildapc or /r/machinelearning and say you want to build a 3-GPU machine, or see what others have built on there. You can use something like PC Part Picker https://pcpartpicker.com/list/ to filter by compatible components.
As for the rest of the components, don't sweat it too much. As to which is actually more important, it depends a lot on exactly what you are doing. 32 GB of RAM, basically any 'mid range' CPU from the past few years and an SSD an you're good to go.
One thing worth considering however is getting a latest gen CPU/motherboard so that you can upgrade the CPU in the future. For example, while you will probably find great deals on AMD AM4 CPUs and Motherboards today, you won't be able to upgrade to the latest AMD CPU in a couple of years since AM4 is end of life. Getting an AM5 CPU/motherboard will cost more up front but you will have the option to upgrade the CPU down the line.
Also as a final point, ask yourself do you really need an ML machine? If you're just 'playing around' consider sites like Colab and others that offer free or cheap GPUs for training ML models. I know lots of people who have done real ML research and publish actual ML papers using only the free tier Colab, so you can get quite far on it.
For GPU it depends on your vram needs, the Stable Diffusion stuff seems fine with 8, but Textual Inversion (adding yourself to the Stable Diffusion model, etc.) and text/transformers work needs more. I wanted the latter so I bought a used 3090 which was relatively cheap (compared to before, anyway!)
True, I guess it really depends on your workload. If you're not CPU bound and don't really expect to be, then getting the best CPU/mobo combo you can afford today and not worrying about upgrading might be the best way to go.
Since those cards already have dedicated power connectors why not have 24 or even 48 volt lines?
There really should be no reason.
I had to throw away many perfectly good power supplies when upgrading the graphics cards because they had not enough pins. So i would count out economic reasons.
Many 3d Printers started to convert to 24v recently because you have thin cables and overburdened plugs for high powered heaters and stepper motors. That caused no issue either it is amazing what modern voltage regulators can do.
The reasons were for better and more efficient power supplies, not having multiple voltage outputs with multiple current ratings, etc.
I'd be happy with a separate tier of PSU that has the standard 12v rails, but also includes a rail with 36 or 48 volts for dedicated GPUs. The less amperage running down those slim wires, the better...
You mean "amp down with up the voltage"?
But this is my reaction too. Running 50 Amps even these short distances is just silly.
I would think about maybe saying that the card is 48V and here are the dongles that let you put four of the regular power supplies in series.
https://www.intel.com/content/www/us/en/products/docs/boards...
But I guess the CPU is still the hub of PCI-E networks right now.
https://imgur.com/a/VqTU1QR
Why would you still put the power connector on the side where the card will definitely get very close to the side of the case?