Octane/Cuda rig- preferred HackT architecture for multiple gpu. 3 or 4 Titan Z's on one

[ClassicMartini]

Greetings from Sydney Australia., Hope I'm posting in the correct area.

Is there a preferred architecture for multiple gpu's?

We're looking to build an "Octane" rig. Have budget for four Titan Z's. And, would like to use o'clocked 8 core cpu, which leads to question - can Hackintosh "take" 2,3,4 high end cards?

Octane "sees" all the Cuda gpu's it has, and USES them. This is not for a gaming rig, or even to drive multiple monitors.

Ideally, we'd like to O'Clock an 5960X on either an X99 E-WS or a Rampage V Extreme (we have the Rampage on hand).

We need BOTH fast single core speed, 4.4GHz, AND fast multi core speed x 8 = 35.2 GHz, AND a few GPU's to hand, in the one box - avoiding render node license - , our 3D operator is adamant on staying on OS X.

Halving the pcie lanes with more cards is not an issue. (query is about max GPU's on one Hackintosh board, not the pcie performance, nor SLI, nor Cuda vs Open GL, or Thea vs Maxwell, VRay

We'd rather not run out and buy all the "top end" stuff, only to discover its not well suited to the task.

Hence asking up front.

Thanks in advance.

N.

 

[cuiper]

This does not make any sense! You want to spend $$$ on hardware for a rendering rig, where every minute downtime cost money/penalties and all this on a "hackintosh" ....
Octane runs on windows and linux, both OS run native on the HW and will most likley have better render times.

 

[slim.jim]

Yea, Windows will likely have MUCH better render times.

 

[shilohh]

Greetings from Sydney Australia., Hope I'm posting in the correct area.

Is there a preferred architecture for multiple gpu's?

We're looking to build an "Octane" rig. Have budget for four Titan Z's. And, would like to use o'clocked 8 core cpu, which leads to question - can Hackintosh "take" 2,3,4 high end cards?

Octane "sees" all the Cuda gpu's it has, and USES them. This is not for a gaming rig, or even to drive multiple monitors.

To answer your question, it should be possible to run 4 GPUs. I have not actually run 4 myself but I had 1 Titan X and 2 GTX 980s (3) running great on my X79 machine. I'm currently running the Titan X and 1 980. My multiple maxwell setup runs great with the web driver and minimal injection via SSDT. Multiple Kepler might be tricky but I can't say for sure. Fermi cards required injection of every property. I think you'd be asking for headaches in OS X using the Titan Z with it's dual chip configuration. Look around and see if anyone has success with the Titan Z and if it actually performs in OS X. Also, it's a 3x width card so the max you'll fit on either motherboard is 3. You could fit 4 Titan Xs but then you'd have no available PCI-E slots for anything else. I don't see why you would consider the Titan Z for this configuration.

4 x Titan Xs:
8x PCI slots covered
1000 watts
530.27 OctaneBench Results
$4,000 = $1,000 msrp x4 limited availability

3 x Titan Zs:
9x PCI slots covered
1125 watts
463.24 OctaneBench Results
$9,000 = $3,000 msrp x 3 discontinued

Ideally, we'd like to O'Clock an 5960X on either an X99 E-WS or a Rampage V Extreme (we have the Rampage on hand).

We need BOTH fast single core speed, 4.4GHz, AND fast multi core speed x 8 = 35.2 GHz, AND a few GPU's to hand, in the one box - avoiding render node license - , our 3D operator is adamant on staying on OS X.

Halving the pcie lanes with more cards is not an issue. (query is about max GPU's on one Hackintosh board, not the pcie performance, nor SLI, nor Cuda vs Open GL, or Thea vs Maxwell, VRay

We'd rather not run out and buy all the "top end" stuff, only to discover its not well suited to the task.

Hence asking up front.

Thanks in advance.

N.

I would go with the R5E over the X99EWS as you might run into headaches from the PLX chips for the extra PCI-E lanes but I have no experience to say for sure.

You won't be using the 5960X with OS X unless you want to run Mavericks or Yosemite with rolled back Mavericks kexts. You'll probably want to go with a 5930K.

 

[ClassicMartini]

Thank you for your excellent response(s).

This is not for me. Our operator is ADAMANT about staying with MacOS

I understand that windows would give better performance, hence hackintosh - we can "flip the switch" and go into windows pretty quickly - without bootcamp limitations.

We're getting our Z's 2nd hand - with good 'provenance'. (Ex Octane users, systems vetted etc). AUD about $1600 is the going rate (840 Gbp, or 1075 Euro, as of 16/3/2016). Just purchased our first Z for eGPU project - that's a whole other story

A few more notes, as I've learnt more about his workflow, software packages etc and some ancillary info on Cuda cards and pcie support.

Our 3D operator will be working at home on his 2012 rMBP, and at the office.

The idea is to have a portable eGPU for home - Z in a portable rig based on Akitio TBolt backplane - and more grunt in the main office workstation - so this changes the FUNDAMENTALS a bit...

• ArchiVis stills, 1080p. (Yeah Titans/980ti etc are overkill).
• 4.4GHz single core is more than adequate for the CPU tasks.
• Main 3D software is SketchUp, planning to migrate to C4D 'sometime'.
  • Fastest - STABLE - single core performance is preferable
  • Thus we could drop the 6 core for better supported 4 core/chipset, if advisable??
  • We will watercool the CPU, as STABLE 24/7 O'clocking is what this build is about
    • We have a professional PC builder on hand for this.
• 10.10.5 is as far as we'd like to go, for at least 2 years. El Cap is of no value at all.
• eGPU will need STABLE Thunderbolt support
• We're running 2 x 34 LG screens (the thunderbolt ones) 980ti can drive those.
• I understand 2 x Z's is problematic, as the web driver doesn't like two on same motherboard? If not we would waterblock the Z's for packaging reasons, warranty is gone anyway.
  • So, a "Z" onboard with a 980ti
    • Other "Z" as the eGPU floater
  • Or Z as eGPU and 2 x 980ti's on mobo?
  • Either way we get a total of about 425 on Octane bench with all three combined, and operator has a BIG jump in home rMBP performance - eGPU/Tbolt - over the built in 650m
• SLI is of NO VALUE for Octane.
• Titan X really no better than 980ti in this workflow. More ram, splitting hairs on Octane optimisation.

So. What's the MOST stable, trouble free CPU/mobo for the above scenario?

Willing to trade cores for stability. Still want strong Thunderbolt support.

Still same "architecture" you recommended?

p.s 3 x Titan Z's = closer to 525 on Octane bench.
4 x X = closer to 500.


Thanks again, amazing site. Please tell me if I'm overstepping the mark, or should post elsewhere.

 

[ClassicMartini]

To answer your question, it should be possible to run 4 GPUs. I have not actually run 4 myself

but I had 1 Titan X and 2 GTX 980s (3) running great on my X79 machine.

I'm currently running the Titan X and 1 980. My multiple maxwell setup runs great with the web driver and minimal injection via SSDT

Look around and see if anyone has success with the Titan Z and if it actually performs in OS X. Also, it's a 3x width card so the max you'll fit on either motherboard is 3. You could fit 4 Titan Xs but then you'd have no available PCI-E slots for anything else. I don't see why you would consider the Titan Z for this configuration.

I would go with the R5E over the X99EWS as you might run into headaches from the PLX chips for the extra PCI-E lanes but I have no experience to say for sure.

You won't be using the 5960X with OS X unless you want to run Mavericks or Yosemite with rolled back Mavericks kexts. You'll probably want to go with a 5930K.

Thank you. You've pointed me in the right direction, am feeling more confidant on how to navigate (this site) and prepare. Your guides are succinct, easy to follow and have a humorous slant.

There are probably many HT learning pathways, but in a nutshell its kinda: "What processor would be best/most supported/least hassle, which requires getting to grips with "Ivy Bridge vs Haswell vs "E" variants etc.

ie x99 is a little on the extreme until apple update the nMP to 7.1/E5 v4

Followed by 'exotic' requirements - in my case its the ability to add Thunderbolt eGPU on the fly with a Titan Z in the mix, which means having 40 pcie lanes... so... 4930k is looking very attractive (today) - like on your Golden Build.

Otherwise outright O'Clock it would be 4790k, but... not enough pcie lanes to handle 3 'big' gpus...

How long does it usually take After Aple release new hardware for the extraction/disassembly to filter down to x86? Speed stepping/sleep etc

 

[shilohh]

Our operator is ADAMANT about staying with MacOS


I understand that windows would give better performance, hence hackintosh - we can "flip the switch" and go into windows pretty quickly - without bootcamp limitations.

I'm curious to see your benchmarks between Windows and Mac.

We're getting our Z's 2nd hand - with good 'provenance'. (Ex Octane users, systems vetted etc). AUD about $1600 is the going rate (840 Gbp, or 1075 Euro, as of 16/3/2016). Just purchased our first Z for eGPU project - that's a whole other story

I would take that first Titan Z and do some testing in OS X before buying more of them. You might find that performance in OS X is worse that you hoped for or you might be presently surprised but I doubt the surprise will be pleasant.

A few more notes, as I've learnt more about his workflow, software packages etc and some ancillary info on Cuda cards and pcie support.
...

So. What's the MOST stable, trouble free CPU/mobo for the above scenario?


Willing to trade cores for stability. Still want strong Thunderbolt support.


Thanks again, amazing site. Please tell me if I'm overstepping the mark, or should post elsewhere.

I have no experience owning or playing with X99 or thunderbolt hardware so I can't really comment on that. I've been using X79 since day 1. I may do a X99 build when the next gen of CPUs come out.


Based on what I've seen in forums here and elsewhere, I can say that users of the 5960X have to remain on 10.9 & 10.10 so 5930k may be a better choice. You'll probably be able to hit higher overclocks with the 5930K as well.


I don't think your overstepping. I solute you for laying out the cash to do a build like this and will be watching intently if you post your experience, findings, benchmarks etc etc etc.

x99 is a little on the extreme until apple update the nMP to 7.1/E5 v4


Followed by 'exotic' requirements - in my case its the ability to add Thunderbolt eGPU on the fly with a Titan Z in the mix, which means having 40 pcie lanes... so... 4930k is looking very attractive (today) - like on your Golden Build.


Otherwise outright O'Clock it would be 4790k, but... not enough pcie lanes to handle 3 'big' gpus...


How long does it usually take After Aple release new hardware for the extraction/disassembly to filter down to x86? Speed stepping/sleep etc

I don't think there is any way to use thunderbolt in X79 so you may need to go X99. You should search to see what hoops you'll be jumping through for thunderbolt. Also look into thunderbolt bandwidth because it may not be worth using that gpu through thunderbolt if you have a throughput bottleneck. I would go X99 if I were doing my first build today. The nice thing is you should be able to use a 5930K until new CPUs are released. Then you should be able to upgrade as they should use the same socket.

 

[ClassicMartini]

I'm curious to see your benchmarks between Windows and Mac.

I would take that first Titan Z and do some testing in OS X before buying more of them. You might find that performance in OS X is worse that you hoped for or you might be presently surprised but I doubt the surprise will be pleasant.

I have no experience owning or playing with X99 or thunderbolt hardware so I can't really comment on that. I've been using X79 since day 1. I may do a X99 build when the next gen of CPUs come out.

Based on what I've seen in forums here and elsewhere, I can say that users of the 5960X have to remain on 10.9 & 10.10 so 5930k may be a better choice. You'll probably be able to hit higher overclocks with the 5930K as well.

I don't think your overstepping. I solute you for laying out the cash to do a build like this and will be watching intently if you post your experience, findings, benchmarks etc etc etc.

I don't think there is any way to use thunderbolt in X79 so you may need to go X99. You should search to see what hoops you'll be jumping through for thunderbolt. Also look into thunderbolt bandwidth because it may not be worth using that gpu through thunderbolt if you have a throughput bottleneck. I would go X99 if I were doing my first build today. The nice thing is you should be able to use a 5930K until new CPUs are released. Then you should be able to upgrade as they should use the same socket.

Shiloh.

In the spirit of "paying it forward" here's my current working hypothesis (gleaned from many knowledgeable sources, especially Andrew RPD) on max number of GPU's on Mac.

You can have 4 cores. But no more than 2 cores on each 'platform'. ie:

• 2 Keplar and 2 Maxwell
• 2 Keplar and 2 AMD
• 2 Maxwell and 2 AMD

There are some exceptions, if you go back far enough with OS/older Cuda cards on Cubix etc.

Most 15 MacBook pro's with Thunderbolt 2 have an nVidia card, thus you could have, say:

• 15" MacBookPro (1 small Kepler core - probably)
• 1 x Titan X (1 maxwell core)

But you couldn't put a Titan Z (2 Keplars) on there instead of the Titan X, as you'd have 3 (Kepler) Cuda cores, and go over your "2 of the same kind" budget.

The nMP has 2 x AMD cores, so - again, you could put a Titan Z on there too, or 2 x Titan X's, and that would be it, no Radeon's

If you took a 13" rMBP - with no discrete card, you could put a Titan Z AND a 295x2 on there.

So, where does that leave us Hackintosher's...

Well, kinda better off. There's no built in video on Ivy Bridge E (x79) 4950k/4960X or Haswell E (X99) 5930k

IN THEORY...

You could put:

• 1 x Titan Z (2 Keplars) OB 175
• 2 x Titan X (2 maxwells) OB 250

Octane Bench: 425!

Which is PLENTY for our use case - ArchVis stills at 1080p.

On, preferably, X79 (stability, proven) - MAYBE X99 if you're a thrill seeker.

So there you have it Shiloh, a working model (formula) for Max GPU's on Mac.

Anyone know better, please let me know.

All mistakes are MINE, all credit/glory/knowledge was gathered from other much smarter people.

 

[shilohh]

Well here's a big hole to poke in your hypothesis. As I mentioned earlier I had 3 maxwell cards installed and running perfectly. I can't speak to their efficiency in octane render but they deffinetly work. I still don't see any value in the Titan Z over the Titan X

 

[ClassicMartini]

Well here's a big hole to poke in your hypothesis. As I mentioned earlier I had 3 maxwell cards installed and running perfectly. I can't speak to their efficiency in octane render but they deffinetly work. I still don't see any value in the Titan Z over the Titan X

Yes, there is a little hole, and it makes it look like I wasn't paying attention to your posts. Oops. Anyway, in the quest to find the exceptions (or to find the 'true' pattern)...

Do you remember which OS you were running with the three Maxwell's?

Why use the Z, that MASSIVE card, that requires MASSIVE juice...?

Z scores 175 on OB
X scores 125 on OB


Titan Z is attractive with water blocking, it becomes a single slot card. There's some pretty crazy stuff over on the Otoy forums (not sure of rules for posting to other sites, and I don't want this post deleted)
Three of Z (waterblocked, for packaging and thermal efficiency) for OB will crush the X.
Don't get me wrong the X is AWESOME