Deep Thought: Water Cooled EVGA SR-2 Dual Xeon 5690 GTX 580

Deep Thought: Water Cooled EVGA SR-2 Dual Xeon 5690 ML Update

Braindeadmac's Build - Dual Xeon 5690 - EVGA Classified SR-2 - 24GB RAM - GTX 580 Ultra Hydro Copper


_MG_6614.jpg

Components

Apple OS X Lion USB Thumb Drive
Search Results - Apple Store (U.S.)

EVGA Classified SR-2 270-WS-W555-A2 LGA 1366 Intel 5520 SATA 6Gb/s USB 3.0 HPTX Intel Motherboard
http://www.newegg.com/Product/Product.aspx?Item=N82E16813188070&Tpk=classified%20sr-2

Intel Xeon X5690 Westmere-EP 3.46GHz 6 x 256KB L2 Cache 12MB L3 Cache LGA 1366 139W Six-Core Server Processor (two)
http://www.newegg.com/Product/Product.aspx?Item=N82E16819117253

CORSAIR DOMINATOR GT 12GB (3 x 4GB) 240-Pin DDR3 SDRAM DDR3 2000 (PC3 16000) Desktop Memory (24 GB Total)
http://www.newegg.com/Product/Product.aspx?Item=N82E16820145329

SeaSonic Platinum-1000 1000W ATX12V / EPS12V 80 PLUS PLATINUM Certified Full Modular Power Supply
http://www.newegg.com/Product/Product.aspx?Item=N82E16817151105

EVGA 03G-P3-1597-AR GeForce GTX 580 (Fermi) Classified Ultra Hydro Copper 3GB 384-bit GDDR5 PCI Express 2.0 x16 HDCP Ready SLI Support Video Card
http://www.newegg.com/Product/Product.aspx?Item=N82E16814130757

Sonnet GE1000LA-E Faster Wired Networking 10/ 100/ 1000Mbps PCI-Express 1 x RJ45
http://www.newegg.com/Product/Product.aspx?Item=N82E16833318003

ATTO ESAS-H608-000 ExpressSAS R608 8-Port Internal 6Gb/s SAS/SATA PCIe 2.0 RAID Adapter
http://www.amazon.com/ATTO-ESAS-H608-000-ExpressSAS-Internal-Adapter/dp/B004SNFZQ4/ref=sr_1_1?ie=UTF8&qid=1335318708&sr=8-1

Trayless Hot-Swap SATA Cage
http://www.newegg.com/Product/Product.aspx?Item=N82E16816215243

StarTech 3 Port 2b 1a PCI Express 1394 FireWire Adapter Card
http://www.newegg.com/Product/Product.aspx?Item=N82E16815158091

BELKIN F8T016 Mini Bluetooth Adapter
http://www.newegg.com/Product/Product.aspx?Item=N82E16833314046

(2) OCZ Vertex 3 VTX3-25SAT3-120G 2.5" 120GB SATA III MLC Internal Solid State Drive
http://www.newegg.com/Product/Product.aspx?Item=N82E16820227706

CaseLabs Magnum MH10 Case
MAGNUM MH10 Case - CaseLabs Store

Koolance CTR-CD12 Pump & Fan Controller With Display
CTR-CD12 Pump & Fan Controller With Display

EK 5.25" Mountable D5 Dual Top - Black Acetal - Single Reservoir w/ Dual Alphacool VPP655 Variable Speed Pump Installed
EK 5.25" Mountable D5 Dual Top - Black Acetal - Single Reservoir w/ Dual D5 Variant Pumps Installed - FrozenCPU.com

EK-FB SR2 - Acetal + Nickel Cooling Block for EVGA Classified SR-2
http://www.ekwaterblocks.com/shop/ek-fb-sr2-nickel-1.html

(2) EK Supreme HF Universal CPU Liquid Cooling Block w/ LGA 2011 Hardware - Rev 3 - Acetal + Electroless Nickel Plated (EK-Supreme HF - Acetal+EN (Nickel))
http://www.frozencpu.com/products/1...eme_HF_-_AcetalEN_Nickel.html?tl=c325s839b133

(2) Black Ice SR1 Low Air Flow Optimized - 480 Radiator
Black Ice SR1 Low Air Flow Optimized - 480 Radiator - Black - FrozenCPU.com

Eight Case Fans: Noiseblocker NB-BlackSilentPro PL-1 120mm x 25mm Ultra Quiet Fan - 900 RPM - 11 dBA
Noiseblocker NB-BlackSilentPro PL-1 120mm x 25mm Ultra Quiet Fan - 900 RPM - 11 dBA - FrozenCPU.com

Eight Radiator Fans: Scythe Gentle Typhoon 120mm x 25mm Fan - 1150 RPM
Scythe Gentle Typhoon 120mm x 25mm Fan - 1150 RPM (D1225C12B3AP-13) - FrozenCPU.com

Already Owned

Eizo FlexScan SX2762W 27" LCD Monitor, 2560 x 1440 Native Resolution, 850:1 Contrast Ratio, 270cd/m2 Brightness, 1.07 Billion Colors, Black
http://www.amazon.com/Eizo-FlexScan-SX2762W-Monitor-Resolution/dp/B0051OVKQS/ref=sr_1_1?ie=UTF8&qid=1335319830&sr=8-1

(2) Western Digital Caviar Black WD2002FAEX 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136792

(6) HITACHI Deskstar 7K3000 HDS723020BLA642 (0f12115) 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive
http://www.newegg.com/Product/Product.aspx?Item=N82E16822145473


Comments:

My goal in building this was to build a case which houses all of my storage needs (currently 8 2TB drives and 2 120GB SSDs, as well as a few hotswapped 2 and 3 TB drives), allows for an overclocked yet quiet build and doesn't look like it belongs on a street corner trying to sell bits and bytes. I need a fair bit of horsepower for some of my work, so the dual Xeons fit the bill.

The current Unibeast doesn't play nicely with this motherboard, so to do a Lion install, I wound up installing Lion onto one of my SSDs from my Mac Pro. If you are interested in underclocking/undervolting this machine, or aggressively overclocking it, don't install OS X 10.7.3, use 10.6.8 or 10.7.1-2. 10.7.3 breaks native power management C-state generation with a user generated DSDT (I'm working on fixing that); the C-states generated by Chameleon/Chimera severely hamper performance on dual processor machines.

For a DSDT.aml file, you can create your own using the edits described on the first post of 270-WS-W555 DSDT native power management modifications - insanely-mac Forum, or you can download the files from d00d at insanely-mac. The currently shipping BIOS (as of spring 2012) has problems with random CMOS resets, and is not recommended for either Windows or OS X. I downgraded my installation BIOS to version A49 prior to installing OS X (must be done from Windows). You can copy his SMBIOS.list if you wish, or use Chameleon Wizard to create your own. You want to use MacPro 5,1 to get best native power management.

Have d00d's or your own DSDT.aml available and place on desktop after machine boots up, I used a USB drive with all the stuff to install handy.

You can use Multibeast to install DSDT.aml, third party SATA support, JMicron SATA support, FakeSMC.kext and the generic TRIM enabler for SSDs.

You MUST apply the AppleRTC patch for CMOS reset; you can conveniently use Multibeast for this, or do it by hand (I've tested both, identical either way). I learned that multibeast could do it after the doing it the hardway. After running Multibeast, don't reboot.


Screen Shot 2012-04-17 at 2.50.03 PM

Apply edits to FakeSMC.kext/Contents/Info.plist to properly identify MacPro 5.1.

Be careful installing FakeSMC monitoring modules as some versions/modules break Turbo Boost. The EVGA Classified uses the Fintek chipset for what it's worth.


If using 3 channel memory, install Disabler.kext linked in the first post of 270-WS-W555 DSDT native power management modifications - insanely-mac Forum (or remove AppleTyMCEDriver.kext). There is a 3-5% performance hit by using dual channel memory in benchmarking; I doubt it makes a substantial difference in the real world. So if you want to use ECC memory, you can, it just has to be dual channel setup to actually utilize AppleTyMCEDriver. Like a real Mac Pro, the SR-2 automagically configures 3 channel and 2 channel memory.

You can now reboot.

For optimum performance with this motherboard, you want to turn off C and P state generation by the bootloader (Chimera and Chameleon are the same here). That's best done in the org.chameleon.Boot.plist file; here's mine for OS X 10.6.8 (for now with 10.7.3, just change the flags for DropSSDT, GenerateCStates, and GeneratePstates to "Yes", as this is broken by 10.7.3). Again, this is only appropriate if using the correctly patched DSDT.aml. You'll want to identify the machine as a MacPro 5,1 in smbios.plist.


You have native power management working if, and only if, you have
Apr 24 20:21:25 localhost kernel[0]: AppleIntelCPUPowerManagement: Turbo Ratios 111122
in your kernel.log and don't have messages like:
Apr 21 12:24:35 Deep-Thought kernel[0]: ACPI_SMC_PlatformPlugin:ushCPU_CSTData - _CST evaluation failed
Apr 21 12:24:35 Deep-Thought kernel[0]: ACPI_SMC_PlatformPlugin::registerLPCDriver - WARNING - LPC device initialization failed: C-state power management not initialized
It is possible to have Turbo Ratios working, but not C-state generation.

The Firewire card, network adapter card, and USB Bluetooth adapters all work out of the box, transparently.

To get EVGA GTX580 Ultimate Copper installed...life will be much easier if you use a recent Chameleon 2.1. (I'm using 2.1svn-r1923) or Chimera 1.9 or higher as your bootloader. Out of the box, I was sort of stunned to see that the card scored a whopping 10 fps on Cinebench, and crawled through the Uningine benchmarks. With these changes, I get an adequate 40.5 FPS on Cinebench.

Use Multibeast to install patches to GeForceGLDriver.bundle and NVDAGF100Hal.kext (as of Multibeast 4.3.1 that's the "10.7.2/10.7.3 Combo OpenCL enabler. However as of Multibeast 4.3.1, NVDAGF100Hal.kext needs to be edited by adding it's identifier (0x1080):


You then have to edit AppleGraphicsPowerManagement.kext:Contents:Info.plist to throttle up the gpu power state, otherwise it runs at low power mode most (all?) of the time. Just cut and paste this entry in the MacPro5,1 section


Caveat emptor--these may not be the optimal settings, I haven't played with them much. Also keep in mind I am using an overkill watercooling system, so I can keep this running full bore and only hit 35C on the graphics card....

I don't recommend the current Nvidia drivers if you are using Lion 10.7.3 (Retail_270.00.00f01), as Apple drivers are actually newer.

Edit July 2012....

Updated for Mountain Lion:
Only two kexts need to be installed, FakeSMC and Disabler.kext. Video card works OOB with 10-15% performance improvement.
Chimera 1.11.0 for bootloader
FakeSMC r609 + plugins

Disabler.kext to disable AppleTyMCDEDriver.kext (triple channel memory)
Patch AppleRTC.kext with the terminal command: sudo perl -pi -e 's|\x75\x30\x89\xd8|\xeb\x30\x89\xd8|' /System/Library/Extensions/AppleRTC.kext/Contents/MacOS/AppleRTC

Video cards with greater than 2GB RAM don't work in Apple's OpenCL, thanks to netkas.org for the fix, in your handy-dandy hex editor,
edit /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL
find ff 50 68 4c 39 e0 73 e7
replace with ff 50 68 4c 39 e0 90 90

Set
Graphics Enabler = yes

Computer P0rn



Is it worth it?

Screen Shot 2012-04-19 at 1.45.12 PM

dear santa...

I spent some time in my thesaurus to find words, but WOW!

Sexy beast

no words.

nice!

sick beast just a dream for me

Monstrous !!

Love the build. I'm currently working on my Sr-2 build so thanks for the helpful tips for install. Are you overclocking the CPU's?

slv335i said:

Love the build. I'm currently working on my Sr-2 build so thanks for the helpful tips for install. Are you overclocking the CPU's?

Click to expand...

Thanks, it's been a lot of fun.

Yes I do overclock. And do consider "underclocking". It's really quite impressive, just a different approach to overclocking. Right now my machine is sort of stuck on "10.7.3" for the primary boot disk for other software reasons, but I may downgrade if I can find the time. This version can't be underclocked terribly well due to breaking automatic C state generation with the user DSDT. So if you don't need 10.7.3, don't use it.

I currently overclock the 5690s, but leave C-state and P-state generation enabled. Can't "overclock" as high as usual overclocking, but still get very good performance with lower temperature and power usage. I'm currently at 4.2GHz 24/7, idle temps are 28-29C, and Geekbench is 34,000 on 10.7.3. Because Turbo Boost works, that means when Turbo kicks in I have 4 cores on each processor at 4.37 GHz and 2 at 4.54 GHz. Haven't tried to push it much past that. LOL the system current already spikes to 700W with Geekbench.