mncrnich Build -- 2 x Intel Xeon E5-2640 Sandy Bridge-EP 2.5GHz, 64 GB RAM, ~30TB Fiber Storage OS X

[mncrnich1]

mncrnich Build -- 2 x Intel Xeon E5-2640 Sandy Bridge-EP 2.5GHz, 64 GB RAM, X9DAi, ~30TB Fiber Storage OS X 10.8.3 Server

mncrnich Build: 2 x Intel Xeon E5-2640 Sandy Bridge-EP 2.5GHz, 64 GB RAM, X9DAi, ~30TB Fiber Storage OS X 10.8.3 Server

​

Component

Apple OS X Mountain Lion with Server @ Mac Apps Store
http://www.apple.com/osx/specs/

Supermicro X9DAi
http://www.supermicro.com/products/motherboard/xeon/c600/x9dai.cfm

Dual Intel Xeon E5-2640 Sandy Bridge-EP 2.5GHz 15MB L3 Cache
LGA 2011 95W Six-Core Server Processor BX80621E52640
http://ark.intel.com/products/64591/

8PCS 8GB DDR3 1600Mhz ECC Reg
(4pcs of 8GB = 32GB per processor / 64GB total)

Supermicro SC743TQ-R760B 4U/Tower 8x 3.5"" SAS / SATA Hotswap
Drive Bays
http://www.supermicro.com/products/chassis/4U/743/SC743TQ-1200-SQ.cfm

OS Drive, Dual 128GB Samsung 840 PRO Series Solid State Drives 7PD128BW @ RAID 1
http://www.amazon.com//dp/B009NB8WR0/

EVGA 512-P3-1310-LR GeForce 210 Graphics Card - PCI Express
2.0 x16 - 512 MB DDR3 SDRAM 2560 x 1600
http://www.amazon.com/dp/B004H4XE6W/

ATTO Celerity FC-162E Dual-Channel 16Gb/s Fibre Channel PCIe 3.0 Host Bus Adapter
http://www.attotech.com/products/product.php?cat=1&scat=33&prod=108&sku=CTFC-162E-000

ATTO FastFrame NS12 Dual Port 10GbE PCIe 2.0 Network Adapter
http://www.attotech.com/products/product.php?cat=14&scat=24&prod=79&sku=FFRM-NS12-000 FFRM-NS12-DA0

EMC VNX 5100 Storage Array
26 SAS Drives @ 2TB
~31TB Active ~52TB RAW
http://www.emc.com/storage/vnx/vnx-family.htm#product-line

APC Smart-UPS RT 3000VA Rack Tower 120V
http://www.apc.com/products/resourc....cfm?base_sku=SURTA3000RMXL3U&total_watts=200


Comments

I wanted to share my new build because I owe a huge gratitude to this community. There is no way this machine would live without all of your support. Please keep it up!

Please note this is a work server which had an appropriate budget. This is not my personal computer!!! This is far overkill for a singular person.


The purpose of this machine is to act as the NAS server for the production workstations. This server connects to the storage via 16Gb/s fiber. This server will take that data and interface with the LAN switch via 10GbE.

I chose the motherboard based upon the user forum here. http://legacy.tonymacx86.com/viewtopic.php?t=55963

It is a 4RU system. I chose this chassis based upon these needs.

• Redundant PSU
• Ability to fit full size PCIe cards
• Additional PCIe expansion slots for the future
• More than 2 drives bays which are hot-swappable
• Rack mountable

We did run into some issues with the install. We needed to run the Chimera boot loader on it's own in order to adjust several flags. I will elaborate further once I document the process.

We also ran into problems with the OS RAID set. For initial installation, we went back to a singular drive then cloned the OS drive over to the RAID set. Each drive within the RAID set requires files from within the Chimera Boot Loader. Specifically these files.

 

[Adrian B]

Well mncrnich1,

Thats quite a beast that you have created, the spec is impressive. Glad that people are putting things like this together. Hope you remembered to disable the auto update feature in OS X after doing all that work. Makes my little build rather pathetic in comparison, but it is quick enough for what I need it to do. I don't suppose you have any idea what sort of Geekbench scores it produces?

 

[mncrnich1]

It makes quite a difference when you have money to throw at a machine. No need to compare your machine. I am sure your personal build is faster than my MacBook Pro.

Hope you remembered to disable the auto update feature in OS X after doing all that work.

I did remember to disable the auto update. Thanks for the reminder!

I don't suppose you have any idea what sort of Geekbench scores it produces?

I have not run Geekbench yet, but because you asked, I will. Give me a day. I will post the results.


Again, big thanks to this community.

 

[mncrnich1]

I don't suppose you have any idea what sort of Geekbench scores it produces?

Geekbench Score: 20443 second one I ran achieved 20778

Benchmark Summary
Integer Score 19929 ||||||
Floating Point Score 34572 |||||||||||
Memory Score 4197 |
Stream Score 5287 |


Geekbench Score 20443 |||||||


System Information
Operating System Mac OS X 10.8.3 (Build 12D78)
Model MacPro3,1
Model ID MacPro3,1
Motherboard Apple Inc. Mac-F42C88C8 0123456789
Processor Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz @ 2.50 GHz
2 Processors, 12 Cores, 24 Threads
Processor ID GenuineIntel Family 6 Model 45 Stepping 7
L1 Instruction Cache 32.0 KB x 6
L1 Data Cache 32.0 KB x 6
L2 Cache 256 KB x 6
L3 Cache 15.0 MB
Memory 64.0 GB 1333 MHz DDR3
BIOS Apple Inc. MultiBeast.tonymacx86.com


Integer Performance
Blowfish
single-threaded scalar 1530
multi-threaded scalar 31544 ||||||||||
Text Compress
single-threaded scalar 2437
multi-threaded scalar 36774 ||||||||||||
Text Decompress
single-threaded scalar 2718
multi-threaded scalar 40917 ||||||||||||||
Image Compress
single-threaded scalar 2268
multi-threaded scalar 31711 |||||||||||
Image Decompress
single-threaded scalar 2657
multi-threaded scalar 28775 |||||||||
Lua
single-threaded scalar 4135 |
multi-threaded scalar 53693 ||||||||||||||||||


Floating Point Performance
Mandelbrot
single-threaded scalar 2116
multi-threaded scalar 43820 |||||||||||||||
Dot Product
single-threaded scalar 3447 |
multi-threaded scalar 53145 ||||||||||||||||||
single-threaded vector 4168 |
multi-threaded vector 74247 |||||||||||||||||||||||||
LU Decomposition
single-threaded scalar 1854
multi-threaded scalar 13923 ||||
Primality Test
single-threaded scalar 5848 ||
multi-threaded scalar 69993 ||||||||||||||||||||||||
Sharpen Image
single-threaded scalar 4934 |
multi-threaded scalar 85506 |||||||||||||||||||||||||||||
Blur Image
single-threaded scalar 5763 ||
multi-threaded scalar 115251 ||||||||||||||||||||||||||||||||||||||||


Memory Performance
Read Sequential
single-threaded scalar 3826 |
Write Sequential
single-threaded scalar 7001 ||
Stdlib Allocate
single-threaded scalar 3006 |
Stdlib Write
single-threaded scalar 3732 |
Stdlib Copy
single-threaded scalar 3422 |


Stream Performance
Stream Copy
single-threaded scalar 5655 |
single-threaded vector 5324 |
Stream Scale
single-threaded scalar 5568 |
single-threaded vector 5022 |
Stream Add
single-threaded scalar 5479 |
single-threaded vector 5393 |
Stream Triad
single-threaded scalar 5873 ||
single-threaded vector 3983 |

 

[nickking]

Hiya - good looking build...nice job!

Did you have any trouble with the ATTO FastFrame NS12 Dual Port 10GbE PCIe 2.0 Network Adapter? I'm currently trying to get the FastFrame NT11 working on a Hack I've just built...the correct kexts appeared in the library after I ran the ATTO installer, but the card isn't recognized in system profile nor is it functional.

Any insight into how you set it up greatly appreciated.

Cheers!

 

[mncrnich1]

Hiya - good looking build...nice job!

Did you have any trouble with the ATTO FastFrame NS12 Dual Port 10GbE PCIe 2.0 Network Adapter? I'm currently trying to get the FastFrame NT11 working on a Hack I've just built...the correct kexts appeared in the library after I ran the ATTO installer, but the card isn't recognized in system profile nor is it functional.

Any insight into how you set it up greatly appreciated.

Cheers!

I did not have any issues initializing and configuring the FastFrame NS12. Knock on wood, it has functioned perfectly.

I am running v4.10.1 of the ATTO config tool and Driver v2.1.2. I am running OS X 10.8.5.

I did have trouble with another ATTO FiberChannel Card. It was the 16Gb Dual FC card. I could not get it to work at all. One test ATTO suggested was taking a Fiber cable and running it from one port to another. If the card is operational, the LED lights will activate independent of any drivers. In the case of the 16Gb card, neither light activated. Turned out the card was bad. Give that a try.

Also, do you see the card initialize at startup within your boot commands? I see a couple ATTO lines after the BIOS loads. I cannot remember exactly, something about enabling the card at a lower level than the OS.

Good luck, let me know how it goes.

 

[nickking]

Thanks for replying so soon.

I've progressed a little further now but in a slightly unscientific way. I'll outline my steps in case it's of use to you or someone else in the future and I'll report back if things change.

1. Out of frustration I decided to download the 10Gbe driver that Small Tree have on their website. My (probably misguided) thinking was that the Small Tree and Atto cards may have the same underlying chipset (from Intel?).

This actually worked. The ATTO card started to show up in System Profiler. Go figure.

However, my 10Gb switch was reporting a link speed of only 1Gb.

2. I called ATTO (didn't tell them I was running a Hack...wanted to see how they'd deal with this under pure OSX). They told me there was a bug in OSX that prevented the ATTO card from being configured properly - in the hardware panel of the network preferences section of system prefs, you don't have an option beyond 1000baseT so it needs to be set to autoselect with full-duplex, flow-control and MTU manually set at 9000.

They sent me a new driver osx_drv_fastframe_222b3.dmg and some instructions on how to configure the card under OSX. I decided to dive right in...they suggested testing the settings in Terminal, but I just copied the .confi they provdided into etc\ on my system drive.

3. My switch is now reporting a 10Gb link. Which is a real relief. The ATTO card isn't cheap.

4. Need to do further testing of speeds. Currently I'm getting about 250 MB/s sustained reading from my RAID (which is connected via a separate genuine MacPro, also using an ATTO card), but the RAID hasn't yet finished initialising and speeds may well increase.

5. As you can see, I didn't test things at every stage of these steps, so it's impossible to say whether the new driver from ATTO was essential, or whether the tweaked settings alone would have got the job done. It's also hard to say whether the new driver would work without the need to install the Small Tree driver.

Happy to provide any extra info to others, if it'll help. This community got me this far and I'm grateful.

Specs for my baby video edit suites (building 5 of them to strap to the 10Gb network) FYI:

Gigabyte Z87X-UD3H
i5-4670 (using integrated gfx)
8GB Crucial Ballistix

 

[nikezzz]

According to Geekbench Browsing you should get higher results:

http://browser.primatelabs.com/geekbench3/search?dir=desc&q=2640&sort=multicore_score

Have you tried to use a SSDT?

Nice build!