Build Questions: A Rock Solid Thunderbolt DAS / Editing File Server in a G5

[spaceman33]

After peeking around and messing with some installs on hardware I had already, I am looking to dive in and build one from scratch.

I have a couple different considerations / possibilities, which I'd like to have do double duty as a video editing solution as well as personal backup.

The reason I began down the path of this build is that I've been currently using a 2012 maxed out mac mini as a file server/editing station with direct attached thunderbolt storage. However, as my storage needs increase, the cost of additional thunderbolt peripheral devices is getting a bit crazy.

I feel the current apple lineup is pushing these very small machines and saying "simple and small is better", and then just adding a pile of pricey hardware through the thunderbolt interface causing a gigantic anti-zen cluster****. Also 75% of the time you can't really ever even reach the theoretical max throughput with the disks they put in them...

However, the good about thunderbolt is that you have smart status indicators and overall its way more stable than running a raid set through USB. I thought about eSata as well but most enclosures cap at 3Gb/s and I can easily get more than 300mb/s off of 5x 7200rpm disks, so I don't want bottlenecks. Using SAS and so forth just adds a lot of different adapters and controllers to my existing desktop situation, so I didn't want to go there either. I also thought about a norco rack mount case but they are gigantic and hold 24 disks.

I would like a one power supply / two cable solution that is not rack mount to hold about 12 7200rpm 3.5 disks and about 4-5 ssds

Here's the "non-hackintosh" solution (no mobo)
Utilizing a PCI-e to thunderbolt device like a sonnet, adding a raid or hba card in it, putting it in a g5 enclosure and then adding all my disks to this enclosure by just basically jamming as many HD cages into the cases as I can. Has anyone done this?

However, here's the rub:
Also, I am using ZFS on OS X, which means more Ram = more better. The more my storage increases, the more the 16gb max in the 2012 mini isn't cutting it.

This is pointing to just biting the bullet, selling the mini and building a full on hackintosh.

Here are my needs for a hackintosh:

*Most relevant* I need to use this for work so I want the most stable system possible / closest to working out of the box. I am not delusional, I get there is risk, but I want the most supported options.

1. Thunderbolt 2 connectivity necessary (Gigabyte boards I saw only had t1?)

2. Xeons if possible but not deal breaker

3. greater than 32 gb ram necessary (Most boards I saw only went up to 32?)

4. Needs to hold at least 16 hard disks

5. Hot swap preferable

6. Visually appealing case solution (fractal r4, g5, lian-li v1000 etc.)

I figure the g5 won't contain this. Maybe I need one of those lian-li cases...


Is this possible? Does anyone have a similar situation to mine or has attempted a similar build that could offer some advice or insight?


Thank you in advance.

 

[Going Bald]

Why not just get a NAS like http://www.newegg.com/Product/Product.aspx?Item=N82E16822108197 and install OS X Server on it on it, then plug in your drives? This can be your file server and will reduce the load on the Mac Mini.
You can get a http://www.amazon.com/dp/B008L5TQIS/?tag=tonymacx86com-20 combo heat sink/2.5" to 3.5" adapter for the SSDs
Or put your MacMini in one of these: http://www.amazon.com/dp/B00GSQDRDC/?tag=tonymacx86com-20
By the time you get finished modding a G5, you will have spent more than the cost of the NAS, anyway.
This will allow you more time to decide exactly what you want in the way of a new video editing build.

The http://www.gigabyte.us/products/product-page.aspx?pid=5479#ov looks promising with 64GB RAM, but probably will not fit the G5. Add the same i7 6700K CPU as in the late 2015 iMac and you have plenty of processing power. Given your requirements, the closest case would be a http://www.lian-li.com/en/dt_portfolio/pc-x2000fn/

 

[spaceman33]

Thanks for your reply but I believe it's misguided. Firstly, you misunderstood my needs, secondly your math is way off.

1. The NAS is out of the question. I need 600mb/s min. The bottleneck there is if I wanted to use 10gbE I could swing that but thats $$$$. The enclosure for $1500 1. Doesn't have it 2. The synology and qnap software is more headache than they're worth. I've used both and the level of integration with OS X leaves a lot to be desired. It will also not support zfs. I'd be spending 1,500 for essentially a lesser quality version of what I already have minus a few hot swap bays.

2. The high point is a nice hunk of material and I've looked at it, but at 2,500 I can do better. Also, it's rack mount. The whole idea here is to have an elegant solution, not turn my home office into a server room. Also No pci-E to sas card... Basically 2,500 for a Norco that can fit a Mac mini. I can get 12+ hot swap bays inside a G5 or other enclosure for a bit over 1k with a pci-E hba and thunderbolt adapter.

3. The gigabyte board looks promising however I'd need more info on its compatibility/ ease of install / case fitting before taking a plunge. Again, building a full machine is the less optimal solution. A "hackintosh das raid array" in a G5 would be ideal for keeping cost down.

4. The case is way overkill. if I can fit 8 ssds in a 1x5.25 with an Icydock, so my full requirements can be met with 2 5.25 bays as long as I can fit an adequate power supply and pci-e to thunderbolt in there if going the das route. I don't need a beast, I just need to be clever. Link here: http://www.icydock.com/goods.php?id=192

If i can fit the Icydock in the 5.25 bay, I can live with the internal cage for the other disks. But if I can't get a solid, recommended motherboard with thunderbolt 2 that can support 64gb of ram then I should consider a DAS solution with the case.

basically the question becomes two part if/then:

1. Is there a thunderbolt 2 mobo that can support 64gb that is well reviewed / tested within the community and is stable in El Capitan? If so has anyone fit it in a G5? If so I can justify canning the mini in favor of a true upgrade rather than a marginal uptick that I'll have to upgrade in the near future.

2. Has anyone built a hackintosh SAN or DAS using thunderbolt? Basically I'm looking to do a cheap build that meets roughly capabilities and capacities of two promise r8s or r6s in one machine. I don't care what size drives, I just need around 12-16 drives and minimum 600mb/s interface. Ideally I'd like it all to look elegant within one enclosure, preferably a G5 as I have one available.

I priced out the sonnet box, sas hba cards, hot swap bays and drives to be less than the cost of one promise machine. So it's doable I think, I just haven't seen it done... There's no build to follow.

 

[Going Bald]

basically the question becomes two part if/then:

1. Is there a thunderbolt 2 mobo that can support 64gb that is well reviewed / tested within the community and is stable in El Capitan? If so has anyone fit it in a G5? If so I can justify canning the mini in favor of a true upgrade rather than a marginal uptick that I'll have to upgrade in the near future.

2. Has anyone built a hackintosh SAN or DAS using thunderbolt? Basically I'm looking to do a cheap build that meets roughly capabilities and capacities of two promise r8s or r6s in one machine. I don't care what size drives, I just need around 12-16 drives and minimum 600mb/s interface. Ideally I'd like it all to look elegant within one enclosure, preferably a G5 as I have one available.

I priced out the sonnet box, sas hba cards, hot swap bays and drives to be less than the cost of one promise machine. So it's doable I think, I just haven't seen it done... There's no build to follow.

1. TB2 boards were basically only Gigabyte or Asus and Asus requires a PCIe card. These were Z77/Z87 boards. Seems the OEMs dropped TB in the Z97 series except for the GA-Z97-UD7 TH, but it only supports 32GB RAM. Gigabyte boards seem the only ones with TB in the Z170 series, but it is not fully supported yet.
2. Have not seen any evidence anyone has done this - google search can turn up nothing.

Problem with the G5 case is size - you almost have to go with a mATX board to get it to fit and then you don't have enough SATA ports or PCIe slots.

Good luck with it, but you are breaking new ground. I would be interested to see a build for it in the Workshop/Case Mods forum if you manage it. Take plenty of pictures during the process.

 

[spaceman33]

Thanks for your reply.

No I haven't seen any evidence of anyone attempting this, and I'm kind of shocked... Especially with the price of thunderbolt peripherals so insanely high. The Promise r8 with 32tb is 4K, and if you are familiar with Raid, going over 2tb in a raid5 or raidz situation is asking for trouble. I think in an 8 disk array going over 16tb with anything but mirrors or striping for pure speed is asking for trouble. Personally I wouldn't trust my data on it regardless of the track record of promise, which is great.

the closest I've seen is a guy building an external graphics card enclosure with an akitio thunder2 pci-E. I think the same principles will work for a disk array.

I am going to price out a build for a DAS with links in a post a little later this evening and curious to hear everyone's input.

i think finding a solution to this could be incredibly useful for the design and editing community here that's a bit disenchanted with thunderbolt peripheral costs and lack of upgrade path.

 

[spaceman33]

Build Questions: A Rock Solid Thunderbolt DAS / Promise R8 Killer a G5

Conceptually the Highpoint Rocketraid illustrates perfectly what I'm trying to attain, however they are quite pricey ($670 currently) and I could get an HBA along with the akitio pci-E unit for about half (see below) Also, they require a whole ton of additional hardware. Functionally I'd like to have the power of 1-3x Highpoint Rocketraid 6328 in one box, along with drives, attached to one power supply and 1-2x thunderbolt2 output.



Here are some proposed build options (I tried to make it open to all, rather than specific to my needs, so others can benefit)

PCI-E to Thunderbolt Adapter:

Akitio Thunder2 PCI-E (4x 5Gb/s PCI lanes , 2.0): http://amzn.com/B00LTAUTHE (~$215 usd currently)

Or

Sonnet Echo Express II with 2x PCI-E: http://amzn.com/B00FMJPWFW (~$425 usd currently, but gives me two PCI-E Slots, reasoning explained below)

Or

Rocketstor 6328 ($670 usd currently), 8 ports: http://amzn.com/B00J06F6EU


HBA Card:

Many options currently out there, especially cheap on eBay considering many servers are going from 6Gb/s to 12Gb/s SAS... a wealth of cards by LSI, Highpoint, and branded stuff for Dell Perc servers etc.)

The main issue I'm still confused about is how the thunderbolt and HBA cards will handle going from 8 lane PCI-E to 4-Lane. If it is stable, it may make sense to use 8 lane cards and then upgrade the thunderbolt-pci-E adapter when they come out (probably for thunderbolt 3). Currently it doesn't make sense to spring for the much more expensive 8 lane Sonnett PCI-E card as it will saturate thunderbolt II easily. It would however make sense if I find the 4 lane PCI-E 2.0 introduces issues.

Most of the time I'll only be using half the disks, as the other disks will be backup that runs while I'm not working.. so I don't need max throughput from all disks at all times, which makes the pci-e 4 lane seem adequate.

For my uses I don't need Raid on the controller, but raid can be disabled on most of these cards to use solely as an HBA (host bus adapter).

Highpoint Rocketraid 2720GL is well reviewed so I'll include that as an option for an 8-port solution: [FONT=Arial][COLOR=#111111]http://amzn.com/B0050SLTPC[/COLOR][/FONT] (Currently ~$150 use) . This is an 8 lane card.

Since each lane can support 6gb/s, that means each can utilize a port multiplier if using 7200rpm disks, as one disk will not saturate a lane. If each lane went to a port multiplier, that could support 40 7200rpm I believe on one controller.

16 Port options that can be found on eBay cheaply (~$100 usd) such as the LSI 9201, LSI 9200, Dell Perc6 cards, and a variety of others by Adaptec, IBM, Highpoint, Areca, etc.

Port Multiplier (optional):

Addonics 6-Port Sata (1 to 5) http://amzn.com/B0031X58H8 (Currently ~$70 usd)


Power Supply:

Dependent on amount of disks, enclosure, etc. Could use some suggestions.

Case and hot swap:

Looking for suggestions. I have a g5 case which I would like to use. I don't necessarily need hot swap, as I'm not running 24-7 VMs or anything and can power down for drive swap) Question: How would I determine which drive is to be removed without a hot swap fail light indicator?

Using cheap, plain HD cages I could just cram as many in as I could into the G5 with some heavy duty fans and cut out the extra expense.

For a 12-24 bay easy solution I came across this:

iStarUSA S35 http://amzn.com/B00DJ7T3YK (~$85 currently)

which could support 24 2.5" disks using:

IcyDock Tougharmor 8x2.5" to 1x5.25 hot swap enclosures: http://amzn.com/B00TL4US8K (~$130 currently)

The cheap solution would be to mount as many of these into the g5 case that I have, and live without hot swap:

Sans Digital HDDRACK5 http://amzn.com/B001LF40KE

Fans and cooling:

Dependent on enclosure. Open to suggestions.


Some notes on why I'm doing this:

Price performance:
Again, the price performance of this build starts to really make sense at greater than 8 disks. The akitio thunder2 quads are around $350 usd and the minis are about $300. So, 8 disk enclosure = 600-700, but enclosing 16 disks 1200-1400, and well, you get the idea. The incentive to build your own increases as your needs do.

Backups:
NEVER UNDERESTIMATE the importance of backups.
If I have 2 enclosures of data, I'd want 2 additional enclosures to back that data up. The more safety you want to have in production multiplies the expense / size / # of disks / etc.

Raid:

1. Raid reduces your useable space. If you are using mirrors, double the amount of disks you need.

2. I'd strongly caution against raid5, especially with greater than 2tb (or 1tb for that matter) disks. I'd use a minimum of raidz2 with a spare or raidz3- this means an extra 2-3 disks per vdev than your actual . This all depends on your speed needs, but the point is to plan for failure, and accurately calculate your true disk needs.

Let's say I need 8tb usable and 600mb/s + writes, i'd use raidz3 with 2tb disks, but I'd need 7, not 4. Using 1tb disks I'd need 11. To have a fast backup, I'd need 22.

Stable, Safe, Fast, Inexpensive and Upgradable:

To have production redundancy, backups and speed all in one box, you need to go BIG- lots of disks. Pricing out my needs on available options from Promise or Areca puts me in the 8k department, but using components from this build I think I can suit my needs for around 1/4 of that, and sacrifice nothing - perhaps even gain quality and upgradability that isn't present in the enterprise units.

No Mess:
Also, all the disks and enclosures I'd need is a serious clusterf*ck. With thunderbolt now the norm, everything has its own cord, own power supply, and additional mess. I wanted one unit that would do it all.

This image accurately depicts how I feel about the new Mac Pro and some of my reasoning behind this build:

 

[spaceman33]

Well, time to break some new ground then. I'm really shocked no one has attempted this yet given the exorbitant price of thunderbolt peripherals.

Hopefully this will be of benefit to others in the community.

Take a peek at some of the build options I just went over and let me know your thoughts.

Also, since the direction is changing for the build, i'd like to replace 'Promise r8 killer' for 'file server' as it seems I can't accomplish what I need a full-on build.

 

[dude05]

the new qnap nas can do the trick may be.

otherwise u have to stick to an X99 with haswell-E processor, 64gbram, 16x hdd using an areca1882ix-16 internal raid card

if u r ok with 2.5" drives, u can always find the icydock case that holds 6-8 2.5 hdd in a 5.25 bay

 

[Sennen]

If it was me I'd be looking at something like the Areca 8050T2 with NAS/enterprise drives although it is fewer drives than you need.

 

[alexjibcam]

Hey there,

Sorry to dig up an old post but I was wondering if you ever finished your build?

I'm very interested for the same purpose for my video production company as a centralised raid server to serve all machines over a thunderbolt bridge network.

Thanks,

Alex