Contribute
Register

Need help with the internal "best practice" storage concept using NLEs with my upcoming Hackintosh

Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
Currently i’m building a new system using FCPX, Premiere/AE NLEs based on a Hackintosh.
This time i want to choose and setup the internal storage subsystem for the workflow wisely.
I’ll do something like a 256GB SSD for OS+applications, but then i was told to do 2+ SSDs in a RAID0 configuration as my 'scratch disk' that i do all of my work on.
It's not redundant, but they said they never heard of an SSD failing.
Finally, i was told to do a cheaper HDD for my 'archive drive‘.
I’m not really sure to choose SSDs or HDDs for a scratch disc in a RAID0.
Personally i’m planning with an external RAID5 backup solution based on a Qnap or Synology system in the upcoming months.
What configuration you might recommend for a wise workflow with this NLEs?
 
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
Someone asked a similar question and he was told in the Adobe forums:

Let's start with the terminology. Once you understand that, things will become a lot easier.

MEDIA: Name for all the raw material you shot, whether video or pictures or audio recordings. It does not matter whether this was captured from tape based video cameras, or imported from card or disk based cameras.

MEDIA CACHE: When Media are imported into a project, PR automatically creates three kind of files that are stored in the location you set up in the Preferences. They are .IMS files or indexed files, .CFA or conformed audio files and last .PEK files, that display the audio waveform in the timeline.

MEDIA CACHE DATABASE: Also automatically created by PR and contains pointers to the files in the Media Cache.

Note that occasionally the Media Cache and Media Cache Database can become corrupted and in that case it is a good idea to Clean the Media Cache from Preferences. When opening a project indexing, conforming and generation of PEK files is automatically done. Just be a bit patient for that process to finish.

PREVIEWS: Also called RENDER files. When you render a timeline to turn the red bar into a green bar for preview purposes, PR stores .XMP files in the PRV directory location you have set in the Project Settings.

Media files are typically very large and require a good sustained sequential read speed. Once ingested (imported or captured) they remain unchanged, because PR is a non-destructive editor. On the one hand this makes a SSD an ideal type of disk, because it is only reading the media files and not writing, but the size may be the limiting factor.

Media cache files are typically small and many, so here sustained R/W transfer rates are important, making it less than ideal to put them on a SSD, because the major drawback of SSD's is their limited and finite number of writes.

Now Tom, to your specific situation:

C: SSD for OS & programs.
D: HDD for media and projects (because of the size of the media and frequent changes in projects).
E: SSD for static page-file and exports.
F: SSD for media cache and previews, although a HDD may give you better life expectancy but also slower performance.

Maybe this article can be helpful as well, because it shows the impact of codecs on your system: Adobe Forums: What PC to build? An update...

You may have to get additional HDD's for raid configuations, if you edit 4K material or 422 material, because it is all about transfer rates.

To give you a rough indications of what you can expect from your drives, I suggest you get HD Tune Pro to test for yourself, but here are some indicative figures:

SSD: R around 450 MB/s, W around 300 MB/s, depending on the brand/model in steady state with SATA 6G. SATA 3G is far slower. Sandforce controllers may top out around 250 MB/s W.

HDD: R around 150 MB/s, W around 140 MB/s
 
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
The logic board im planning with has PCIe M2 slots. and i’ll use one with a AHCI SSD for the primary boot drive. Futhermore there are plenty SATA Express and SATA3 ports:
http://www.gigabyte.com/products/product-page.aspx?pid=5497#ov

I already thought about forgetting M2 SSDs and just getting normal SATA SSD + HDDs.
Concerning to my upper question i need as much as possible usable internal SATA ports free. I think a ~500MB/s SATA SSD would be enough for a OS+application drive, correct me if i’m wrong.
For now i make my mind about the internal storage concept. I mean which type of disc drive (SATA SSD or HDD) i should use as a single drive, RAID 0 or 1 for what usage in terms of FCPX, Premiere/AE and maybe Lightroom.
This time i just want to do it right and thats why i need some help.

I watched this youtube video here:
His concept is the following:
3x Samsung EVO SATA SSDs:
2x 250GB
1x 512GB

2x WD Blue SATA HDDs:
2x 1TB 7200RPM

250GB SSD OS + Applications
250GB SSD media scratch disk and project files
512GB SSD export drive
2x 1TB HDD as RAID1 data drive for storing data files. actual footage dump from his camera goes on here ????

Then there is the concept mentioned before from the Adobe forums which seems to be more Adobe specific:

SSD for OS & programs.
HDD for media and projects (because of the size of the media and frequent changes in projects).
SSD for static page-file and exports.
SSD for media cache and previews, although a HDD may give you better life expectancy but also slower performance.

It's really confusing since i have to find a concept which is good for both worlds in FCPX and Premiere/AE, maybe also for Lightroom. Then i have to throw in that i just have six SATA-6GBit/s-ports plus i need a separate place to store daily stuff on which has nothing to do with NLEs. The boot drive is no option cause i just want it as a clean OS+application drive.
 
Last edited:
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
The problem is that the RX 480 isn't able to work as primary GPU at the moment, so you'll need to configure your iGPU as primary (which can introduce problems FCPX).

As for the PCIe lanes: There are also some PCIe lanes coming from the PCH (through DMI connection), so you can easily use a fast GPU and PCIe m.2 storage at the same time.

The Samsung SM951 is available both in AHCI and NVMe flavors. AHCI tends to be a bit more pricey, but that's just how it is. Compare with Apple SSD prices and be happy! :thumbup:
I red on a german site a test of my choosen logic board that it has 3 SATAe ports which are natively bound at the Z170 chipset.
-> http://www.gigabyte.com/products/product-page.aspx?pid=5497#ov

This means you have six SATA-6GBit/s-ports which could be used at the same time.
Gigabyte bound the upper M2 slot a bit problematic to the chipset.
It works for sure with max. 4 PCIe-3.0-lanes but it’ll make SATA-ports 0 to 3 unusable so that you’ll just have two SATA ports left.
Even when your M2 module effectively just uses two lanes, SATA ports 2 and 3 won’t be usable.
They mentioned that it’d be much better to use the lower M2 slot so that you’ll just loose SATA port 0 and the left 5 SATA ports will still be available.
The even more important question i have is, that if a M2 SSD uses 4 PCIe lanes and a non Xeon i7 Skylake CPU just serves 16 PCIe lanes and the graphics card with an RX480 GPU wants 16 lanes, the M2 SSD will cut the lanes available for the CPU which will reduce it’s overall speed.
I’ve no experience with this kind of SSD memory chip PCBs, but i saw that they’re expensive, especially AHCI ones which ones which are recommended for Hackintoshs and are slower than NVMe types.
I already thought about forgetting M2 SSDs and just getting normal SATA SSD + HDDs.
Concerning to my upper question i need as much as possible usable internal SATA ports free. I think a ~500MB/s SATA SSD would be enough for a OS+application drive, correct me if i’m wrong.
For now i make my mind about the internal storage concept. I mean which type of disc drive (SATA SSD or HDD) i should use as a single drive, RAID 0 or 1 for what usage in terms of FCPX, Premiere/AE and maybe Lightroom.
This time i just want to do it right and thats why i need some help.
 
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
The latest version of Premiere Pro has Metal support under Project Settings for Video Rendering and Playback.

Speaking as someone who supports a ton of media companies they almost all use Adobe Premiere for editing. I know of two clients who use Avid, one client who still uses FCP 7 (somehow) and one who uses FCP X. And that client with FCP X is starting to switch to Premiere. So you should learn Premiere, it's easy to pick up if you've ever used FCP 7. Oh and they're all on Macs which is all AMD so sure it may be more CUDA but with Metal support who knows.

I'll throw a demo of Premiere and run some tests between CPU, OpenCL and Metal. I can't test CUDA since I don't have an Nvidia card. Though it'll be a benchmark project (probably from http://ppbm7.com/index.php/homepage/instructions) though I just remembered I have some test footage from a client from a C300, two Sony cameras and a RED all in camera raw format so we'll see how Premiere handles those.
Because of your knowledge, i'd like to ask you if you could point me to a direction about the internal storage concept here: https://www.tonymacx86.com/threads/need-help-with-the-internal-best-practice-storage-concept-using-nles-with-my-upcoming-hackintosh.211785/#post-1409710
 

Fl0r!an

Moderator
Joined
Jul 6, 2010
Messages
2,714
Motherboard
Asus Z170M-Plus
CPU
Intel i5-6600K
Graphics
AMD Radeon R9 280
Mac
MacBook Pro, Mac Pro
Mobile Phone
iOS
The m.2 slots do not share bandwidth with your GPU. The GPU slot is directly connected to the CPU's 16 PCIE lanes, while the m.2 slots are attached to the PCH, which uses DMI 3.0 connection (which offers a transfer speed equal to 4 PCIE 3.0 lanes).

The PCH includes a PCIE switch to provide up to 20 PCIE lanes, which obviously will share their bandwidth though, as the DMI connection can't feed all of them at full speed.
It's still quite a lot, 4x PCIE 3.0 equals to approx. 4GB/s, so you still need some insane SSDs to saturate that bus. (Same is true for the x16 link on your GPU btw).

PCIE m.2 SSDs shouldn't affect the functionality of your SATA ports at all (unless the mainboard vendor has implemented this in a very stupid way). Can you link the test you've mentioned?
 
Joined
Oct 5, 2013
Messages
507
Mac
Classic Mac
Mobile Phone
How about a slightly different approach?

Use a 500 GB or 1 TB SSD internal drive for apps and scratch disk (maybe a second smaller one as well), and build an external RAID --- type 10, not 0 or 1. Becareful about using only striped RAID approaches, if one drive crashes, you lose everything. Type 10 (mirrored and striped) is not too difficult to set up for someone just starting out and you get speed and safety. Later you can work up to type 5 or some other flavor. You can connect to the RAID using eSATA or USB.
 
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
The m.2 slots do not share bandwidth with your GPU. The GPU slot is directly connected to the CPU's 16 PCIE lanes, while the m.2 slots are attached to the PCH, which uses DMI 3.0 connection (which offers a transfer speed equal to 4 PCIE 3.0 lanes).

The PCH includes a PCIE switch to provide up to 20 PCIE lanes, which obviously will share their bandwidth though, as the DMI connection can't feed all of them at full speed.
It's still quite a lot, 4x PCIE 3.0 equals to approx. 4GB/s, so you still need some insane SSDs to saturate that bus. (Same is true for the x16 link on your GPU btw).

PCIE m.2 SSDs shouldn't affect the functionality of your SATA ports at all (unless the mainboard vendor has implemented this in a very stupid way). Can you link the test you've mentioned?
Sure. You have to translate yourself the block unter the picture where the SATAe ports are shown: http://www.hardwareluxx.de/index.php/artikel/hardware/mainboards/36530-gigabyte-ga-z170x-gaming-3-im-test.html?start=1
 
Joined
Dec 26, 2016
Messages
123
Motherboard
Gigabyte GA-Z170X-Gaming 3
CPU
i7 6700K
Graphics
RX 480
Mac
MacBook, MacBook Pro
Classic Mac
Power Mac
How about a slightly different approach?

Use a 500 GB or 1 TB SSD internal drive for apps and scratch disk (maybe a second smaller one as well), and build an external RAID --- type 10, not 0 or 1. Becareful about using only striped RAID approaches, if one drive crashes, you lose everything. Type 10 (mirrored and striped) is not too difficult to set up for someone just starting out and you get speed and safety. Later you can work up to type 5 or some other flavor. You can connect to the RAID using eSATA or USB.
Thank you for your thought.
I just started a new thread not to flood this one with my off topic store concepts headache.
Please feel welcome to add your knowledge here cause i already stated in detail my thoughts here: https://www.tonymacx86.com/threads/...h-my-upcoming-hackintosh.211785/#post-1409710
 
Top