TrueNAS and Other Custom NAS Builds

[CaseySJ]

I should add that I recently built a heavily non-conventional TrueNAS Scale system based on these components:

• Intel Alder Lake i5-12600K with Asus ROG Strix Z690-G Gaming WiFi mATX -- excellent combo price from NewEgg
• 64GB (2x32GB) DDR5 5600MHz CL30 G.Skill Ripjaws S5
• ID-Cooling IS-55 low-profile air cooler
• Treasure Nordic 8-bay NAS chassis from Ali Express
• QNAP QM2 2P10G1TB PCIe x8 card with AQC113C (10GbE) and two PCIe x4 NVMe SSD slots
• Five 14TB Seagate Exos hard drives (manufacturer re-certified with 2-year warranty)
• Intel Optane M10 64GB for use as a cache drive
• Motherboard has 6 SATA ports, but I needed a few more so I added a low-cost PCIe x1 card with 4 additional ports

Even though the CPU and motherboard are a bit overkill, I don't want a slow NAS. This system doesn't stay powered on 24x7. It runs about 12 hours a day (from whole-home solar panels for zero carbon footprint), but here E-cores are valuable because of lower power consumption.

...

Brief update after a month:

• My TrueNAS system starts up and shuts down 100x faster than my 2022 QNAP TS-h973AX-32G-US with an AMD Ryzen V1500B 2.2GHz embedded processor with 4 cores and 8 threads
• My TrueNAS system barely skips a beat when doing multiple CPU- and IO-intensive tasks at the same time; multi-tasking on the QNAP slows to a crawl even when it's performing something simple like a remote RSYNC backup
• Speed and responsiveness of the QNAP are quite disappointing
• I've transferred nearly everything from QNAP to TrueNAS using a series of RSYNC transfers
• I've also transferred everything from an older ReadyNAS system (8TB)
• The TrueNAS system consumes about 80-85W as measured with Kill-a-Watt
  • The home's solar array generates 2 times as much power as the home consumes

Alas, the QNAP is not conducive to my needs so it will likely be sold. QNAP's operating system (QTS and QuTS) is much richer and much nicer than TrueNAS (in my opinion of course), but it demands a more powerful processor than the Ryzen V1500B.

My TrueNAS Scale system is the fastest, most responsive, and most satisfying NAS that I've ever owned. I would not settle for anything slower.

 

[oli.mathieu]

Brief update after a month:

• My TrueNAS system starts up and shuts down 100x faster than my 2022 QNAP TS-h973AX-32G-US with an AMD Ryzen V1500B 2.2GHz embedded processor with 4 cores and 8 threads
• My TrueNAS system barely skips a beat when doing multiple CPU- and IO-intensive tasks as the same time; multi-tasking on the QNAP slows to a crawl even when it's performing something simple like a remote RSYNC backup
• Speed and responsiveness of the QNAP are quite disappointing
• I've transferred nearly everything from QNAP to TrueNAS using a series of RSYNC transfers
• I've also transferred everything from an older ReadyNAS system (8TB)
• The TrueNAS system consumes about 80-85W as measured with Kill-a-Watt
  • The home's solar array generates 2 times as much power as the home consumes

Alas, the QNAP is not conducive to my needs so it will likely be sold. QNAP's operating system (QTS and QuTS) is much richer and much nicer than TrueNAS (in my opinion of course), but it demands a more powerful processor than the Ryzen V1500B.

My TrueNAS Scale system is the fastest, most responsive, and most satisfying NAS that I've ever owned. I would not settle for anything slower.


View attachment 567061

many Thanks for the infos

 

[etorix]

Thanks for the update.

My TrueNAS system starts up and shuts down 100x faster than my 2022 QNAP TS-h973AX-32G-US with an AMD Ryzen V1500B 2.2GHz embedded processor with 4 cores and 8 threads

My first thought was that this hardly matters, but I'm of the old school of never shutting servers down and never spinning down spinners. My NASes also run scheduled replication, SMART tests and scrub tasks at night. I hope you also have regular snapshot, SMART and scrub tasks for protection (ransomwares… or user errors) monitoring, data integrity.

(E cores only in my case… Hybrid architectures are not yet supported by TrueNAS.)

 

[CaseySJ]

…
My first thought was that this hardly matters, …

We should avoid statements like this because they assume our way is the only right way. That’s one of the points I keep raising, that there are (and should be) different ways of doing things.

 

[gandem]

Brief update after a month:

• My TrueNAS system starts up and shuts down 100x faster than my 2022 QNAP TS-h973AX-32G-US with an AMD Ryzen V1500B 2.2GHz embedded processor with 4 cores and 8 threads
• My TrueNAS system barely skips a beat when doing multiple CPU- and IO-intensive tasks at the same time; multi-tasking on the QNAP slows to a crawl even when it's performing something simple like a remote RSYNC backup
• Speed and responsiveness of the QNAP are quite disappointing
• I've transferred nearly everything from QNAP to TrueNAS using a series of RSYNC transfers
• I've also transferred everything from an older ReadyNAS system (8TB)
• The TrueNAS system consumes about 80-85W as measured with Kill-a-Watt
  • The home's solar array generates 2 times as much power as the home consumes

Alas, the QNAP is not conducive to my needs so it will likely be sold. QNAP's operating system (QTS and QuTS) is much richer and much nicer than TrueNAS (in my opinion of course), but it demands a more powerful processor than the Ryzen V1500B.

My TrueNAS Scale system is the fastest, most responsive, and most satisfying NAS that I've ever owned. I would not settle for anything slower.

This setup is absolutely blowing my mind. I too have an old (ancient) ReadyNAS system that desperately needs to be replaced. I never once considered building one. A couple of questions...

Are you using the Intel Optane M10 as a cache on the MB or the NIC?
What are you using to boot the TrueNAS iso- USB?
You added more SATA ports- What RAID setups are you using?
Are you using the hardware RAID functionality on the MB?
What PSU are you using?

I'm considering using an appliance such as this as a household backup for many MacOS/Windows/iOS hosts along with a photo archive, but also as a near line for my video projects. I don't run a media server to stream anything. I've already gone through the pain of upgrading the wired network all through my home to 2.5Gb, with a 10Gb switch to my main home video editing system (Z690 ProArt Creator)

While this may be an overkill system for how you're using it, do you think this system would have the power to run the uses I talk about above?

The timing for something like this is right for me, as I have multiple needs. When I look at the appliances that exist, the price is not exactly cheap. I'm wondering if this could be a good fit?

 

[CaseySJ]

@gandem,

This is a current hot topic for me so I’ll reply properly after I get back from one of my favorite craft coffee shops. It’s all about the latté after an intense morning workout.

 

[etorix]

Are you using the Intel Optane M10 as a cache on the MB or the NIC?

As L2ARC (ZFS parlance for a second level cache, the first one being RAM). Do not consider this feature unless you have at least 64 GB RAM.

What are you using to boot the TrueNAS iso- USB?

Advised against, because these tend to wear out quickly. Any cheap and small SSD (possibly on a USB adapter not to use a port) is recommended.

You added more SATA ports- What RAID setups are you using?
Are you using the hardware RAID functionality on the MB?

(Cue flames and thunder…) Thou shall not use RAID controllers with ZFS!
AHCI mode on the motherboard. Get one with enough ports; else use a LSI HBA (again: for ZFS).

Hardware recommendations may vary (though not by much, with the possible exception of using RAID controllers) if you go for OpenMediaVault or UnRAID rather than TrueNAS or XigmaNAS.

While this may be an overkill system for how you're using it, do you think this system would have the power to run the uses I talk about above?

Your needs are a basic file server, over 10 GbE. Such a system is way overpowered for that.
Unless you want to edit 4K ProRes/8K directly on the NAS—then, reportedly, one goes SSD-only, preferably NVMe-only.

Better open a thread under Buying Advice or General Hardware Discussion for NAS building. Or directly on the forum of the NAS OS, if you've decided on one.

 

[CaseySJ]

This setup is absolutely blowing my mind. I too have an old (ancient) ReadyNAS system that desperately needs to be replaced. I never once considered building one. A couple of questions...

I have a 4-bay Netgear ReadyNAS RN214 that has two 1 Gigabit Ethernet ports. It's populated with four 6TB hard drives in RAID-5.

• Total used space is 8TB, which would have taken much too long to transfer via network so I copied everything to a USB 3.0 12TB HDD formatted as ext4
• But TrueNAS does not recognize ext4 so I formatted a spare 14TB HDD as zfs in Ubuntu 22.10 and rsync'ed everything from the ext4 drive to the zfs drive
• Then I installed the zfs drive into an available bay in the TrueNAS enclosure, which has 8 drive bays with 2 bays available
• Next, I opened the shell in TrueNAS Scale and rsync'ed everything from the standalone drive to the TrueNAS storage pool
• However, the rsync command will be terminated whenever we leave the shell or put the Mac to sleep. So it was necessary to do this:

% tmux
% rsync -av <src> <dest>

Press CTRL-B then D to leave the above command running and return to main shell

// to reopen the shell that is running our rsync command, we type

% tmux attach

Are you using the Intel Optane M10 as a cache on the MB or the NIC?

The Optane M10 is currently used as a read cache, but I'll switch it to a write cache because there's plenty of physical memory and the system IO is not read-limited. I don't mind experimenting -- in fact that is one of my goals.

What are you using to boot the TrueNAS iso- USB?

There are 8 bays in this chassis. Six of them are populated with 14TB Seagate Exos HDDs. The 7th and 8th will be used in the future. Boot drive is a 128GB NVMe SSD. The are 3 NVMe slots on the motherboard, and another two on the QNAP 10GbE add-in-card.

You added more SATA ports- What RAID setups are you using?

RAID is managed entirely by TrueNAS Scale. I'm using RAID1Z that is similar to RAID5 in that it tolerates 1 drive failure. No hardware RAID is used.

I purchased a cheap 4-port PCIe x1 SATA add-in-card from Ali Express. It has a good ASMedia chip. TrueNAS Scale has absolutely no problem with that controller.

Are you using the hardware RAID functionality on the MB?

No.

What PSU are you using?

I'm using this PSU:

https://www.amazon.com/dp/B08BF1H5R1/?tag=tonymacx86com-20

I'm considering using an appliance such as this as a household backup for many MacOS/Windows/iOS hosts along with a photo archive, but also as a near line for my video projects. I don't run a media server to stream anything. I've already gone through the pain of upgrading the wired network all through my home to 2.5Gb, with a 10Gb switch to my main home video editing system (Z690 ProArt Creator)

While this may be an overkill system for how you're using it, do you think this system would have the power to run the uses I talk about above?

Yes, this system will eat that for breakfast! I'm running Jellyfin (similar to Plex), which works quite well with the Intel iGPU (QuickSync). I don't have a discrete GPU in this system.

I took advantage of an excellent combo deal on NewEgg, which bundled the motherboard and CPU for a nearly $150 discount. I was so tired of the slow-as-molasses commercial NAS systems that I vowed to make all of my own decisions in picking components.

The timing for something like this is right for me, as I have multiple needs. When I look at the appliances that exist, the price is not exactly cheap. I'm wondering if this could be a good fit?

Yes it can be an excellent fit because you have full control over the components that you believe will provide the experience you're seeking.

 

[CaseySJ]

Incidentally, Carbon Copy Cloner's SparseDisk images require a handful of special settings in TrueNAS Scale. After that, I have had no problem cloning my Hackintosh systems via network.

 

[zipb]

I've more or less given up on NAS for storage after 15 years.

For now, it's all internal NVME or SSD (for my main hack, 12TB), backed up to DAS (mostly spinning rust in a USB3 cradle) and Backblaze. Clean and simple. Wife and kids have external SSD's for backups, 1 per computer.

I once had to deal with a corrupted TimeMachine backup on a QNAP RAID-5 EXT4 setup infected with Qlocker ransomware. Never again. Glad I had a more or less current copy of the QNAP on an external USB disk.

Even with a simple QNAP NAS setup, it was fun but high maintenance and just not worth it.
To do it properly(building a fast and robust fileserver for example) you need:
a. good quality components and reliable software
b. specialized skills and know-how. I am not willing to invest the time developing these.

Even backing up my massive sample library to NAS was a pain. Slow as molasses for unclear reasons.