TrueNAS and Other Custom NAS Builds

[oli.mathieu]

"L2ARC" implies ZFS. The 5-SATA-ports-on-M.2 thingie is NOT suitable for use with ZFS: Dubious controller, and likely SATA port multiplier, which is an absolute no-go!
Get a LSI SAS HBA, or rather a motherboard with 8 SATA ports. Since this looks like a professional build (at least for a SME), why not go professional with a genuine server motherboard?

The 4-port NIC looks like overkill, and if bought on AliExpress there's a non-zero chance to end up with a fake Intel card. One port and a switch is a safer option than four direct connections to four clients.
You are not going to saturate a 10G link with eight spinning drives—and certainly not going to be happy with direct editing on the NAS if that's the plan.

What's the L2ARC intended for? This is a read cache: It's not going to help with writes, and it's likely too small to hold all of the raw rushes of a project as a working set. Worse of all, 64 GB RAM is not enough to support that big a L2ARC (maximum: 5 to 10 times the RAM). As planned, a 4 TB L2ARC would negatively affect performance by evicting ARC (the "Level 1 cache").

Thanks

I tried to find some professional motherboard (like Supermicro or AsrockRack)in France ... but All the model I was looking for were unavailable. it was my first idea ..... to have ECC support
There is not much choice for mate montherboard with 8 Sata port .... none are available in France (Scammers excluded).
can you suggest some model. I'll double Check

I found the same Nic X710_TA4 to another seller.
The idea is to mimic the network connections of this.

Yes 8 HDD will never have the demanding Bandwidth ➧ L2ARC
Yes I only want to have fast read.
Your recommendation is to have 128GB or RAM and 1TB L2ARC, isn't it ?

Thanks Again

 

[CaseySJ]

I needed a few extra SATA ports and ended up buying this on Ali Express:

https://www.aliexpress.us/item/3256804826549031.html

It has been working very well with TrueNAS Scale.

For 10Gbps I purchased the QNAP QM2-2P10G1TB Dual M.2 2280 PCIe NVMe SSD & 10GbE Expansion Card. It uses a x8 slot and provides:

• One 10Gbps Ethernet port
• Two M.2 NVMe PCIe 3 x4 slots for storage

It also has been working well with TrueNAS Scale.

 

[etorix]

There is not much choice for mate montherboard with 8 Sata port .... none are available in France (Scammers excluded).
can you suggest some model. I'll double Check

Pretty much all micro-ATX server boards with C246 (Supermicro X11SCH, AsRockRack E3C246D4U), C256 (X12STH, E3C256D4U), C621/C622 (X11SPM, EPC621D6U) chipsets have (at least) 8 SATA ports. Plus the AsRockRack Ryzen server boards (X470D4U, X570D4U, B550D4U). I'd skip the DDR5 generation to rather go for cheaper DDR4 ECC RAM, preferably (lots of) RDIMM.
I suppose that the purpose is to share through SMB, so few cores but 3+ GHz clocks. My preference would go to a X11SPM (possibly -TF or -TPF with on-board 10G) with a low core count Xeon Gold—used 1st gen. Xeon Scalable are dirt cheap by now!—and lots of RDIMM.
As for French shops, I don't know—and the harder part would be to find professional refurbishers, so you could have refurbished CPU and RDIMM with proper invoices. For a new motherboard, I know German resellers, and there's always Anafra in the Czech Republic.

I found the same Nic X710_TA4 to another seller.
The idea is to mimic the network connections of this.

Stylish… but expensive and rather short on specifications (what's handling this RAID5???). Very Apple-ish, in the bad sense.

Your recommendation is to have 128GB or RAM and 1TB L2ARC, isn't it ?

More like it. But actually lots of RAM first (hence a bias towards RDIMM), use the system for some time and then use arc_summary to find out if you'd benefit from a L2ARC.

 

[oli.mathieu]

@etorix
many thanks for your advices. See some motherboard available in Anafra
Have a recommandation about with or without SoC :
ASRock - C3758D4U-2TP & ASRock - W680D4U

I see that for the same motherboard the maximum amount of Ram isn't the same between ECC & Non-ECC & RDIMM
Your recommandation is RDIMM isn't it ?

Stylish… but expensive and rather short on specifications (what's handling this RAID5???). Very Apple-ish, in the bad sense.

I totally agree

 

[oli.mathieu]

I suppose that the purpose is to share through SMB, so few cores but 3+ GHz clocks.

Yess

My preference would go to a X11SPM (possibly -TF or -TPF with on-board 10G) with a low core count Xeon Gold—used 1st gen. Xeon Scalable are dirt cheap by now!—and lots of RDIMM.

Very good/valuable advice : many thanks

 

[etorix]

I see that the Atom C3758 is "not in stock": Not good. But Supermicro A2SDi should be. This is a very nice low power NAS platform, if not necessarily the strongest SMB performer.
The difference between UDIMM and RDIMM is that RDIMM is designed for high capacity. An Atom C3000 can actually handle 256 GB RDIMM (and that's because it doesn't do LR-DIMM to go even higher…).

To put some numbers on my thinking:
A professionally refurbished Xeon Gold 5122 goes for 79,- E without tax. [email protected] GHz will do very well for serving SMB.
Professionally refurbished 32 GB RDIMM: 79,95 E a piece, without tax. (For private use, without invoice, that might be 50,- E from a friendly seller on the forum at ServeTheHome.)
A X11SPM (400-500 E new, excluding tax) will take six of these for 192 GB RAM, which may be more than actually needed—or may then support a sizeable L2ARC.

Actually, that's somewhat overkill to handle 8 HDDs. And not as many PCIe lanes as EPYC Rome/Milan to go for NVMe storage.
Back to the drawing board, beginning with a precise definition of the requirements. (You may also take it the TrueNAS forum for further advice, possibly even to the French speaking section for "local" advice—except that the most regular contributors in this section are… Swiss.)

Update: I was too pessimistic about used RDIMM prices. Right now, 32 GB RDIMM-2666V go for 30-35 E.

 

[oli.mathieu]

I see that the Atom C3758 is "not in stock": Not good. But Supermicro A2SDi should be. This is a very nice low power NAS platform, if not necessarily the strongest SMB performer.
The difference between UDIMM and RDIMM is that RDIMM is designed for high capacity. An Atom C3000 can actually handle 256 GB RDIMM (and that's because it doesn't do LR-DIMM to go even higher…).

Okay Thanks

To put some numbers on my thinking:
A professionally refurbished Xeon Gold 5122 goes for 79,- E without tax. [email protected] GHz will do very well for serving SMB.

That's the one I spotted yesterday evening

Professionally refurbished 32 GB RDIMM: 79,95 E a piece, without tax. (For private use, without invoice, that might be 50,- E from a friendly seller on the forum at ServeTheHome.)
A X11SPM (400-500 E new, excluding tax) will take six of these for 192 GB RAM, which may be more than actually needed—or may then support a sizeable L2ARC.

Seems more than reasonable. Thanks .... I'll double check

Actually, that's somewhat overkill to handle 8 HDDs. And not as many PCIe lanes as EPYC Rome/Milan to go for NVMe storage.
Back to the drawing board, beginning with a precise definition of the requirements. (You may also take it the TrueNAS forum for further advice, possibly even to the French speaking section for "local" advice—except that the most regular contributors in this section are… Swiss.)

Good advice .. again
Thanks you

 

[CaseySJ]

** How to Install Home Assistant on Orange Pi 5 and 5 Plus **
​

In a previous post I described how to configure TrueNAS for compatibility with Carbon Copy Cloner's sparsedisk backup option. Several weeks later I found myself referring back to that post to troubleshoot an issue.

This guide similarly serves two purposes. It shows how to install Home Assistant and provides a reference for future troubleshooting needs.

I'm using the following components:

• Orange Pi 5 Plus with 256GB eMMC and 5V USB-C Power Supply
• Aluminum case with WiFi/BT antennas
• Wi-Fi 6 / BT 5.2 card by Orange Pi (not all cards are compatible)

External References:

• Home Assistant website is located here:
  • https://home-assistant.io
• Setup instructions in this guide were compiled from 3 sources:
  1. https://gist.github.com/renatoccosta/c30f0b4216c8caaf1f202b0a0561b5d3
  2. https://community.home-assistant.io/t/orange-pi-5-with-home-assistant/498585/7
  3. https://community.home-assistant.io...ised-on-a-raspberry-pi-using-debian-12/247116

Orange Pi 5 supports installation on 3 different types of media, all of which are bootable:

• TF Card
  
  • This is simply a micro-SD card
  • Cheap to buy, but also slowest speed around 100 MB/s
  • Less reliable in the long run compared with other two options
• eMMC
  
  • Just a tiny bit larger than micro-SD, but snaps onto the motherboard
  • Supports 400 MB/s transfer speed
  • Orange Pi sells 32GB, 64GB, and 256GB versions
  • I have the 256GB version
• NVMe SSD
  
  • Orange Pi 5 Plus only supports full size 2280 NVMe SSDs, which also happen to be the most common
  • Orange Pi 5 (not Plus) supports the smallest size 2230 form factor
  • Both provide 4 lanes of PCIe 3.0
  • NVMe SSD is fastest of the three options, but also generates considerably more heat
  • It may not be suitable for use in a passively cooled chassis

Orange Pi boots from TF Card (micro-SD) by default. To boot from eMMC or NVMe SSD we need to follow a special procedure provided in the Wiki page:

• Orange Pi 5 Plus wiki page
• Orange Pi 5 wiki page

I experimented with both eMMC and NVMe SSD. For my passively cooled case (lovely single piece of aluminum), the best option turns out to be eMMC because it runs significantly cooler than NVMe SSD and provides more than sufficient disk I/O for Home Assistant.

---

Installation Procedure: Download and Install Operating System

Home Assistant can be installed in one of four different ways:

1. Home Assistant OS (HA OS)
   
   • Single fully integrated disk image that includes both the operating system and Home Assistant
2. Container
   
   • Provides the benefits of containerization (quick hosting on various platforms; container backups; container recovery/failover; etc.)
   • But the container does not include Add-Ons as we can see in the table below
3. Core
   
   • This is for Python aficionados who want maximum flexibility in setup
4. Supervised
   
   • This installs Home Assistant on a compatible operating system
   • We install the O/S ourselves, and we can install any other applications that we like

I chose Supervised because:

• Home Assistant does not provide an HA OS image for Orange Pi; they do provide an image for Raspberry Pi 4
• I have a Z-Wave network that requires an Add-On called Z-Wave JS UI
• I wanted to run my existing HomeSeer application on the same system at least until everything from that application is fully migrated to Home Assistant

Home Assistant is compatible only with Debian Linux. Orange Pi offers official Debian images for:

• Bullseye -- Debian 11
  
  • Do not use this for Home Assistant because package systemd-resolved has a conflict and will not get installed
• Bookworm -- Debian 12
  
  • Use either the Desktop or Server version
  • If you use Server version, be prepared to ssh into the Orange Pi from a different computer, but it is also possible to connect a local monitor, keyboard and mouse
  • If you use the Desktop version, it will provide a complete GUI desktop for local monitor, keyboard and mouse; this version will consume slightly more compute resources
    • I chose the Desktop version anyway

​

Our first step is to download Debian 12 (Bookworm) and burn the image to either (a) micro SD, (b) eMMC, or (c) NVMe SSD. We can use Balena Etcher for micro SD, but for eMMC and NVMe SSD I strongly recommend following the Orange Pi guidelines that make use of their own RKDevTool_Release_v3.15:

• eMMC instructions
• NVMe SSD instructions

RKDevTool will not only install the image, it will also resize partitions to fit the full size of the disk. The instructions above will also modify boot ROM to boot from either eMMC or NVMe SSD.

RKDevTool may or may not work on Windows 11, but it works well on Windows 10. I have simply installed Windows 10 on VMWare Fusion on my Z390 Designare running macOS Sonoma 14.2. Yes we can burn the image to eMMC or NVMe SSD by running Windows in a virtual machine on macOS!

---

Installation Procedure: Download and Install Home Assistant

Now that Debian 12 has been installed and booted, we can open a local window to determine the IP address of our Orange Pi. In a local terminal window, typing ip addr will give us the information we need, but we'll have to look for it as shown:

ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enP4p65s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether ...
    inet 192.168.1.122/24 brd 192.168.1.255 scope global dynamic noprefixroute enP4p65s0
       valid_lft 85874sec preferred_lft 85874sec
    inet6 ... scope global dynamic noprefixroute 
       valid_lft 1634sec preferred_lft 1634sec
    inet6 ... scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: enP3p49s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether ...
4: wlP2p33s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether ...
5: wlan1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether ...

In item 2 we can see the IP address is 192.168.1.122. Now we can continue using the local terminal window or remotely login to the Orange Pi via ssh. In this section I'll use remote ssh from macOS running Terminal so I can capture screenshots for this guide, but you should use a local terminal. Connect a monitor, keyboard and mouse to your Orange Pi 5 or 5 Plus and perform these steps from a local terminal window.

In a Terminal window we open a secure-shell (ssh) to the Orange Pi and we're greeted as follows:

​

The standard login IDs for Orange Pi are:

• User account:
  • Login: orangepi
  • Pasword: orangepi
• Root (sudo) account:
  • Login: root
  • Password: orangepi

We can change passwords later, but we'll use default ones for now.

Installing Home Assistant is done by invoking a couple of scripts that will both (a) download the software and (b) install the software for us. But before we can invoke those scripts, we have a few prerequisites to take care of.

Update the OS
We start by making sure we have the latest patches and other O/S updates by running these commands:

sudo apt update
...wait for update to finish...

sudo apt upgrade
... wait for upgrade to finish...

Now we'll create a new user account for Home Assistant with the adduser YOURUSERNAME command. Choose a user name. In the example below I chose "casey". Enter a password for this user when asked.

sudo adduser casey

... this will appear as output ...
Adding user `casey' ...
Adding new group `casey' (1001) ...
Adding new user `casey' (1001) with group `casey' ...
Creating home directory `/home/casey' ...
Copying files from `/etc/skel' ...

New password: 
Retype new password:

Then provide the following information when asked (optional):

passwd: password updated successfully
Changing the user information for casey
Enter the new value, or press ENTER for the default
    Full Name []: Casey SJ     
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] Y

Now we'll add our new user to the sudo group so we have elevated system privileges (replace 'casey' with your own user name':

sudo usermod -aG sudo casey

Next, we'll install necessary packages as follows:

switch to sudo shell
sudo -i

# install dependencies
apt install apparmor jq wget curl udisks2 libglib2.0-bin network-manager dbus lsb-release systemd-journal-remote systemd-resolved -y
... wait for installation to finish ...
... select OK if you see a warning about systemd replacement ...

Do not install Docker because it is already installed in the Orange Pi Debian image!

Now we need to make some changes to the Orange Pi boot environment. It's quite simple, as follows:

# Type this command:
orangepi-config
... if you see an error or warning message, press Enter (not CTRL-C) to continue ...

This will open the config menu as follows...

Step 1: Press Enter to select "System and security settings"

Step 2: Press down arrow to select "Edit boot environment" then press Enter

Step 3: Use arrow keys to move cursor to the last row and just past the last character on that row so that a new line can be added by pressing Enter

Step 4: Press Enter to add a new line:

Step 5: Copy and paste this into the new line, then press Tab to select "Save" at the bottom of the dialog, and press Enter

extraargs=apparmor=1 security=apparmor systemd.unified_cgroup_hierarchy=0

Finally, select BACK and EXIT from the screens that appear:

Now we need to reboot by issuing the reboot command:

reboot

When the system restarts, log back via local terminal or ssh as your new username (which was 'casey' in my case), and let's verify apparmor and cgroups as follows:

ssh [email protected]
systemctl status apparmor.service

Do you see active (exited) as shown:

Now we'll verify cgroups:

findmnt -lo source,target,fstype,options -t cgroup,cgroup2

Do you see a bunch of lines like this:

If everything looks good and you've made it this far, the process is nearly over!!

Now we switch back to sudo and finish up the process:

sudo -i
... type your user password when asked (which is the password for your new user name) ...

... then type these commands ...
cd /usr/local/src

wget https://github.com/home-assistant/os-agent/releases/download/1.6.0/os-agent_1.6.0_linux_aarch64.deb
dpkg -i os-agent_1.6.0_linux_aarch64.deb

... so far so good? then continue below ...

wget https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb
dpkg -i homeassistant-supervised.deb

Important: You will likely be asked to choose a processor architecture from a pop-up window. Choose qemuarm-64

Congratulations!

Home Assistant operates on port 8123 by default so we can open a web browser as follows:

http://ip-addr:8123

In my case the IP address is 192.168.1.122 so I'll type:

http://192.168.1.122:8123

We can also try mDNS names:

• Orange Pi 5 Plus: http://orangepi5plus.local:8123
• Orange Pi 5: http://orangepi5.local:8123

Home Assistant will take a few minutes to get set up. And then you'll see the welcome screen:

It is a good idea to reboot the computer when you see the 2nd screenshot above ("Welcome!"). Home Assistant will restart automatically each time. Then continue with Home Assistant setup.

---

Removing Home Assistant (uninstall)

To remove Home Assistant, run these commands:

sudo -i
systemctl stop hassio-supervisor.service
systemctl stop hassio-apparmor.service

systemctl disable hassio-supervisor.service
systemctl disable hassio-apparmor.service

rm -rf /usr/sbin/hassio-supervisor
rm -rf /usr/sbin/hassio-apparmor

apt remove homeassistant-supervised

 

[CaseySJ]

Home Assistant has a large community of independent developers and enthusiasts who have provided add-ons and integrations (i.e. "drivers") for an impressive number of brands and devices. The Home Assistant Community Services add-on (HACS) is full of gems!

As I continue to build up my Home Assistant configuration, here's a look at its current state. At first I created separate dashboards in the sidebar to group related functions, but then I discovered dashboard tabs! I've moved everything from those sidebars into tabs and will delete the sidebars soon.

Left: A very busy main page; will be restructured later
Right: UniFi Protect integration into Home Assistant, showing all cameras

Left: Climate controls (Nest and Dyson integration)
Right: LG Washer and Dryer (ThinQ integration)

Left: Security page (Elk M1 Gold integration)
Right: Summary of my NAS (TrueNAS integration)

---

WARNING:

• The TrueNAS integration has a problem that causes massive buildup of Home Assistant log file and exhausts all available storage space
• Either disable the TrueNAS integration or modify configuration.yaml to exclude those logs from recorder

UPDATE:

• We can fix the log buildup problem by making changes as shown below to these two files, both of which are in homeassistant/custom_components/truenas folder:
  • truenas_api.py
  • truenas_controller.py
• Then restart Home Assistant from Settings -> System

 

[CaseySJ]

Development build of Home Assistant OS for Raspberry Pi 5 has recently been made available! RPi 5 does not support U-Boot that was supported by all previous versions of Pi. Hence the delay in finding and implementing an alternative supportable solution.

Raspberry Pi 5 support · home-assistant operating-system · Discussion #2844

The Raspberry Pi 5 looks like a really another really platform from the Raspberry Pi folks to run Home Assistant OS on . Obviously we'd like to support it as fast as possible. Currently the main ...

github.com

Update:

• Just installed Home Assistant OS (HAOS) on Raspberry Pi 5 after following instructions and discussion in the above link (fairly trivial installation compared with Orange Pi 5)
• Everything seems to be running just fine, but it's only been about 1 hour