Contribute
Register

TrueNAS and Other Custom NAS Builds

@gandem,

I have become somewhat of an expert in the Infuse Metadata Department, at least for commercially released movies, TV shows, documentaries and mini-series.

How are you naming your files?

Have you seen the Metadata 101 guide?
Apologies, I have been traveling for business.
Anyway, the original camera formats save the files in two different naming conventions:

  1. 20070816184537.m2ts (Date recorded)
  2. C0001.mxf (generic file name that just increases by 1 on every start/stop)
When I load these files into a Sony app like Catalyst Browse, all of the embedded metadata is there for both both formats. Infuse is not reading any of it.

I have read the Metadata 101 guide, but they don't have much info for embedded data as it seems more geared toward Movie/TV. I'm currently investigating what apps/methods I can export metadata with.
 
* Z-Wave Migration from HomeSeer to Home Assistant *

The last piece of the puzzle was finally put into place earlier this morning with the successful migration of all Z-Wave devices from HomeSeer to Home Assistant. This post explains the hurdles and solution.

Background
I've been running a network of Z-Wave devices with HomeSeer for over a decade. The set of devices includes light toggle switches, light dimmer switches, entry door locks, toggle-able AC outlets, and multi-function sensors (motion, illumination, temperature).

The Z-Wave controller has changed over the years, from an old Aeotec Z-Stick to the latest Gen 3 version of HomeSeer Z-NET based on Raspberry Pi.

Some of the devices were added to the Z-Wave network over a decade ago while others were added more recently. Although HomeSeer is a member of the Home Assistant working group and therefore Z-NET is supported by Home Assistant, many Z-Wave devices did not migrate smoothly.

After some experimentation, it turns out that if older devices are first excluded from the Z-Wave network and re-included, they will be included differently, in a more updated manner by the latest drivers/protocols on the Z-NET controller. All devices that have been similarly included will be readily visible to the Z-Wave add-on in Home Assistant.

Configuring Z-Wave in Home Assistant
To use the HomeSeer Z-NET controller, which operates over the network rather than being a locally inserted USB device, we must follow a slightly different procedure.

Instructions are provided here by HomeSeer. Although these instructions leave out a handful of crucial details that this post seeks to remedy, these instructions must still be followed.

We start by installing the Z-Wave JS UI add-on from the Add-Ons section of Home Assistant. But Add-Ons are only available if we install Home Assistant OS or Supervised:

Screenshot 2023-12-13 at 8.29.03 AM.png


I'm running Home Assistant Supervised on an Orange Pi 5 Plus as documented here. When selecting the Add-On from the Add-On Store be careful to select Z-Wave JS UI and not Z-Wave JS.

Screenshot 2024-01-21 at 11.46.59 AM.png


Now we can start the Add-On and select OPEN WEB UI to configure it for accessing our Z-NET controller over the network:

Screenshot 2024-01-21 at 11.49.31 AM.png


In the Web UI we select Settings followed by Z-Wave:

Screenshot 2024-01-21 at 11.51.30 AM.png


In the Z-Wave section we just need to configure 5 parameters:
  1. Serial Port
    • Enter the IP address of your Z-NET
    • You can determine the IP address from your router or by typing this URL in a web browser:
      • find.homeseer.com
    • Then click Z-NET or Z-NET-R2
    • The IP address will now be shown in the URL/address field of the browser
  2. Then we specify 4 authentication keys as follows:
    • S2 Unauthenticated
    • S2 Authenticated
    • S2 Access Control
    • S0 Legacy
These keys are obtained from HomeSeer, from Plugins -> Z-Wave -> Manage Interfaces:

Screenshot 2024-01-21 at 12.00.07 PM.png


HomeSeer will display all 4 authentication keys in the format:
  • AA-BB-CC-DD-EE-FF-GG-HH-II-JJ-KK-LL-MM-NN-OO-PP (32 bytes)
  • There are 16 two-byte values in each authentication key
    • We need to remove the hyphens
    • If one of the two-byte values is 0 to F, it will be displayed without a leading 0, like this:
      • A-BB-CC-D-EE-FF-GG-HH-II-JJ-KK-LL-MM-NN-OO-PP
      • Notice that A and D are single-byte values so we must insert a leading zero, like this:
        • 0A-BB-CC-0D-EE-FF-...
      • Now we have a full 32 byte field (don't forget to remove the hyphens)
Screenshot 2024-01-21 at 11.52.53 AM.png


Now when we go back to Control Panel in the Z-Wave JS Web UI, we should see a list of all Z-Wave devices queried from the Z-NET controller.

The Z-Wave add-on will now probe each Z-Wave device for its (a) Protocol Info and (b) Node Info. This process can take some time. For my case with 32 devices it took about 30 minutes. This allows the Z-Wave add-on to determine the type of each device and what functions it supports. Until this process ends, the Z-Wave devices will not be available to Home Assistant.

When the query -- or interview -- process completes, we will see something like this (notice "Complete" in the "Interview" column:

Screenshot 2024-01-21 at 8.12.03 AM.png
Screenshot 2024-01-21 at 8.12.41 AM.png


For each device there is a Name and Location field. These fields most likely will not transfer automatically so we'll need to go through each one and enter the values. This is easy to do by clicking each row to reveal a properties page in which Name and Location can be typed in.

When we have a bunch of Z-Wave devices, how do we know which ones are which? Fortunately, HomeSeer has a function called Device Information and from there we can Export the device list in CSV format:

Screenshot 2024-01-21 at 12.15.57 PM.png
Screenshot 2024-01-21 at 12.19.26 PM.png


When we open the CSV file in Numbers, we see something like this:

Screenshot 2024-01-21 at 12.22.11 PM.png

The green Node column allows us to cross-reference the nodes between HomeSeer and Home Assistant, and to enter the names and locations of each Z-Wave device.

Notice also the 4 red boxes. These devices do not have Version and Firmware information. These devices were added (i.e., included) to the Z-Wave network a long time ago. As such, these devices will not transfer to Home Assistant unless we first exclude them and then include them again in HomeSeer (not in Home Assistant).

NOTE:
  • Other devices can also fail to migrate; if this happens, just exclude and re-include them in HomeSeer
  • Excluding and Including some devices require the Z-NET controller to be brought close to the device!
  • In my case the Z-NET controller is located in an upstairs server rack and had to be brought down to the first floor to exclude and include a number of devices located on that floor
After excluding and including all problematic devices, I ended up with this:

Screenshot 2024-01-21 at 12.33.00 PM.png

The device on Row 12 does not physically exist and has since been deleted from the controller. Note that all remaining devices now have Version and Firmware information. This allowed the devices to transfer properly to Home Assistant.

Z-Wave JS UI also provides a nice Z-Wave network diagram where (a) blue nodes are reachable in 1 hop, (b) green nodes are reachable in 2 hops, and (c) orange nodes are reachable in 3 hops.
Screenshot 2024-01-21 at 8.16.29 AM.png




IMPORTANT:
  • The Z-NET controller can be connected to only one software client at a time, whether HomeSeer or Home Assistant, but not both at the same time
  • If we want to perform Z-Wave operations with HomeSeer we must (a) disable the Z-Wave JS UI add-on in Home Assistant and (b) enable the Z-NET controller in HomeSeer:
Screenshot 2024-01-21 at 12.41.33 PM.png
Screenshot 2024-01-21 at 12.40.01 PM.png


  • Similarly, if we want to perform Z-Wave operations with Home Assistant we must (b) disable the Z-NET controller in HomeSeer and (b) enable the Z-Wave JS UI add-on in Home Assistant:
Screenshot 2024-01-21 at 12.38.12 PM.png
Screenshot 2024-01-21 at 12.39.23 PM.png
 
Last edited:
Tinkering with Home Assistant's dashboard layout is quite addicting! With built-in and third-party add-ons, there are endless possibilities...

Here are two versions of almost the same entities. The attached mp4 screen recording shows the various colors and animations.

Screenshot 2024-01-23 at 6.19.19 PM.png
Screenshot 2024-01-23 at 6.19.32 PM.png
 

Attachments

  • Screen Recording 2024-01-23 at 6.13.11 PM.mp4.zip
    2 MB · Views: 6
Last edited:
** Question for IT Gurus **

I'm assembling a second TrueNAS system, this time in a 2U rack mount NAS chassis. The chassis supports 8 SATA drive bays and uses a backplane with two SFF-8087 connectors.
  • Chassis comes with a pair of Mini-SAS SFF-8087 to 4x SATA reverse breakout cables
  • To reduce cable clutter, I'm thinking of using a SATA III controller such as the one shown in the diagram below that has two SFF-8087 ports
  • This SATA controller comes with a pair of SFF-8087 to 4x SATA forward breakout cables
Question:
  • Is it possible to connect a pair of SFF-8087 to SFF-8087 cables directly between the backplane and the controller?
    • This is shown by the 4 red arrows
  • Or is it necessary to use the breakout cables with SATA connectors?


Screenshot 2024-01-28 at 5.19.45 AM.png
 
I'm no SAS guru, but the question is easy to answer.
  • Chassis comes with a pair of Mini-SAS SFF-8087 to 4x SATA reverse breakout cables
So that unspecified chassis must have a passive backplane, without any SAS expander; it just wants 8 SAS/SATA lanes, whose inputs are packaged by sets of four in SFF-8087 connectors. The reverse breakout cables (a costly rarity…) are intended to be plugged into the 8 ports of a motherboard.

  • Is it possible to connect a pair of SFF-8087 to SFF-8087 cables directly between the backplane and the controller?
Yes. On the controller side, the connectors are also just packaging four lanes each.

But if this is intended for TrueNAS you'd do yourself a favour by either going for a motherboard with 8 SATA ports (or more…) or by going for a LSI 9200-8i or 9300-8i (if SATA SSDs are considered) instead of the SATA controller. I have no idea whether the Marvell 9215 is supported under ZFS (default assumption is "NOT"), and ZFS stories about misbehaving storage controllers typically end up in total pool loss.
 
Quick search for Marvell 9215.
Answer from a real expert:
The Marvell AHCI controllers are known to be generally flaky, with variations between the specific models that are too numerous to bother keeping track of.

And a user report on what appears to be the same card as yours:
I was running a Marvell 9215 card, this one in particular, and could only connect 4 drives to it. If I had 8 connected it would only see 4 still. I ran my server with it for the last nearly 2 years, with 4 hdd connected to it, and 5 (1 boot drive) connected to the SATA ports on my motherboard.

Today I finally swapped in a 9201-16i card and plugged in my 8 hdds to it, and it worked as it should.
 
I'm no SAS guru, but the question is easy to answer.

So that unspecified chassis must have a passive backplane, without any SAS expander; it just wants 8 SAS/SATA lanes, whose inputs are packaged by sets of four in SFF-8087 connectors.
It does indeed have a passive backplane without a SAS expander. The chassis in question is the RackChoice 8-bay 2U model:
61DsTM2CtdL._AC_SL1500_.jpg


The reverse breakout cables (a costly rarity…) are intended to be plugged into the 8 ports of a motherboard.
I connected those cables directly to two accessible SATA ports on the motherboard, but the drives did not spin up and the board did not see the drives. It's possible that (a) the backplane is faulty or (b) the supplied cables are not reverse breakouts despite the manufacturer's claims.

I've ordered a replacement SFF-8087 reverse breakout cable from Amazon that should be arriving in the next couple of hours. Will see if those do the job.

Yes. On the controller side, the connectors are also just packaging four lanes each.
Good to know. I attached a SFF-8087 to SFF-8087 cable directly between the backplane and the SATA HBA shown in the diagram, but alas (a) drives did not spin up and (b) HBA did not see the drives.

But if this is intended for TrueNAS you'd do yourself a favour by either going for a motherboard with 8 SATA ports (or more…) or by going for a LSI 9200-8i or 9300-8i (if SATA SSDs are considered) instead of the SATA controller. I have no idea whether the Marvell 9215 is supported under ZFS (default assumption is "NOT"), and ZFS stories about misbehaving storage controllers typically end up in total pool loss.
I have had this card in my Amazon shopping cart for the last 4 hours. Seems I'll return the Syba HBA card and order one based on the LSI SAS 2038 (6Gbps) or LSI SAS 3008 (12Gbps).
  • If I decide to reuse a set of existing SATA drives I'll likely choose the cheaper LSI 2038 model
  • If I decide to buy new 12Gb/s SAS drives, I'll of course opt for the latter
First I just need to determine why a set of existing SATA drives are not being recognized (or spinning up). A chat with Amazon tech support resulted in a replacement unit being ordered. It will arrive on Tuesday.
 
Last edited:
** UPDATE **
  • Just replaced the power supply and power cables with a new Lian Li SFX PSU
  • Discovered that the hard drives I was testing are desktop drives rather than NAS drives; they won't power up in this chassis
  • After inserting a couple of 14TB Exos NAS drives and using the supplied SFF-8087 to reverse breakout cable, both drives powered up and appeared in BIOS :)
I am now in the process of canceling the Amazon replacement unit. Will run more tests now.
 
Last edited:
** UPDATE #2 **
  • The Syba HBA is also working when connected directly to backplane with SFF-8087 to SFF-8087 cable!! :)
  • I have 3 spare Exos drives so I'll do a test installation of TrueNAS with this HBA
  • I've ordered a new HBA based on LSI SAS 2308
  • Amazon replacement 2U chassis has been successfully canceled; I am keeping what I have
 
Last edited:
** UPDATE #3 **
  • TrueNAS SCALE has been installed temporarily with three 14TB Seagate Exos Enterprise drives
  • A single SFF-8087 to SFF-8087 cable is connected to the backplane and HBA, significantly reducing cable clutter
  • The Syba HBA with dual Marvel 9215 controllers is currently in use
  • RAIDZ1 pool created (name is Cumulus)
Performance Numbers (3 physical drives):

# dd if=/dev/zero of=./test.dd bs=1024 count=256K
262144+0 records in
262144+0 records out
268435456 bytes (268 MB, 256 MiB) copied, 0.522821 s, 513 MB/s

# dd if=/dev/zero of=./test.dd bs=1024 count=500K
512000+0 records in
512000+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.01254 s, 518 MB/s

# dd if=/dev/zero of=./test.dd bs=1024 count=1M
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.06979 s, 519 MB/s


This system is based on Intel i5-13500 with a TDP of 65W. The nice thing about this processor is the existence of 6 P-cores and 8 E-cores. If I wanted to save power, I could disable 5 of 6 P-cores and leave all E-cores enabled (we cannot disable all 6 P-cores).
Screenshot 2024-01-28 at 2.57.10 PM.png
 
Last edited:
Back
Top