The home-brew fusion drive seems do not move data automatically?

[kirbyzhou]

I have created a home-brew fusion drive with 128G+2T, but it seems do not move data automatically.

The following is how I verify that.

# fill the SSD space by zero bytes
dd if=/dev/zero bs=$((1024*1024)) of=~/ttt count=$((200*1024)) 
# create a test file
dd if=/dev/zero bs=$((1024*1024)) of=~/20g count=$((20*1024)) 
# read it frequently, the iostat shows nearly all io happens on disk1 (HDD).
for ((k=1;k<50;++k)); do for ((i=0;i<12000;++i)); do j=$((i*17231%(20*1024))); dd if=20g of=/dev/null bs=$((1024*1024)) count=1 skip=$j; done; done &> /dev/null 
# reboot
reboot
# reread and check iostat, the iostat shows nearly all io happens on disk1 (HDD).
for ((i=0;i<12000;++i)); do j=$((i*17231%(20*1024))); dd if=20g of=/dev/null bs=$((1024*1024)) count=1 skip=$j; done &> /dev/null

finally, reboot and recheck the iostat, I found nearly no IO happens on my SSD.

How can I get the automatically data moving?

 

[thierry37]

Thanks kirbyzhou for creating a separate topic for this question.
Maybe better here, than talking on the "how to create a FD".

I did some test with your commands but it was not so conclusive.

My SSD is full. So, I should have 4GB free by the Fusion Drive. This, I am already not sure it's true.

I created the file and was reading it, repetitively.
When I stop ready, there is still some disk activity.
I was hopping the system is moving the file to the SSD.

After all activity stopped, I read again.
This time, I can see both the SSD and the HDD with activity.
But not the only SSD. Which should be the case.

I continue to read and read.
But I could not make a conclusion that all the file was moved to the SSD...


I wanted to try the test from jollyjinx, but the "mkfile" is giving me an error.
He is created a lots of files, to fill the SSD, then he reads the files on the HDD, many times.
These files should be moved to the SSD.
I did not managed to get the files created...

TIN> cd /Volumes/bla TIN> for d in {00..13} do mkdir $d                    for f in {000..100} mkfile 100m $d/$f done

http://jollyjinx.tumblr.com/post/34638496292/fusion-drive-on-older-macs-yes-since-apple-has

 

[kirbyzhou]

you can take this scriptlet instead of 'mkfile 100m $d/$f'

  dd if=/dev/zero bs=$((1024*1024)) count=100 of=$d$f

From jollyjinx's test case, I found that, all his files are small, but my files are big.
Maybe fusion drive DONOT take affect on big files.
I will retest it later to verify that.

 

[thierry37]

From what I read, it is supposed to work at the level of the blocks of data. Not the files.
So maybe it would not move immediately the whole big file?

ill try your scriptlet. Thanks.

 

[thierry37]

Kirbyzhou, I tried your scriptlet and I get the same kind of error.
"unexpected token"




I have the same syntax error for the "mkfile"

Any idea?




But I can see that my system is making some space on the SSD.
I created a 5GB file, following your script.
It writes on the SSD, then a bit on the HDD.
When finished, it's clear that some files are moved from SSD to HD.
See below the iostat

Disk1 is SSD, disk1 is HDD

When finished, you can see 3-4 lines with almost no i/o, then it starts again. At the transfer rate of my HDD (50-60MB/s)





I create another 5GB file and it is written on the SSD again.

 

[kirbyzhou]

You have missed a 'do' at the end of the line of the second 'for'.

 

[kirbyzhou]

But I can see that my system is making some space on the SSD.
I created a 5GB file, following your script.
It writes on the SSD, then a bit on the HDD.
When finished, it's clear that some files are moved from SSD to HD.
See below the iostat

Disk1 is SSD, disk1 is HDD

When finished, you can see 3-4 lines with almost no i/o, then it starts again. At the transfer rate of my HDD (50-60MB/s)


View attachment 51929


I create another 5GB file and it is written on the SSD again.

Could you please run 'diskutil /dev/disk0; diskutil /dev/disk1;' and post the result here?
Mine is below, I think the 'Internal: No' and 'Ejectable: Yes' may be the reason why my fusion is not real fusion.
But I have already installed IOAHCIBlockStorageInjector.kext, though the Desktop icon is already internal.

kirbyzhou@KirbyPro:~ $ diskutil info /dev/disk0 
   Device Identifier:        disk0
   Device Node:              /dev/disk0
   Part of Whole:            disk0
   Device / Media Name:      RunCore 120G-C SSD Media

   Volume Name:              Not applicable (no file system)

   Mounted:                  Not applicable (no file system)

   File System:              None

   Content (IOContent):      GUID_partition_scheme
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified

   Total Size:               120.0 GB (120034123776 Bytes) (exactly 234441648 512-Byte-Blocks)
   Volume Free Space:        Not applicable (no file system)
   Device Block Size:        512 Bytes

   Read-Only Media:          No
   Read-Only Volume:         Not applicable (no file system)
   Ejectable:                Yes

   Whole:                    Yes
   Internal:                 No
   Solid State:              Yes
   OS 9 Drivers:             No
   Low Level Format:         Not supported

kirbyzhou@KirbyPro:~ $ diskutil info /dev/disk1 
   Device Identifier:        disk1
   Device Node:              /dev/disk1
   Part of Whole:            disk1
   Device / Media Name:      ST2000DM001-1CH164 Media

   Volume Name:              Not applicable (no file system)

   Mounted:                  Not applicable (no file system)

   File System:              None

   Content (IOContent):      GUID_partition_scheme
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified

   Total Size:               2.0 TB (2000398934016 Bytes) (exactly 3907029168 512-Byte-Blocks)
   Volume Free Space:        Not applicable (no file system)
   Device Block Size:        512 Bytes

   Read-Only Media:          No
   Read-Only Volume:         Not applicable (no file system)
   Ejectable:                Yes

   Whole:                    Yes
   Internal:                 No
   Solid State:              No
   OS 9 Drivers:             No
   Low Level Format:         Not supported

kirbyzhou@KirbyPro:~ $

 

[thierry37]

You have missed a 'do' at the end of the line of the second 'for'.

Thanks for the info. I don't know this script language, and I was just copying from the webpage.

I'll have a look at my diskutil tonight and will post the results for you.
Along to some more test, now that you found my error in script.

 

[thierry37]

I have this for the SSD and you were right, we have few differences.

Device Identifier:        disk1
   Device Node:              /dev/disk1
   Part of Whole:            disk1
   Device / Media Name:      SanDisk SDSSDX120GG25 Media

   Volume Name:              Not applicable (no file system)

   Mounted:                  Not applicable (no file system)

   File System:              None

   Content (IOContent):      GUID_partition_scheme
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified

   Total Size:               120.0 GB (120034123776 Bytes) (exactly 234441648 512-Byte-Blocks)
   Volume Free Space:        Not applicable (no file system)
   Device Block Size:        512 Bytes

   Read-Only Media:          No
   Read-Only Volume:         Not applicable (no file system)
   [B]Ejectable:                No[/B]

   Whole:                    Yes
  [B] Internal:                 Yes[/B]
   Solid State:              Yes
   OS 9 Drivers:             No
   Low Level Format:         Not supported

and for the HDD

Device Identifier:        disk2
   Device Node:              /dev/disk2
   Part of Whole:            disk2
   Device / Media Name:      SAMSUNG HD502IJ Media

   Volume Name:              Not applicable (no file system)

   Mounted:                  Not applicable (no file system)

   File System:              None

   Content (IOContent):      GUID_partition_scheme
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified

   Total Size:               500.1 GB (500107862016 Bytes) (exactly 976773168 512-Byte-Blocks)
   Volume Free Space:        Not applicable (no file system)
   Device Block Size:        512 Bytes

   Read-Only Media:          No
   Read-Only Volume:         Not applicable (no file system)
   [B]Ejectable:                No[/B]

   Whole:                    Yes
 [B]  Internal:                 Yes[/B]
   Solid State:              No
   OS 9 Drivers:             No
   Low Level Format:         Not supported

Edit: and you were right for the missing "do"
The test from jollyjinx is working now.
I typed

 for d in {00..20}; do mkdir $d; for f in {00..10}; do mkfile 100m $d/$f; done;  done

and for the repetitive reading, I typed

while `true`; do for d in {10..17}; do for i in $d/*; do echo $i; dd if=$i of=/dev/zero bs=1m; done; done; done

Edit 2:
I finally go this test going on, the right way.
Creating folders 0 to 17, with 10 files in each (100MB files)
It is writing on the SSD first, then it starts on the HDD.
When I stop the process, the system is moving files from the SSD to the HDD, to get the 4GB of free space.
I let it finish, then I start reading all the files in the folders 10 to 17, which were written on the HDD.
It is reading from the HDD. I leave it for 10 min, then when I stop, again the system seems to move files. (both SSD and HDD are working)
I launch again the reading of the folders 10 to 17 and it's really fast. All is on the SSD.

For now, I am completely satisfied. It seems to work correctly, with this test.
Let see with real life usage, after few weeks, to see if the files I am using the most, are really moved to the SSD.

 

[kirbyzhou]

After deleting the 'IOAHCIBlockStorageInjector.kext', my drive becomes internal, but fusion still not 'fusion'.
Maybe I should try recreate the fusion drive on the 'internal disk' env.