What you are seeing with FCPX 10.4.8 on Catalina 10.15.4 is consistent with what I am seeing.
With
FCPX 10.4.6 on Mojave 10.14.6 it's much better - at least HEVC encode works on RX580.
There's nothing wrong with your config apart from the device properties you set for the IGPU. It is not necessary since you disable it in the BIOS. Even if it is enabled, WEG Kext would setup a headless IGPU for you so no need to use it.
On a real iMacPro1,1 and many of the new Macs where there's no IGPU, they do have the T2 chip (even one's with IGPU have T2 chip).
The T2 chip does MORE than just security. It also does video encode and decode.
So when a real T2 chip is not available, it falls back to using the CPU - which what we are seeing.
For those fortunate folks with i9's with 6, 8, etc cores this eCPU's are much more powerful that you will not notice the load placed on the them while encoding HEVC.
I do not have RX 5700XT so cannot comment how well it works with FCPX and Compressor as far as Decoding and Encoding is concerned. Just be aware of folks here on these forum who claim "RX 5700 XT is fully supported OOB". In general what they really mean is that the dGPU accelerates the macOS GUI and that does not mean it it will necessarily work certain Apps - even Apple's Apps. They need to prove CPU utilisation, dGPU utilisation and of course output quality.
Anyway, just to double check what you are seeing, I thought I would do a quick test with the latest
Catalina 10.15.5.
Unfortunately, Catalina 10.15.5 taken another backward step for FCPX 10.4.8 & RX580. Even as iMacPro1,1.
The Export seems to do nothing but stall at 0% when taking HEVC project and
exporting it as H.264.
This is really bad.
The CPU & RX580 are both idle and the system is running fine - just the export stuck at 0%:
View attachment 472156
That's really weird (that it cannot export to H.264).
Then, taking the same project with HEVC clip and exporting as HEVC maxes out the CPU as it is doing all the work.
RX580 is sitting idle. :
View attachment 472157
Taking an H.264 project and exporting as HEVC again maxes out the CPU (with RX580 doing the some Decoding):
View attachment 472158
Taking the H.264 4K project and exporting as H.264 FHD does use RX580 for the Decoding & Encoding while the CPU is mainly idle:
View attachment 472161
Ignoring FCPX for the moment and using Compressor 4.4.6, results are the same.
Taking a 4k HEVC input and Exporting it as H.264 took no time at all and RX580 was doing most of the work:
View attachment 472165
But taking an H.264 source and transcoding to HEVC, the CPU was (again) maxed out (and had to be cancelled).
At least Compressor was actually do the transcode unlike FCPX which was hung at 0%.
View attachment 472166
So with a system as iMacPro1,1 running Catalina 10.15.5 + RX580 + NO IGPU (i.e disabled):
- FCPX 10.4.8 HEVC project export to H.264 stalls FCPX - I had to cancel the export.
- FCPX 10.4.8 HEVC project export to HEVC, Encode on RX580 not supported - CPU used to max.
- FCPX 10.4.8 H.264 project export to H.264, Decode & Encode on RX580 supported - no CPU usage.
- FCPX 10.4.8 H.264 project export to HEVC, Decodes H.264 but NO Encode on RX580 supported - CPU Maxed.
- Compressor 4.4.6 HEVC to H.264 transcode, Decode & Encode by RX580. A big surprise it works well.
- Compressor 4.4.6 H.264 to HEVC transcode, Decode only on RX580 as CPU maxed out.
- HEVC decode works fine (i.e. playing back HEVC content is silky smooth).
Now switch back to Mojave 10.14.6, iMacPro1,1, IGPU DISABLED, FCPX 10.4.6, Compressor 4.4.5.
This time, same HEVC 4K project exported to H.264 4K, RX580 load was 100% and it was Decoding and Encoding:
View attachment 472194
Then, taking the 4K H.264 project and exporting it as HEVC.
Again, this export, RX580 was maxed out at 100% decoding and encoding. But the CPU load was higher.
Interestingly, the export was much quicker (to HEVC than to H264):
View attachment 472201
Next test was Compressor 4.4.5.
Taking the same 4K HEVC clip and transcoding to H.264 was no problem.
RX580 was maxed out 100% decoding and encoding while the CPU utilisation was low:
View attachment 472205
Next test was a surprise with Compressor.
Took the 4K H264 clip and transcoding to HEVC.
CPU loading was ~ 90% and RX580 load was ~ 3% and it was neither decoding nor encoding:
View attachment 472208
So with a system as iMacPro1,1 running Mojave 10.14.6 + RX580 + NO IGPU (i.e disabled):
- FCPX 10.4.6: H.264 to HEVC Encode is supported and works well on the RX580 (CPU is used too).
- FCPX 10.4.6: HEVC to H.264 decode and encode fine on the RX580 (very little CPU used).
- Compressor 4.4.5: HEVC to H.264 decode & encode fine on the RX580 (very little CPU used).
- Compressor 4.4.5: H.264 to HEVC decode & encode does NOT work on the RX580 (CPU @ 90%)
- HEVC decode works fine (i.e. playing back HEVC content is silky smooth).
So my conclusion is that if your workflow requires HEVC output for FCPX, use Mojave 10.14.6 & FCPX 10.4.6 and Compressor 4.4.5.