I think the situation of which encoder and decoder it uses is more complicated than it seems at first glance.
I'm using 10.14.4 with an iMac 14,2 board id. WEG with no extra boot arguments. Vega 56 and a connectorless intel iGPU enabled (i7-8700k)
First thing: just because VDADecoderChecker says 'successfully connected to Intel plugin, offline GenXX' does NOT mean that's the hardware decoder in use. That message seems to be part of some device enumeration AppleGVA does and doesn't indicate the final device that is chosen.
The only reliable way to see what hardware decoder/encoder it picked is to set these:
defaults write com.apple.AppleGVA gvaDebug -bool YES
defaults write com.apple.AppleGVA enableSyslog -bool YES
Then open console.app and search for 'GVA'. You should see GVA debug messages as you launch apps that use it. You will see one line in particular 'GVA: plugin is ATI' or 'GVA: plugin is Intel'. THAT'S the one it is using.
With no force set, VDADecoderChecker consistently uses the intel plugin for me. Obviously this depends on your SMBIOS settings/and or WEG boot arguments. However, if i set forceATI, it absolutely reports the ATI plugin is used.
My encoder app also consistently picks the intel plugin, unless forceATI is set, and then it picks the 'ATI' one.
However, when using Quicktime player, things get more interesting. I think Apple may have changed some things recently behind the scenes, but again I have no way to reliably test older Mojave versions.
If i load a 4K HEVC movie with Quicktime, it uses the Intel plugin by default. Setting forceATI works, and it picks the ATI plugin.
However, if I open a normal 1080 h264 movie with Quicktime...I get no GVA debug output at all. As if Quicktime isn't using the GVA framework to decode h264. I don't know if that means it is selecting the software decoder for some reason, or if they've written some custom metal based decoder outside of GVA framework. I need to fully disable SIP to attach to Quicktime Player to trace the VideoToolbox calls to see what is going on.
Using a recent version of VLC behaves the same as VDADecoderChecker for both h264 and HEVC files.
The quicktime h264 thing is weird, and requires more investigation.
iMac14,2 has forceOfflineRenderer set to true, so it always defaults to iGPU for both encoder/decoder. When I get a chance to reboot I'll disable forceOfflineRenderer and see what's changed.
One more thing to note: the technique of using Activity Monitor to see what dylibs have been loaded to determine which encoder/decoder is being used is also unreliable. The device enumeration can cause both of them to be loaded. Just because you see an AMD bundle loaded does not mean it is actually being used.