I finally got this going. I could have sworn i tried this before and it did not work, but i suspect part of the issue was i was using the SwitchResX trial. I had read about issues with certain features in trial (or "found" serial) modes, I decided to buy it (its only $16) and it ended up being the best $16 i've spent in a while.
First i installed it on my real mac laptop, which as i reported before, was working correctly. I then used switchresX to "view" the exact settings that the laptop was using natively to run the display properly at the scaled resolution i wanted. I was then able to make an exact replica of those settings as a custom resolution on my hackintosh. Apply and viola- i almost could not believe my eyes. I now have both of these monitors running side by side in glorious scaled (5120x2160-scaled to 6720x2834- "UI Looks Like" 3360x1417) HiDPI mode off my Radeon VII. As an added bonus, somehow through this process the system preferences display settings now show the proper settings menu with scaled options for "larger text" and "more space" as it should for a HiDPI capable monitor- instead of just a list of resolutions like it did before. I can also choose 3840x1620, 5120x2160, 2560x1080 and 2048x864 using this menu. This is all working perfectly now!
g\
