Suggestions for updating my aging Dual 4K setup from Nvidia to AMD?

[nightgolfer]

It's better to give too much details than too few, so thanks for that.

Thank you, that makes me feel much better. And thank you for the great reply, super detailed and all the info I could hope for! I'll try and reply as thoroughly as I can:

High resolution, but only 2D work? No 3D, no video? If so there's no need to go for a high-end card such as a RX 6800.
A RX 6600 (PCIe 4.0x8) will work as PCIe 3.0x8 in your Z270 build, with half the bandwidth of a x16 card… but I doubt that 2D work ever needs that much bandwidth.

I do some 3D, but it's Rhino stuff and not much rendering, so that's not going to pull any teeth. I do video editing and rending in AE/Media Encoder, so the boost definitely helps there. Also, the files I work with in PhotoShop are really, really big, sometimes 3 GB+ with thousands of layers. While I'm never quite sure where the bottleneck is for that type of work, I think having a heavy-hitting GPU can't hurt. I didn't know about the PCIe differences, but I think it will be OK, and also future-proofing as much as I can is important. I do want a new MB and CPU at some point, just need the money.

Any working GPU will work with any kind of working CPU or chipset. No issue, and no constraint, here.

Fantastic! This is exactly the type of info I don't have, and stuff that's surprisingly hard to figure out when you're a hardware novice like me. Thank you very much for this.

Anything newer than Polaris will actually require upgrading macOS, which may require updating your Clover or OpenCore EFI. The easiest and safest here may be to get a new drive and make a new install on it. Keep the old drive safe (preferably unplugged), so you can revert to it.

As mentioned, my main goal is updating as far beyond High Sierra as I can for software reasons. I want to make sure that my machine will last as long as it can. I don't know how long hackintoshes will keep being a thing, so I want to make sure that I'm as updated as I can be and then hopefully have the machine for many years. I really don't want to go back to Windows (ever) if it can be avoided, but I can't afford the Apple price tag for the type of machine I need to do my work.

My current plan is just like you described, getting a new NVME M.2 (500 GB this time) drive and swapping out the existing one, start fresh. I'm actually trying to figure out what type to get right now, any suggestions? Dortania talks about an SSD Hall Of Fame, but I can't find it anywhere, only this page referencing it: https://github.com/dortania/bugtracker/issues/192

All should work, including the RX 5600 XT according to Dortania's GPU Buyers Guide.
Polaris cards from Powercolor and some other vendors were advised against, but this may not longer apply to Navi.
A $230 Vega seems too good a deal to be a good deal… I'd be wary of old watercooled gear, especially with rust around bolts (leaks?).
For second-hand, vetting sellers is obviously as important as checking the hardware.
RX 6600 (namely AsRock Challenger non-XT) currently starts at 335 EUR in the Netherlands. At that price, I'd buy new!

You're right, there are actually new cards available, and I didn't know about the non-XT versions. Looks like an RX 6600 will give me pretty much what I already have, which is pretty good: https://gpu.userbenchmark.com/Compare/Nvidia-GTX-1070-vs-AMD-RX-6600/3609vs4128

I will definitely consider settling for RX 6600, thank you for the suggestion!

Navi 10 (RX 5000 series) requires at least Catalina.
Navi 23 (RX 6600) requires the full jump to Monterey.

I'm thinking taking the plunge and going full Monterey. If I swap out the NVME M2 and start clean, the only thing I'll need to change back in case of emergency (if any) would be BIOS settings, right? I like the idea of being able to roll back if I can't get it to work, as I can't spend too many days on upgrading the machine before it becomes a problem.

Thank you so much for your help! I'm feeling a lot better about the whole process now, and I've started reading the Dortania guide and I'm halfway to making a Monterey/Open Core USB stick. I can't seem to get my hands on anything other than 12.5.1, but I guess it's worth the attempt. Right?

 

[nightgolfer]

RX 5600 XT should work but the performance is just slightly better than RX 580. I would recommend RX 6600 XT from the list.

Thank you! @etorix has also suggested a new RX 6600, which I am now considering. The missing XT might not be a huge issue for my working conditions.