I got it to work by flashing the BIOS, and since 2 months ago I'm using macOS as my main workstation using DVI at even 144Hz without any issues. Here's the details:
First let's make sure if you have the same video card.
My original video card:
ASUS GTX 750 Ti OC 2GB
Thanks, I have the same card as you (ASUS GTX750TI-OC-2GD5), and I followed your steps successfully, the DVI port works (just the bottom port btw, the top one doesn't), but sadly with that BIOS I can't set 144 Hz on my monitor (anything above 60 Hz got artifacts), so I had to go back to my...
I had a successful install of macOS Sierra 10.12.6 (16G29), but after installing the lastest NVidia Web Drivers (WebDriver-378.05.05.25f01.pkg) I got No Signal Screen from HDMI or DVI, only VGA works.
I've tried AGDPFix and lastest version of Lilu and NvidiaGraphicsFixup without results.