I officially give up!
Yesterday I've installed the "Security Update" from the App-Store.
In parallel I've installed a GT 640.
After the first boot everything seemed to work nicely.
I've installed the GT 640 as a second GPU next to my 660 Ti.
I got the same glitches which I had before but only on the monitor which was connected to the 660 Ti.
So I wanted to switch completely to the 640.
But during the last uptime I've noticed some strange "SSD busy lags" so I decided to remove the 640 and boot the system as it was before. (To exclude any failures caused by the other 640)
The system didn't came back on....
Something about "too many corpses being created"
I think it was caused by the Security Update as I didn't change anything else than the hardware. Or it was just a coincidence.
I've tied to repair the partition but the "stupid" APFS didn't allow it because the recovery partition was in the same container as the system partition. So I created an USB drive with High Sierra on it.
Using the USB drive I could perform the fsck_apfs which returned me an "object map checksum error"
Long story short:
I've used my TimeMachine backup to fully restore my system back to Sierra.
(First install Sierra with USB dirve so that I got rid of the APFS, then used the Restore function from the recovery partition)
As many of you I now will wait until the 660 problem on High Sierra is fixed.
UPDATE:
I found the source of the problem I had:
https://www.tonymacx86.com/threads/...isabling-nvidia-driver-driver-updated.239245/