If you run 5-6 VMs on a 4 core system, I'm sure there will be performance degradation. It's like running 8-9 VMs on a single X5690. If running lots of VMs is something that you do normally, than you should obviously go with more cores.
It all really depends on how many VMs you typically run and how many cores you allot to each VM. If running lots of VMs is something you do on a regular basis, Xeons with lots of cores and vSphere/ESXi makes a lot of sense.
For someone looking for FCPX performance, I would suspect that single core performance means more. You also must consider how well the software you use utilizes lots of cores. Here's a link to a video where a 4 core i7-6700K ran circles around a dual Xeon system with 32 cores and 64 threads. (Skip to about 11:50)
When considering the purchase of one of those old Mac Pros, you must think about power consumption and heat. My old Mac Pro with dual X5680s would suck down over 400W under full load. My current i7-8700K runs at about 260W. Also, being that the old Mac Pros were never designed for two 130W CPUs (the highest TDP CPUs Apple shipped were 95W CPUs), fan management must be manually adjusted to keep temps low. Trying to find a balance of quiet and cool was challenging for me. Even when such an equilibrium was achieved, the fans were running harder than what they normally would have. Operating cost is not only higher because of the higher power consumption, my old Mac Pro was so hot that I had to crank my air conditioner to max during summers.
Lastly, remember, the MacPro4,1/5,1 represents
decade old technology. That means no USB3.1, no NVMe support, no SATA3. By the time you compensate for all these shortcomings, you are out of PCI-e slots and your investment has deepened. Plus, Apple could drop macOS update support for such an old system at any moment.
In my opinion, unless you have a very specific need for all those cores, stay away from these old machines. While they were fantastic in their time, they are simply getting too old to invest any money into now.
For comparison sake, here are benchmark results from my i7-8700K @5GHz.