Hello everyone,
I wanted to update you about my random system freezes/crashes, which I mentioned on this thread several times such as in #3293 or #3447.
First of all, I would like to clarify that my random freezes and crashes were caused by three major factors:
- [Freezes after 5-15h] Asus BIOS for the Rampage VI Extreme motherboard that is buggy and/or unstable with XMP and C-States both enabled
- [Crashes/Reboots] Wrong audio kexts for my config (VoodooHDA doesn't work properly on my R6E motherboard)
- [Random Reboots] Unstable overclocking (Voltage was too low and AVX offsets way too low for my config)
I would also like to insist that the above statements are true for my configuration only, and are not due to mistakes in @kgp's guide which remains valid and flawless for his configuration (especially since we do not have the same motherboard).
I would also like to mention that none of this would have happened if I had left my configuration on stock settings, non-tweaked and non-overclocked (meaning no "sync all cores" in the BIOS settings, nor other Asus boosts that push the CPU out of its designed boundaries, which is something clarified in @interferenc's post: #3476). Also, stability issues could have been prevented if I had followed a strict and proper Overclocking methodology to bring my setup stable and working for my own Voltage/AVX Offsets/Core Multiplier settings.
After spending one week debugging the issue and trying to understand where it was coming from, I finally have the main answers:
Regarding the freezes issue that was preventing me from having more than 24h of uptime on MacOS was in fact happening on Windows as well as I later discovered...
The freezing issue was BIOS related and probably only related to my type of RAM and motherboard. I have raised the issue on Asus Forum and detailed how I managed to isolate the BIOS settings that were causing it. You can read more about it there: https://rog.asus.com/forum/showthre...er-hours-with-BIOS-1004&viewfull=1#post699861
In short, the freezing issue was due to an incompatibility (I would like to insist that this has been verified for my config only) when both "XMP" and "Autonomous Core C-State" are enabled. I am awaiting for an answer from Asus about what clearly appears to be a bug on their Rampage VI Extreme motherboard, and, in the meantime, I will be keeping XMP enabled but will disable the option "Autonomous Core C-State" (or leaving it on "Auto" as it seems not to cause issues when set this way).
The freezing issue was due to a bad PCI-E card that was plugged to the motherboard. After removing it the issue was gone. You can read more about it here:
https://rog.asus.com/forum/showthre...reezes-when-overclocked&viewfull=1#post700794
Regarding the random reboots, they were due to a bad overclocking. After intensively testing the BIOS settings that I had initially set based on
@kgp's guide I later discovered that my Rampage VI Extreme with i9-7980XE cannot sustain the Voltage and Offset settings mentioned on his guide. Again, this guide was made for
@kgp's own configuration, and the mistake was mine and mine only!
While I was trying to understand where this issue was coming from, I initially thought it was the audio kext VoodooHDA causing instability to my config. This kext was indeed buggy: for instance when I was running the
Valley.app benchmark with sound enabled the application would crash after a few seconds (which is something that was not happening when the sound setting was disabled). I also noticed the same audio issues happening randomly when playing music with iTunes but the full system would freeze this time. Error logs about VoodooHDA crashing were displayed in the Console when audio issues were happening, which is why I concluded that the culprit was VoodooHDA... After removing this VoodooHDA kext and switching to the
@toleda solution which patches AppleHDA.kext, audio became more stable (cf
#3387 for the Rog Rampage VI Extreme motherboard)...
I would also like to mention that the clues about the bad overclocking were already there on this topic: @kgp (maybe others too) repeatedly mentioned that AVX offsets correctly setup was what resolved stability issues, cf #3055. However, something that should have been mentioned in red is that the AVX offset values he suggests are for his own CPU/Motherboard/Config, and are likely to be different for other people's configuration! Later, @interferenc also gave additional clues in his post #3476 that the core issue of these freezes is most likely to be Voltage-related, and indeed it was!
So, what was the real issue about the random reboots about (aside from the XMP freeze issue)?
I believe it was a bit of a combination between AVX offset settings that were not appropriate for the Core Voltage that I had set and the XPM/C-State issue mentioned about the freezes. Regarding the AVX settings, as I understand it, AVX instructions require much more voltage than normal CPU instructions. However, AVX instructions are not that commonly encountered on an Operating System and therefore, on a daily usage bad AVX offset settings "might not" lead to crashes... until they do.
My configuration was (what I thought at the time) stable on Windows with everything set to Auto with "Sync All Cores" in the BIOS regarding CPU settings and using the AVX offset values suggested on this guide. But I hadn't followed a rigorous overclock stability testing approach, and because the CPU temperatures on stress load were comfortable I thought everything was stable and I was wrong.
What I understood later is that although the overclocking configuration appeared to be stable on Windows, it was not stable at all on MacOS. I believe the reason behind this is that the tasks I perform on MacOS are more representative of an intensive daily use than under Windows and thus the CPU is more likely to receive instructions that trigger its instability and result in freezes/reboots: such as playing audio with iTunes while having VMWare Fusion running a Virtual Machine, for example.
Also, I would like to mention that good temperatures are NOT an indicator of stability. And having no crashes in intensive workloads is also NOT the ultimate indicator or stability (although it's a good one). My CPU freezes were happening when the CPU was not under load, but when multitasking (i.e. playing audio while running a GPU benchmark).
So, how do I know if my AVX offset settings are good?
The thing with AVX instructions is that they are not found in all processes and applications running on your machine. So, you can just have your setup running very fine on non-AVX instructions with good temperature for several hours/days until there are specific tasks that use AVX and that cause freezes. When AVX instructions hit the CPU there is a sudden power burst because they require more energy than other types of CPU instructions, hence the CPU freeze if Voltage/AVX Offset is not well calibrated.
The killer stability test for me to verify that my system was rock stable was to play audio on MacOS while stressing the CPU with AVX instructions. If confirmed by other members here to be a good stability test I would suggest becoming a standard for anyone following
@kgp's guide and attempting overclocking (again, overclock just means running the CPU outside its designed boundaries, cf
#3476).
A note of caution before the next notes: THIS IS NOT a professional stress testing methodology, but my own methodology to find sustainable CPU Core Voltage and AVX settings. Use it at your own risks, I will not be responsible if it causes any damages to your hardware. Make sure to backup your OS before proceeding as you will face a lot of crashes and freezes during the procedure.
- Download Prime95 for Mac OS X p95v294b7.MacOSX.zip (you can also get the CLI version): https://www.mersenne.org/download/
- Open the application (press cancel on joining the network)
- Go to Options -> Torture Test -> Select Blend -> hit the Run button
- While this is running, make sure to monitor the temperatures with Intel Power Gadget
- While the test is running, play an MP3 song of your choosing with iTunes or by pressing the space bar directly on the song from Finder (in other words, in Quick Look)
How to interpret the results:
- If your Voltage settings are stable, then the song will play nicely. If not, you will get an instant CPU freeze. In which case, either you increase the AVX offsets or you increase the CPU Voltage (I would rather choose to increase the AVX Offsets)
- If your temperatures are way too high (i.e. hitting 94°C) you need to increase the AVX Offsets.
My results were the following:
When AVX offsets were set to anything lower than 4 (both AVX and AVX-512), I would get an instant freeze. When setting the AVX offsets to 5 (and above), the music would play nicely. Which is why I believe that this test should be standard as it instantaneously tells you if your AVX configuration is stable or not without the need to go through hours of CPU testing and benchmarking. However, it is MANDATORY to follow a proper overclock methodology to verify the stability of your system.
I hope my notes above can help those who have similar stability issues after tweaking their BIOS settings and playing with overclocking.
Edit: A last note about overclocking... Don't mix benchmarking and stability testing!
https://browser.geekbench.com/v4/cpu/6510963 (unstable)
