General NVMe Drive Problems (Fatal)

[powerbookboy]

For what it's worth I have a Sabrent 2TB Rocket which is ~95% full. I have been using this with zero problems for ~3 years in a AsRock Z390 ITX Phantom / 9900K using MacOS 10.15 and more recently 12.6. Since switching this drive into my new AsRock Z690 ITX Phantom / 12900KS I have started to get repeated hard crashes with the dreaded "3rd party NVMe controller. Loss of MMIO space." showing in the crash report. This always occurs when the machine is doing a Read/write heavy workload. Most often it occurs when open large projects in PHPStorm. NVME Fix is installed. Have tried setting long TRIM timeouts, disabling TRIM, but with whatever I try I get random "Loss of MMIO space" errors.

When the Z690 machine is booted into Windows, with the Sabrent drive still attached, the machine is rock solid, which suggests no obvious failing hardware issues. Running from a SATA SSD with MacOS the Z690 build is rock solid.

If I take that "problem" drive out of my Z690 build that's crashed moments ago and put it back in my "solid" Z390 build, it immediately works and is rock solid for days//weeks. So the issue as I see it is more the combination of NVME drive and specific motherboard than a combination of NVME drive and MacOS.

Either that or there is some magical OpenCore patch that I present on my Z390 build that stops it occuring that is not present on the Z690.

I've got a Crucial T500 arriving shortly. Will see how that behaves. If I have problems with that I'll try a WD. But I'm really not convinced that there some magic in the WD drives. I'll guessing that it's more people that are following the advice from the "Hackintosh gods" may also be running from a more narrow set of hardware that's got a more "golden" OpenCore configuration.

Shame that this might push me to buy a Apple Mac. I much prefer to feel like I own my machine, previous Hhacks have been wonderfully reliable; but this is doing my head in...

 

[c-o-pr]

There is no science of compatible NVMe drives and there never has been.

The advice to stick with WD is nothing more than there being no history of pathology, not because of engineering correctness.

NVMeFix has always been explicitly about power management work-arounds not reliability, so it's not a known to be a cure for any integrity issue.

ApfsTrimTimeout was a stopgap for a certain era of macOS (Monterey) APFS space manager pathology of spending a long time waiting for all free space to be trimmed. There's nowhere any documentation as to why there's an incompatibility, nor is there any spec detail pertaining to NVMe pertaining to trim which describes correct implementation: the NVMe spec lays out a command set which the drive / OS makers can interpret as they wish.

Apple has always rolled its own drives and while tolerating 3rd party drives is free to design its own interfaces and semantics.

Many drive makers are integrators of silicon from other sources where they (e.g. Sabrent) have little design control over key controller attributes, and rely on the silicon provider to do all the key compatibility work, which as a rule for commodities PC drives must be about a Windows-centric design view of compatibility.

While Macs have been built from PC compatible parts for 15 years, there's never ever been a promised compatibility with PC. Bootcamp relied on Apple drivers.

So where does this leave you?

The Sabrents are known to be problematic and you are confirming it.

The solution for you is to find a drive that's not known to be incompatible. (The double-negative is important: there are no known compatible drives, because there's no standard by which to judge compatibility)

Based on the experiences of the forums, WD NVMe drives are considered to be the safest bet.

And they are available at such low prices that it should be no big deal to switch.

As to the long-term of hacks, the door is slowly closing, but this is in keeping with the long history of PC trends.

 

[powerbookboy]

The Sabrents are known to be problematic and you are confirming it.

I'm not sure i understand the logic behind this statement? I have been using a Sabrent Rocket 2TB drive for ~3 years across OS 10.15, 12.6 and now 13.6. It's been absolutely 100% rock solid in an ASROCK Z390 ITX. If I switch that exact same drive into my ASROCK Z690 ITX board, using the exact same version of OpenCore but with appropriate customisations for that board it will start to randomly crash with "3rd party NVMe controller. Loss of MMIO space." If I swap it back into my Z390, flip OpenCore, it's ( to date ) rock solid within 5mins.

That testing scenairo excludes a specific version of MacOS having issues communicating with a specific drive/drive controller. The only variable is the motherboard/UEFI settings and the applied OpenCore patches.

I can confirm that the same "3rd party NVMe controller. Loss of MMIO space." issue occurs with the Crucial T500. Same drive, swap the OpenCore, works perfectly in a Z390, random crashes in a Z690

I'm going to get a WD SN750 (not plus ) 500GB for cheap and see what happens with that and will report.

I think this entire discussion should be caveated by saying that it can be proven that their are interactions between specific motherboards and specific SSDs that cause these issues.To date my reading has generally been people giving a more blanket "WD drives won't give you any hassle" type statement with no less emphasis being given to the motherboards that people are using those drives alongside.

Your "not known to be incompatible" would perhaps be better phrased as "has shown the widest range of compatibility across a number of boards at this time"? My Sabrent drive has and can be shown to be perfectly compatible in specific motherboard/Opencore/UEFI combos.

I appreciate it's a perhaps an annoying linguistic distinction, but you can find a lot of posts over the years saying successfully "Samsungs <whatevers> are great", "Sabrent Rockets are great", and now "WD SNxx0 are great". And all those statements might have been true at the time of writing. The problems is Google has a long memory and spews those more absolute statements back at people searching for these sort of problem (possibly years later) by which time we might have found problems with the WD drives...

Meh, I'm nit-picking. Will update this thread once I have data from fitted and thrashed WD..

 

[c-o-pr]

I feel your pain...

Re "loss of MMIO space", I'm ignorant of that syndrome.

My problem with Sabrent was Rocket 4 with Asus z590 (2021) Big Sur and everything seeming to work for a couple weeks, then drive seizing and losing all my data. Then after RMA the replacement drive seemed to work for a couple days, then seized and lost all my data.

Was it the drives? The board? The SW+use-case? With the replacement it looked like a thermal overload where drive temp would spike to 85C before it crashed.

After doing a lot of forum searching I came across warnings that the generation of Phision controller in Rocket 4 model was known to go wonky under macOS. But I never discovered the details as to why.

Re "compatibility" of any PC kit under hack...

We could wake up any day and find a SW or firmware update wrecks a working system.

And with very latest tech, like NVMe SSD it doesn't mean much to say that a device works in one board but not another. Even within PC industry problems of incompatibility due to churn are rampant.

As an aside...

Asus just posted a version 20 (!) of BIOS for z590 Hero XIII that includes warning that thunderbolt 1/2 devices will become incompatible. Presumably this is because of MMIO-space exploit hazards and work-arounds that weren't understood in early Thunderbolt gens have been fixed but at the expense of backwards compat. Is this a feature or a bug?

It's a recent example of the ripple of incompatibility within "supported" PC HW space.

Well, macOS has never been within that realm. And Apple rolls it's own firmware. If you have ever tried putting Apple-branded hard drive in cheap USB enclosure you may have seen that just because an Apple drive is SATA from a major drive maker doesn't imply it will work properly, I've seen this quietly losing my data.

There are too many variables to say anything about hack compatibility except that some configs are known to be workable... So far...

It took me years and the equivalent of 5 Macs i'm hours, but I eventually got my z590 into a sweet spot and use it everyday with excellent performance. It still glitches every other month or so and needs some tending but useful and was fun challenge to build.

Best to you and please do share any further findings.

 

[powerbookboy]

Have now fitted a WD SN750 ( not SE ) 512GB drive into my Z690. Machine was stable for a few days, but now I have once again had a period of repeated crashes with "3rd party NVMe controller. Loss of MMIO space." errors showing on rebooting.

This behaviour is now consistent across a Sabrent Rocket 2TB, a Crucial T500 2TB and a WD SN750 512GB. The issue has been seen on OS 12.6, 13.6 and 14.3. The issue does not occur when those exact same drives are then swapped into my Z390 board, with the exact same data on them, just a swapped OpenCore. I've also tried various combinations of swapped PSU, video card, SSD, RAM, Thunderbolt disabled/enabled, forced PCI-Gen3 in the BIOS, multiple different BIOS revisions.

It seems to occur when the drives get substantially full, but that might just be coincidence that they start doing this when ~90+ full. I've not seen it occur when booted into Windows (however my uptime in there is measured in hours not days, so not directly comparable).

But that's pretty conclusive proof that for this specific error message, WD SSDs are not "magically" compatible in some mysterious manner.

There is either an intermittent issue with this specific ASRock Z690 ITX Phantom Gaming motherboard that is causing NVMe drives to just fall off at random, or there is some more systemic issue affecting Z690 boards that is not present on Z390.

At this point I'm rather stumped as to what I can do? I can to try to gather more information if someone how knnows what they are doing ( like @CaseySJ ) can give guidance on what/where/how to collect.

Beyond that I think it's time to really seriously think of switching full time to Windows.

 

[TurbineSeaplane]

@powerbookboy

If it helps as a data point, I've had a SN750 2TB for a couple years now and have been fighting the "3rd party NVMe controller. Loss of MMIO space." errors off and on the whole way. I've gone up to 6 months with no issues and then suddenly it would come back. Then it would go away.. Then maybe randomly come back.

I've never had this drive more than 65% full ever

It flared up again 2 weeks ago and I said screw it and started fresh on Win 11, popped in an RTX GPU and have been using that until I figure out the plan. I've been looking at preowned Mac Minis and Studios and weighing options. Everything that's ever a good deal is some 8/256 base model thing... and having USB-C/TB cable spaghetti all over doesn't sound fun to me.

All this while, though, I'm finding Win11 to be just fine -- more than that -- "good", even?

I don't know. All of Apple's incessant overcharging and locking everything down may have finally lost them a customer who's been around since the 1980s. (me)

 

[powerbookboy]

@powerbookboy

If it helps as a data point, I've had a SN750 2TB for a couple years now and have been fighting the "3rd party NVMe controller. Loss of MMIO space." errors off and on the whole way. I've gone up to 6 months with no issues and then suddenly it would come back. Then it would go away.. Then maybe randomly come back.

I've never had this drive more than 65% full ever

It flared up again 2 weeks ago and I said screw it and started fresh on Win 11, popped in an RTX GPU and have been using that until I figure out the plan. I've been looking at preowned Mac Minis and Studios and weighing options. Everything that's ever a good deal is some 8/256 base model thing... and having USB-C/TB cable spaghetti all over doesn't sound fun to me.

All this while, though, I'm finding Win11 to be just fine -- more than that -- "good", even?

I don't know. All of Apple's incessant overcharging and locking everything down may have finally lost them a customer who's been around since the 1980s. (me)

Yeah, first Mac was an LC2 back in 1990.

I rather like Windows 11. It's just I've got to port my web developments environment over to Windows/Linux which is somewhat non-trivial to do.

I think I'm going to have to really bite the bullet, spend a few days getting Windows 11 setup for work and see how it behaves. If it remains stable and I can get work done I think it's also time to say goodbye to MacOS.

Oh well, I did at least prove that I can get my stable Z390 configured to run MacOS 13 and 14 without any problems. So not entirely wasted time???

 

[TurbineSeaplane]

Oh well, I did at least prove that I can get my stable Z390 configured to run MacOS 13 and 14 without any problems. So not entirely wasted time???

Not wasted at all!

I've honestly loved my time with Hacks -- over a decade now!

The thought has crossed my mind to go back to a SATA SSD and keep on going, as that seems totally bullet proof (vs NVME issues). I just hate to take that speed hit.

Has that idea crossed your mind also?

 

[trs96]

The thought has crossed my mind to go back to a SATA SSD and keep on going, as that seems totally bullet proof (vs NVME issues). I just hate to take that speed hit.

The SN750 made it's debut in January 2019. It's out of production now, it no longer gets firmware updates. All the WD drives I've bought since 2022, (mainly WD Blue SN570 and WD Black SN770s) I've updated the firmware to the latest. I've ran Big Sur, Monterey and Sonoma on them all and never gotten any MMIO error messages. I've used them in Gigabyte builds, pre-builts, just about every kind of hackintosh there is. They still boot up fast and the speeds have remained constant over 2 years.

So whatever has caused the problems with the SN750 appears to have been fixed with firmware updates on the newer WD models. Because NVMe drives have come down so far in price, it's probably worth giving one a try to see how it goes.

 

[TurbineSeaplane]

The SN750 is a 5 year old model of NVMe. I wouldn't expect it to work flawlessly with a Z690 board hacked to run macOS Monterey or newer. It's an EOL drive that WD replaced with a newer model in 2022. You'd need a WD drive updated with the latest firmware. Try an SN770 or WD SN850X. 1000's of people here use them with their hacks without problems.

Hmm. Interesting

For me, I'll probably just move on as a Hack on macOS is just going to get harder and harder to navigate exactly via OC with more and more distance between Apple releases on Intel hardware

It'd be an all new purchase and build for me coming from z370