Z690 Chipset Motherboards and Alder Lake CPU

[CaseySJ]

It would be good to dump x86_validate_tolopogy() to see how macOS probes the cpuid or logical/physical processors... to devise a workaround. And even if we workaround it, it's not clear how the thread scheduler will schedule threads on the Golden Cove cores vs the Gracemont cores.

One step at a time!

@Elias64Fr

Thanks to Apple's Open Source commitment, here is the source code for x86_validate_topology() from macOS 11.5 (Big Sur). After examining this, we may be able to patch it, but this is not always practical. This particular code is located here:

xnu-7195.141.2 --> osfmk --> i386 --> cpu_threads.c

First, we see that x86_validate_topology() says: @cpu_threads.c:1043

This means line 1043, which is this:

In this example we see that nCPUs is 12 and real_ncpus is 20. Although we could patch line 1041 from this:

    if (nCPUs != real_ncpus) {

to this:

    if (nCPUs > real_ncpus) {

... the question is whether macOS will still only activate nCPUs instead of real_ncpus. We could also replace the first instruction in the function with return, which means we bypass topology check completely.

And here is the full function:

/*
 * Validates that the topology was built correctly.  Must be called only
 * after the complete topology is built and no other changes are being made.
 */
void
x86_validate_topology(void)
{
    x86_pkg_t           *pkg;
    x86_die_t           *die;
    x86_core_t          *core;
    x86_lcpu_t          *lcpu;
    uint32_t            nDies;
    uint32_t            nCores;
    uint32_t            nCPUs;

    if (topo_dbg) {
        debug_topology_print();
    }

    /*
     * Called after processors are registered but before non-boot processors
     * are started:
     *  - real_ncpus: number of registered processors driven from MADT
     *  - max_ncpus:  max number of processors that will be started
     */
    nCPUs = topoParms.nPackages * topoParms.nLThreadsPerPackage;
    if (nCPUs != real_ncpus) {
        panic("x86_validate_topology() %d threads but %d registered from MADT",
            nCPUs, real_ncpus);
    }

    pkg = x86_pkgs;
    while (pkg != NULL) {
        /*
         * Make sure that the package has the correct number of dies.
         */
        nDies = 0;
        die = pkg->dies;
        while (die != NULL) {
            if (die->package == NULL) {
                panic("Die(%d)->package is NULL",
                    die->pdie_num);
            }
            if (die->package != pkg) {
                panic("Die %d points to package %d, should be %d",
                    die->pdie_num, die->package->lpkg_num, pkg->lpkg_num);
            }

            TOPO_DBG("Die(%d)->package %d\n",
                die->pdie_num, pkg->lpkg_num);

            /*
             * Make sure that the die has the correct number of cores.
             */
            TOPO_DBG("Die(%d)->cores: ", die->pdie_num);
            nCores = 0;
            core = die->cores;
            while (core != NULL) {
                if (core->die == NULL) {
                    panic("Core(%d)->die is NULL",
                        core->pcore_num);
                }
                if (core->die != die) {
                    panic("Core %d points to die %d, should be %d",
                        core->pcore_num, core->die->pdie_num, die->pdie_num);
                }
                nCores += 1;
                TOPO_DBG("%d ", core->pcore_num);
                core = core->next_in_die;
            }
            TOPO_DBG("\n");

            if (nCores != topoParms.nLCoresPerDie) {
                panic("Should have %d Cores, but only found %d for Die %d",
                    topoParms.nLCoresPerDie, nCores, die->pdie_num);
            }

            /*
             * Make sure that the die has the correct number of CPUs.
             */
            TOPO_DBG("Die(%d)->lcpus: ", die->pdie_num);
            nCPUs = 0;
            lcpu = die->lcpus;
            while (lcpu != NULL) {
                if (lcpu->die == NULL) {
                    panic("CPU(%d)->die is NULL",
                        lcpu->cpu_num);
                }
                if (lcpu->die != die) {
                    panic("CPU %d points to die %d, should be %d",
                        lcpu->cpu_num, lcpu->die->pdie_num, die->pdie_num);
                }
                nCPUs += 1;
                TOPO_DBG("%d ", lcpu->cpu_num);
                lcpu = lcpu->next_in_die;
            }
            TOPO_DBG("\n");

            if (nCPUs != topoParms.nLThreadsPerDie) {
                panic("Should have %d Threads, but only found %d for Die %d",
                    topoParms.nLThreadsPerDie, nCPUs, die->pdie_num);
            }

            nDies += 1;
            die = die->next_in_pkg;
        }

        if (nDies != topoParms.nLDiesPerPackage) {
            panic("Should have %d Dies, but only found %d for package %d",
                topoParms.nLDiesPerPackage, nDies, pkg->lpkg_num);
        }

        /*
         * Make sure that the package has the correct number of cores.
         */
        nCores = 0;
        core = pkg->cores;
        while (core != NULL) {
            if (core->package == NULL) {
                panic("Core(%d)->package is NULL",
                    core->pcore_num);
            }
            if (core->package != pkg) {
                panic("Core %d points to package %d, should be %d",
                    core->pcore_num, core->package->lpkg_num, pkg->lpkg_num);
            }
            TOPO_DBG("Core(%d)->package %d\n",
                core->pcore_num, pkg->lpkg_num);

            /*
             * Make sure that the core has the correct number of CPUs.
             */
            nCPUs = 0;
            lcpu = core->lcpus;
            TOPO_DBG("Core(%d)->lcpus: ", core->pcore_num);
            while (lcpu != NULL) {
                if (lcpu->core == NULL) {
                    panic("CPU(%d)->core is NULL",
                        lcpu->cpu_num);
                }
                if (lcpu->core != core) {
                    panic("CPU %d points to core %d, should be %d",
                        lcpu->cpu_num, lcpu->core->pcore_num, core->pcore_num);
                }
                TOPO_DBG("%d ", lcpu->cpu_num);
                nCPUs += 1;
                lcpu = lcpu->next_in_core;
            }
            TOPO_DBG("\n");

            if (nCPUs != topoParms.nLThreadsPerCore) {
                panic("Should have %d Threads, but only found %d for Core %d",
                    topoParms.nLThreadsPerCore, nCPUs, core->pcore_num);
            }
            nCores += 1;
            core = core->next_in_pkg;
        }

        if (nCores != topoParms.nLCoresPerPackage) {
            panic("Should have %d Cores, but only found %d for package %d",
                topoParms.nLCoresPerPackage, nCores, pkg->lpkg_num);
        }

        /*
         * Make sure that the package has the correct number of CPUs.
         */
        nCPUs = 0;
        lcpu = pkg->lcpus;
        while (lcpu != NULL) {
            if (lcpu->package == NULL) {
                panic("CPU(%d)->package is NULL",
                    lcpu->cpu_num);
            }
            if (lcpu->package != pkg) {
                panic("CPU %d points to package %d, should be %d",
                    lcpu->cpu_num, lcpu->package->lpkg_num, pkg->lpkg_num);
            }
            TOPO_DBG("CPU(%d)->package %d\n",
                lcpu->cpu_num, pkg->lpkg_num);
            nCPUs += 1;
            lcpu = lcpu->next_in_pkg;
        }

        if (nCPUs != topoParms.nLThreadsPerPackage) {
            panic("Should have %d Threads, but only found %d for package %d",
                topoParms.nLThreadsPerPackage, nCPUs, pkg->lpkg_num);
        }

        pkg = pkg->next;
    }
}

 

[Elias64Fr]

I'm not exactly sure what you mean, but I'm happy to attach the file for your magic hands!

I can use either of the two files attached, whichever is easier.

Here are two files SSD-CPUR-Z690-fakeX that should be loaded at same time to related APIC-1-P-E-HT-fakeX.

 

[Elias64Fr]

@Elias64Fr

Thanks to Apple's Open Source commitment, here is the source code for x86_validate_topology() from macOS 11.5 (Big Sur). After examining this, we may be able to patch it, but this is not always practical. This particular code is located here:

xnu-7195.141.2 --> osfmk --> i386 --> cpu_threads.c

First, we see that x86_validate_topology() says: @cpu_threads.c:1043

View attachment 534677

This means line 1043, which is this:
View attachment 534676
In this example we see that nCPUs is 12 and real_ncpus is 20. Although we could patch line 1041 from this:

    if (nCPUs != real_ncpus) {

to this:

    if (nCPUs > real_ncpus) {

... the question is whether macOS will still only activate nCPUs instead of real_ncpus. We could also replace the first instruction in the function with return, which means we bypass topology check completely.

And here is the full function:

/*
 * Validates that the topology was built correctly.  Must be called only
 * after the complete topology is built and no other changes are being made.
 */
void
x86_validate_topology(void)
{
    x86_pkg_t           *pkg;
    x86_die_t           *die;
    x86_core_t          *core;
    x86_lcpu_t          *lcpu;
    uint32_t            nDies;
    uint32_t            nCores;
    uint32_t            nCPUs;

    if (topo_dbg) {
        debug_topology_print();
    }

    /*
     * Called after processors are registered but before non-boot processors
     * are started:
     *  - real_ncpus: number of registered processors driven from MADT
     *  - max_ncpus:  max number of processors that will be started
     */
    nCPUs = topoParms.nPackages * topoParms.nLThreadsPerPackage;
    if (nCPUs != real_ncpus) {
        panic("x86_validate_topology() %d threads but %d registered from MADT",
            nCPUs, real_ncpus);
    }

    pkg = x86_pkgs;
    while (pkg != NULL) {
        /*
         * Make sure that the package has the correct number of dies.
         */
        nDies = 0;
        die = pkg->dies;
        while (die != NULL) {
            if (die->package == NULL) {
                panic("Die(%d)->package is NULL",
                    die->pdie_num);
            }
            if (die->package != pkg) {
                panic("Die %d points to package %d, should be %d",
                    die->pdie_num, die->package->lpkg_num, pkg->lpkg_num);
            }

            TOPO_DBG("Die(%d)->package %d\n",
                die->pdie_num, pkg->lpkg_num);

            /*
             * Make sure that the die has the correct number of cores.
             */
            TOPO_DBG("Die(%d)->cores: ", die->pdie_num);
            nCores = 0;
            core = die->cores;
            while (core != NULL) {
                if (core->die == NULL) {
                    panic("Core(%d)->die is NULL",
                        core->pcore_num);
                }
                if (core->die != die) {
                    panic("Core %d points to die %d, should be %d",
                        core->pcore_num, core->die->pdie_num, die->pdie_num);
                }
                nCores += 1;
                TOPO_DBG("%d ", core->pcore_num);
                core = core->next_in_die;
            }
            TOPO_DBG("\n");

            if (nCores != topoParms.nLCoresPerDie) {
                panic("Should have %d Cores, but only found %d for Die %d",
                    topoParms.nLCoresPerDie, nCores, die->pdie_num);
            }

            /*
             * Make sure that the die has the correct number of CPUs.
             */
            TOPO_DBG("Die(%d)->lcpus: ", die->pdie_num);
            nCPUs = 0;
            lcpu = die->lcpus;
            while (lcpu != NULL) {
                if (lcpu->die == NULL) {
                    panic("CPU(%d)->die is NULL",
                        lcpu->cpu_num);
                }
                if (lcpu->die != die) {
                    panic("CPU %d points to die %d, should be %d",
                        lcpu->cpu_num, lcpu->die->pdie_num, die->pdie_num);
                }
                nCPUs += 1;
                TOPO_DBG("%d ", lcpu->cpu_num);
                lcpu = lcpu->next_in_die;
            }
            TOPO_DBG("\n");

            if (nCPUs != topoParms.nLThreadsPerDie) {
                panic("Should have %d Threads, but only found %d for Die %d",
                    topoParms.nLThreadsPerDie, nCPUs, die->pdie_num);
            }

            nDies += 1;
            die = die->next_in_pkg;
        }

        if (nDies != topoParms.nLDiesPerPackage) {
            panic("Should have %d Dies, but only found %d for package %d",
                topoParms.nLDiesPerPackage, nDies, pkg->lpkg_num);
        }

        /*
         * Make sure that the package has the correct number of cores.
         */
        nCores = 0;
        core = pkg->cores;
        while (core != NULL) {
            if (core->package == NULL) {
                panic("Core(%d)->package is NULL",
                    core->pcore_num);
            }
            if (core->package != pkg) {
                panic("Core %d points to package %d, should be %d",
                    core->pcore_num, core->package->lpkg_num, pkg->lpkg_num);
            }
            TOPO_DBG("Core(%d)->package %d\n",
                core->pcore_num, pkg->lpkg_num);

            /*
             * Make sure that the core has the correct number of CPUs.
             */
            nCPUs = 0;
            lcpu = core->lcpus;
            TOPO_DBG("Core(%d)->lcpus: ", core->pcore_num);
            while (lcpu != NULL) {
                if (lcpu->core == NULL) {
                    panic("CPU(%d)->core is NULL",
                        lcpu->cpu_num);
                }
                if (lcpu->core != core) {
                    panic("CPU %d points to core %d, should be %d",
                        lcpu->cpu_num, lcpu->core->pcore_num, core->pcore_num);
                }
                TOPO_DBG("%d ", lcpu->cpu_num);
                nCPUs += 1;
                lcpu = lcpu->next_in_core;
            }
            TOPO_DBG("\n");

            if (nCPUs != topoParms.nLThreadsPerCore) {
                panic("Should have %d Threads, but only found %d for Core %d",
                    topoParms.nLThreadsPerCore, nCPUs, core->pcore_num);
            }
            nCores += 1;
            core = core->next_in_pkg;
        }

        if (nCores != topoParms.nLCoresPerPackage) {
            panic("Should have %d Cores, but only found %d for package %d",
                topoParms.nLCoresPerPackage, nCores, pkg->lpkg_num);
        }

        /*
         * Make sure that the package has the correct number of CPUs.
         */
        nCPUs = 0;
        lcpu = pkg->lcpus;
        while (lcpu != NULL) {
            if (lcpu->package == NULL) {
                panic("CPU(%d)->package is NULL",
                    lcpu->cpu_num);
            }
            if (lcpu->package != pkg) {
                panic("CPU %d points to package %d, should be %d",
                    lcpu->cpu_num, lcpu->package->lpkg_num, pkg->lpkg_num);
            }
            TOPO_DBG("CPU(%d)->package %d\n",
                lcpu->cpu_num, pkg->lpkg_num);
            nCPUs += 1;
            lcpu = lcpu->next_in_pkg;
        }

        if (nCPUs != topoParms.nLThreadsPerPackage) {
            panic("Should have %d Threads, but only found %d for package %d",
                topoParms.nLThreadsPerPackage, nCPUs, pkg->lpkg_num);
        }

        pkg = pkg->next;
    }
}

interesting source code with file named cpu_xxx.c and cpu_xxx.h on Apple's Open Source.

 

[CaseySJ]

Here are two files SSD-CPUR-Z690-fakeX that should be loaded at same time to related APIC-1-P-E-HT-fakeX.

Will try these files soon.

 

[rushstrike]

It would be good to dump x86_validate_tolopogy() to see how macOS probes the cpuid or logical/physical processors... to devise a workaround. And even if we workaround it, it's not clear how the thread scheduler will schedule threads on the Golden Cove cores vs the Gracemont cores.

One step at a time!

Build OpenCore from my fork: https://github.com/TaiPhamD/OpenCorePkg/commit/83727c5bccf6fdfe13d807c592914f6fd0fab848

This fork requires you to enable the providecurrentcpuinfo quirk . But I basically modified that quirk to just by pass x86 validate and commented out adjusting CPU fsb, etc .,since that seems to screw up somewhere else.

I tried running this patch, and it lets you pass topology validation. But you may run into other errors like I did where my system froze on init fletcher64 avx2.

 

[AdamJ]

Dear @CaseySJ,
I think simple return from x86_validate_topology(void) is not a good strategy. But there will be no harm from trying, the code does not changes any global variables, it only checks package, die, core). The data structure does not take into account the possibility of different number of threads per packages. Ultimate solution may require the change of that data structures to reflect that possibility.

But I think @Elias64Fr solution may be better, ie. to populate the data structures at OC level to trick Mac OS.

Best

 

[CaseySJ]

Dear @CaseySJ,
I think simple return from x86_validate_topology(void) is not a good strategy. But there will be no harm from trying, the code does not changes any global variables, it only checks package, die, core). The data structure does not take into account the possibility of different number of threads per packages. Ultimate solution may require the change of that data structures to reflect that possibility.

But I think @Elias64Fr solution may be better, ie. to populate the data structures at OC level to trick Mac OS.

Best

Yes, if we can trick macOS rather than patch macOS, that is the preferred solution. I'm about to try the new SSDTs by @Elias64Fr, but I suspect we may have a more fundamental problem based on this line:

    nCPUs = topoParms.nPackages * topoParms.nLThreadsPerPackage;

... which immediately precedes the topology check:

    if (nCPUs != real_ncpus) {
        panic("x86_validate_topology() %d threads but %d registered from MADT",
            nCPUs, real_ncpus);
    }

The first line is computing the total number of detected threads (nCPUs) by doing this:

• Multiplying nPackages by nLThreadsPerPackage.
• Each "package" is a physical chip package.
• nLThreadsPerPackage is actually defined as follows (line 228):

    topoParms.nLThreadsPerPackage = topoParms.nLThreadsPerCore * topoParms.nLCoresPerPackage;

• So nLThreadsPerPackage is the total number of threads in one physical chip package.
• When Hyper Threading is enabled, nLThreadsPerCore will be 2.
• This line also shows us that macOS assumes all cores in a package have the same number of threads because there is only one nLThreadsPerCore.

Even if we use Elias' SSDT, we may need to change the value of topoParms.nLThreadsPerPackage. Why? Because:

• The i5-12600K has 10 physical cores and 6 hyper threads for a total of 16 threads.
  • The package count is 1.
  • nLCoresPerPackage is 10 (6 P-cores and 4 E-cores).
  • nLThreadsPerCore is where the problem occurs!!
• If we enable E-Cores, nLThreadsPerCore is set to 1 regardless of hyper threading!
• This is why we always see 10 threads but X registered from MADT.

One solution would be:

• Patching the kernel to set nLThreadsPerCore to 2.
• Because this will result in 20 threads, we will need to use Elias' modified MADT (APIC) to define 20 processors.

 

[CaseySJ]

Here are two files SSD-CPUR-Z690-fakeX that should be loaded at same time to related APIC-1-P-E-HT-fakeX.

Just completed both tests:

• BIOS:
  
  • P-cores = 6
  • E-cores = 4
  • HT = enabled
• Test 1:
  
  • APIC --> fake1 with SSDT-CPUR --> fake1 (and original APIC table dropped)
  • Result: x86_validate_topology() 10 threads but 20 registered from MADT

• Test 2:
  
  • APIC --> fake1 with SSDT-CPUR --> fake1 (and original APIC table dropped)
  • Result: x86_validate_topology() 10 threads but 20 registered from MADT

 

[dehjomz]

The solution would be to set nLThreadsPerPackage to 2 when E-cores are enabled.

I wonder why the logic sets nLThreadsPerPackage to 1 (even when Hyperthreading) is enabled. Sounds like when macOS encounters a 'package' with just one physical thread but no hyper threads (i.e., a Gracemont core), it sets the value of nLThreadsPerPackage to 1, instead of 2. Clearly it wasn't designed for a hybrid x86 topology in which packages don't necessarily have the same number of threads. The P-package has 2 threads, the E-package has 1. But since Apple hasn't released any Macs based on Alder Lake, the old code remains...

Another problem we face is we don't know if any of the functions called by x86_validate_topology() do any work in the background, and set any kernel parameters or other values.

Perhaps if the eventual solution is to indeed patch macOS, then we could set the value of nLThreadsPerPackage to the proper value, and let the function operate normally, rather than just going with an immediate return. But given that E-cores only have 1 thread per package, it's unclear what will happen by setting nLThreadsPerPackage to '2'. Perhaps nothing will happen and things will be fine.

The plot thickens.

 

[etorix]

And if we modify SSDT-CPUR-Z690 to redirect faked core on related real core (for exemple Proc Id 0x0D to 0x0C, 0x0F to 0x0E, …). I mean two CPxx have same PRxx

This is turning from Mad Scientist to pure Evil Genius…
But wouldn't that result in OS X eventually scheduling two threads to one E-core?

Anyway, I'm out of my league at this height. I haven't even found yet the official way to tell apart real cores and hyper-threads in the APIC table. Best to go looking into the xnu code how OS X does it.