@apfelnico found the solution for the boot issues on big sur and the error code 97.
"It seems Apple's new USB-ASMedia connection is broken. We use "ASMedia2142" on our boards, but Apple's driver is for "ASMedia3142". This driver runs first, then it doesn't run anymore on further reboots. Then the higher-level driver takes over, and it doesn't work with our chip. But the old working "AppleUSBXHCIPCI" is still in the system, which is included with Catalina. We can fix this by loading this Kext for the ASMedia controllers."