The EFI partition on the USB stick contains the correct version of the Clover bootloader and the necessary boot data (in the EFI folder) and that is why your system is able to boot from it.
If the version of Clover on the EFI partition of your SSD does not match the version on your USB stick, and/or the correct boot data is not located there, then you won't be able to boot from the SSD.
You can boot into Catalina using your USB stick, and then (1) mount the EFI partitions of the USB stick and the SSD using a mounting tool like EFI Mounter available from the downloads section, (2) make a copy of the EFI folder on the EFI partition of your USB stick to another disk (not absolutely necessary but recommended), (3) delete the EFI folder on the EFI partition of your SSD (NOT on the USB stick, make sure you select the correct partition), (4) copy the EFI folder from the EFI partition on the USB stick to the EFI partition on the SSD, (5) unmount the EFI partition.
Finally eject and remove the USB stick and restart the computer. Hopefully your system should now be able to boot from the SSD without needing the stick.