usb audio crackling

So here's what's going on. I use optical audio through my Astro A50 headset for everything but Skype (voice calls over optical just drives me crazy, so I use the USB audio). After about ten minutes of being in a Skype call and starting up a game of League, or whatever Steam game I'm playing at...