PC Emitting High Pitch Noise After Windows 7 Loads

I have a really strange problem with my new desktop PC. Right when Win7 starts and the desktop appears, the PC case starts emitting a noise with a really high pitch. I tried almost everything to find out which component emits this noise (it's not the fans, nor the CPU, nor the video card, nor the hard drives). After hours no end of trying everything I could think of, I found out that the only way to stop the noise is to disable an item called "USB Input Device" from the Device Manager of Win7. I have several items called that way, but only one is causing the noise. When I do disable the device, the mouse stops working and the noise is gone entirely.

But as soon as I enable the device, or plug another USB mouse on the PC, the noise comes back. But here's the interesting thing: I found out that if I start Win7 in safe-mode, even with all services and network on, it doesn't emit the noise, and the mouse works perfectly anyeays. I even see that same device enabled in the Device Manager, but it doesn't make a sound. I cannot use Win7 in safe-mode all the time, but I also cannot use the PC without a mouse. I'm thinking on buying a PS2 mouse instead of a USB mouse and try that, but I really like both my wireless Logitechs.