Uninstalling Old Video Drivers Issue

I've been running video since the old voodoo 3 cards, though technically it's been longer but I got my own computer when the voodoo 3's were out. =P Anyways, I've been working with drivers since then, and I usually update every time a new update comes out and never uninstall. Heck, I don't even re-boot after installing!

From my understanding, drivers are the same as kernel modules (, or at least what we talk about in our linux world). And so shouldn't they just work after loading? I mean in linux. Well lets say in linux removing driver's is a lot more of a pain than windows and so I really want to know what all the fuss is about.

But I can't find it anywhere that says exactly why you should do it, and if it's even worth doing. I've always seen it as a waste of time, and all the old files should be overwritten or cleaned out. Idk, I've just never ran into errors from just installing, I've never seen any real evidence that proves uninstalling has any gains.