Soft Recording Volume With Blue Yeti / Increasing Volume Distorts Audio

so i have a blue yeti, and i'm an online animator/voice actor. whenever i record audio on my blue yeti (or any other mic) i record about 4-6 inches from the mic, and turn the gain down enough that the audio doesn't peak as i yell or whatever. this is what you're supposed to do if you don't want your audio levels peaking and making your recording sound horrible and corrupted. however, afterwards, it means i have really quite recordings. not a problem right? now that it's recorded right, i can boost the volume of the entire clip afterwards right? wrong. i'm not an audio expert so i must be missing something. but why in gods name would the audio distort and peak after it's recorded successfully? it's all there, it's all clean audio. if i turn the volume on my speakers all the way up it sounds crystal clear! all i want to do is do that inside the computer so that when i upload it to Internet people don't have to turn their speakers so far up. but if i increase the clips volume in the computer, it distorts! it sounds awful! why?! i tried in sony acid and audacity and they both do the same thing! i can take sound effects, and recordings made by other producers and crank them up nice and loud, but mine just fall apart!