I'm going by two How-To writeups that I found online (one here, one elsewhere) and they both said to remove it just to have easier access to everything.

I'm wondering if I'm going to be able to get the lower ball joint component off - the bolts hardly look like they're there, though that was last night about 11pm when things were pretty sleepy/fuzzy. Light of day will help out with that.

Thus far I've invested about $120 in tools (some i needed anyway) and the $75 for the new CV, and $85 for both ball joints, for a total of $280. Much better than the $700+ a few shops quoted me.