It likely would have come out if I forced it, but I didn't want to rip the cv boot, and didn't have any spare clamps handy to remove and replace to try to get them off and on as it was not necessary to the operation. I'd try to cut the clamp and peel the boot back before I removed the fork myself, but that's just me. 2 clamps and 5 minutes as opposed to possibly snapping a bolt off, but on that note I've had a lot of luck with the sporty not breaking when I tweak on it.
On a different tangent I changed the brake pads today as the one pad was almost non-existant. Found both calipers to float freely, but the pads themselves were super tight in the carriers. I tried the new pads to find that I would have had to beat them in with a hammer if I wanted them to fit. I filed the edges a little where they fit to the carrier, so they wouldn't require a sledge hammer to adjust. My brakes actually feel better, with more positive feel and less pedal travel. Too early to tell if this is the fix, but worth a try I think.