right, the retaining flange doesn't just bolt to the outside of the axle, but kinda slips down in the tub with a real tight, almost interference like fit. This is what builds up rust and can make it really tough to pull the shaft off. Once you break the rust free, the flange will start to slide out of the tube a good inch or so and then the shaft will be free.

-Rob