Your idea is exactly what I used as a drift. (steel pipe) I simply set the flange end of the axle on a block of wood and hammered the bearing then retainer on. The pipe I used was too long to fit under my shop press.

If you have ABS don't forget to review the FSM to determine the depth to which you must press on the ABS rotor.