does your CJ have a sway bar? what tire pressure are you running? could be a multitude of things that is causing your CJ to wander. when I first got my CJ7 back in the early 80's there were no large radial off road type tires and most off roaders ran the Armstrong Tru-Trac. those Tru-Tracs were great off road but their on road manners made for some white knuckle driving! what a difference it made when I went with my first set of BFG M/T's! it was like night and day there was so much difference.
anyhow, ask around for a reference for a reputable front end shop and have them have a look at your ball joints, check all bushings and possibly replace them with the non rubber type. how much play is in your steering box? I could go on all night here........