Tire pressure differences normally only cause pull in one direction. You mention it likes to wander... and it has no lift, but running 31's. Puny, but bigger than stock.. a cheap check might be yer steering stabilizer.. I was running 31's, and mine rusted out, pin-hole. Cause of that, I had to replace everthing beyond the pitman arm. Well, also cause of rust! Don't drive much on road, so it got pretty far gone before I noticed I had to fix it.

'04 Wrangler x 2" lift, 32's..