Does it makes sense to think that it might be too long and actually shifting my rear axle over?

Yes. But it is a tradeoff. The panhard bar locates the axle laterally. A longer one is needed if you want to maintain "center" when the vehicle is lifted. It also allows for more travel before things bind. But...this means that as the suspension is compressed it is moved laterally (in your case, left) farther than it would be with the stock bar. In otherwords, when the axle is compressed to the point of the original ride height, it is no longer in the exact same location. Hence, rubbing.

Just clearance what you need to and be happy.

Michael