Are the e-brake cables hitting the springs?

If so, you might need something like this:

[Linked Image]

http://www.4crawler.com/4x4/ForSale/Shims.shtml#ToyotaEbrakeExtensions