Yeah I wouldnt pull the cam until its the only possible things left that could be wrong. I would do more trouble shooting up top first. I dont think the lifters should leak down over night either, they might loose a little, but to go all the way down, i doubt that. If they do leak down it with click and tick for about the first 20-30 seconds until the lifter pump back the little they lost. You can do another way to measure the cam with out pulling the cam too, if you tak all the rockers off and keep them in order and check out all the push rods are in good shape and dont find anything (again keeping them in order), make sure the lifters all have recently has pressure on them, and one at a time put in a push rod and hold a dial indicater on the end of the rod and have someone turn the engine over by hand. this will be easier to do with the spark plugs out. Granted this will not be as good as measuring the cam directly, but it will require a lot less remove and replace work than actually pulling the cam. Just take your time and hit each out down the line.

Sean