Gday Dougie, That diagphram that you mentioned is called the Anaroid governor, this senses any change in boost pressure and alters the fuel delivery to compensate. Its very important that the tube is not restricted in any way because the fuel-air mixture will become lean of fuel and you will have a drop in power. When you feel the surge in power as the turbo spins up, thats that governor doing its thing but it must get a clear signal from the boost side of the turbo (Engine side). If in doubt, replace the hose and It dousnt sound right that there is oil getting into the diagphram, check with your local diesel injection specialist on that one. It sounds like you have found your power problem though.

Good luck