You are experiencing the same issue I have been dealing with. I had a very loud throwout bearing and decided I couldn't take it anymore, I bought a new clutch and bearing and put it in myself. Start the engine....what do I hear....the same whirring noise...just quieter. It is definately the throw out bearing as I can put my stethascope right on it while the engine is running and when someone pushes in the clutch it goes away.

I have just gotten used to it...I believe it is the nature of Toyota throwout bearings.

Another board member did suggest putting a little bit extra grease on the pivot point of the clutch arm to "dampen" the noise and this did help a little.