I think you should jack up the front end and put jackstands under each lower a arm and crawl under and wiggle the suspension joints and steering joints. Crawl out and grab the tire top and bottom and feel for slack in the wheel bearings by pulling alternately at the top and bottom. Have somebody turn the steering wheel while you watch the joints in the steering for slop.

My bet is wheel bearing on the side it pulls to while braking. And I bet the idler arm is toast, too, since they all are after about a year or so. Ball joints won't do what you describe. Tie rod ends, pitmann arm, wheel bearings, and idler arm will, but usually won't make a grinding/whirring noise.

But that's just me thinking...