Papadanny is right, the regulator is built in to the alternator... you'll need a new one. I've replaced an alternator on my '89 due to a failed regulator and the symptom was overcharging, bright lights and a pegged voltmeter. The alternator is very powerful for quick charging... the regulator is really a constant voltage limiter if you want to look at it that way. On the same note, if your battery was weak, close to dead, you'd probably see a bit higher than normal readout on your voltmeter as the regulator would allow more current through to get it charged up. Have you checked the battery/alternator with a tester? I guess it could also be a bad gauge, but I'd think that the draw/output of the new battery was a bit of shock to the system and you blew a diode or two (I think there's a total of 3 and somehow they add up to the total voltage output of the alternator).

If there's a good charge on the battery for now, go to Pep boys or Auto Zone and have them test... they'll spot a bad alternator. Just don't buy a replacement there unless you want to replace it again soon.