Nothing to remove, the egr valve is basically the only mechanical smog device left on it, and it does nothing to reduce power. You can increase the performance a little by cutting some joles into the intake box to increase more air flow, but that is about it for any cheap gains.

Your best way to increase performance(other than tearing into the engine), is to install a good hi-flow cat and a turbo style muffler, though better performance will be achieved if you install headers as well. Next you would want to install a hi-flow intake system, something like the K&N kit or one of the other aftermarket kits, I would also modify the kit to be a cold air intake system. A Jacobs ignition upgrade and a larger throttlebody would also be a good idea. Other than that you would have to consider increasing enlarging the cylinders and reworking the heads. With the exception of the new cat and muffler these modificatuons aren't cheap, basically between $3-700, with the heads and engine being more.