Hey Robbie.
I get a decent amount of sway and drift if I remove front AND rear simultaneously. More than I liked. I reattached the front and drive with the rear removed and its fine for me. I never tried with the front removed and the rear attached.

Why don't you unhook them, zip tie them out of the way, drive around and if you don't like it, reattach the fronts or rears and then give that a shot. Whichever ones you can deal without, remove. You may save yourself some work while you find a combo that works for you.

Individual feedback will vary with individual driving styles.
later,
-j