The issue is weight. You can have a small boat and trailer that weigh 1000#'s and it's no big deal to tow it. You could also have a boat that isn't terribly long, but with a thick hull, big V-8 engine, and a heavy trailer, as well as coolers and gear is over 4000#, which is a problem.

I've heard recomendations that you should look at the the recomended max tow weight of a vehicle, and plan to tow a load that is 50-80% of that rating, and that seems prudent to me.