In a realistic view, you do not have to worry about your team in an in-game environment. Everything is easy, casual, and requires little strategy or team-building knowledge.
However, in a metagame / professional environment, a team of Sweepers can be either a devastating force, or a massive Achilles' Heel. Let's say you have Salamence, Lucario, and Tyranitar against Aggron, Shuckle, and Cradily. Even though Sweepers usually get stalled out by Walls, the sweepers would still win.
The sweeper examples I gave are so powerful, and have such strong attacks, that they are OP'd sweepers called "Wallbreakers".
In another case, Trick Room sweepers will usually beat normal sweepers, as long as Trick Room is actually in effect; especially Trick Room sweepers like Dunsparce, who can para-flinch in this environment without having to use Glare.
As you can see, there are nearly countless situations that can either put sweepers in a dominating point, or at a minority. there are too many variables to consider, like Attack and Speed stats, abilities, movesets, Choice sweeper vs. "Naked" sweeper, and of course, these variables, only pretaining to your opponent as well.
This question is unanswerable. I can give those disadvantages, laying claim to the fact that sweepers have low Defense / HP, are predictable, and will not last long if they can't keep a sweep going, but the situation is too varied to give a final verdict.
