The shortest answer is that you want hazards to pressure the opponent when the switch in and if you can help it, not deal with them yourself. It also depends greatly on the team but I'll first provide a concrete example
Imagine someone is being a moron by running a specs solar power Charizard. Zard is a very hazard weak Pokemon and if you're playing against it, you are gonna want rocks to limit the thing to only three switches, barring random chip damage. Why? Well, do you want to be on the receiving end of a stab fire blast that is boosted by by sun, specs and solar power? No, you don't. As such you want hazards to limit the opponents options. This is just the most basic example as of course stealth rock doesn't affect everything equally which is why spikes is such a huge deal
Now, still on that example. Sun teams mostly carry Mandibuzz otherwise they get brutally ripped apart by Garchomp and Dragapult. Even though Mandi provides defog, sun teams still carry rocks on their Torkoal because they love the pressure hazards exerts as something like rocky helmet Blissey will struggle to contain that very same lizard
This does not apply to only sun teams. This applies to every team that isn't hyper offense because ho doesn't have the time to slow down and remove hazards. You want hazards to limit the number of times your enemy can switch in and at the same time, you don't want to deal with hazards unless you have to. Something like Heatran is gonna struggle to switch into specs Blacephalon shadow balls if it has to contend with hazards or how Landorus can easily be put into Garchomp's kill threshold if it is constantly being forced to come in with rocks up
Here is where teams come in. Since it has been established that you want hazards to pressure the enemy if they switch, your team comp would also affect how you deal with them. Take this team I made for example. Two of them shrug off rocks, one can ignore and the other two are neteral. The last one is the only one weak so how would that team play, if I can't reliable two shot the enemy, then I might as well use defog to remove the rocks otherwise the main breaker is gonna have a very difficult time. When you compare it to this, this one doesn't really need hazard removal since most are neutral or resistant to it so in which case, I will prioritize keeping hazards up as that will put enemies into Tyranitar's kill threshold
In the end, the basic idea is that hazards put pressure, me want. But if my team is heavily pressured by hazards, me no want. If the enemy team is more vulnerable to hazards than mine, keep them on as much as possible. If both teams are equal, set them up or get rid of them based on the openings on the current state of the battle. Flexibility is the name of the game. The biggest reason for having both rocks and defog on the same team is because the mons that learn those moves are just great mons overall. Landorus, Heatran, Tornadus and Corviknight are all excellent setters / removers and they just fit naturally on a team
Special mention would go to hyper offense team. Why? Don't put hazard removal on hyper offense unless you really know what you're doing. The whole point of ho is to bombard the enemy with attacks before they can respond and if they try to defog, they will get punished by a kill or something like defiant boost on Galarian Zapdos
Finally, if you're in a situation where you have arock weak team with no hazard removal, wtf are you even doing? You're just begging to receive a huge amount of pressure in only one turn. Defog is rather negotiable since sometimes, hazard stacking teams wouldn't pack a defogger and would instead pack a defog deterrent like the Galarian Zapdos mentioned earlier
