It is true that you can lose a lot of HP when you are raising your attack/speed stats, but once you have done this a few times, then the fun begins. You usually end-up sweeping an entire team, even Pokemon that are specialised in defensive stats. When you raise your defensive stats, you can become a pain to defeat, this is usually combined with status corrupting moves like toxic, will-o-wisp,...So yeah, stat raising do make a great difference in both competitive and in-game battling.
Hope this helped