As of Gen III, the Burn status condition no longer has the condition-bearer's stat modified, instead a Pokemon with the Burn status condition has its damage reduced by 0.5. This is somewhat equivalent (not exactly) to having the Atk stat drop by 2 stages - this being a sort of an estimate, without actually considering abilities, move conditions, or type weaknesses.
However, with the use of Belly Drum, although it reduces half of the move-bearer's maximum HP, it maximizes your power, thus raising the Atk stat of the Pokemon by 6 stages (4x the original attack stat).
As you can see, modifying a move's power versus a stat is not the same thing, but raising the stat will almost always give a better outcome in your favour during a battle - in concern to attacking stats.
So, in this case, and to answer your question, the Atk drop will be negated after the use of Belly Drum. With these two conditions in play, you will still end up having a higher Atk stat than you're original attack.
Also, as a note, if your Azumarill has the ability Huge Power, you may not even need Belly Drum, as it cancels out the attack negation. *Huge Power doubles the Atk stat.
Hope I helped! :)