Hmm, Mr Mime sits in the base 90 Speed tier
Which I feel is quite an important benchmark for speed, and Mr Mime tends to run Calm Mind Baton Pass sets with Encore to stop setup sweepers, so it needs all the speed it can get. I feel that max investment in speed is the way to go in order to outspeed base 100 speed Pokemon without a positive speed nature like Charizard.
EDIT: For an Assault Vest set I don't really think you are outspeeding anything relevant with the 128 Speed, so I suggest running full investment in Special Defense in order to get a bigger boost from Assault Vest.