Why, you ask? For a few reasons: first, a mixed sweeper is rarely used as a jack-of-all-trades (I'll explain later). Second, by splitting your defenses, you make it all the more easier for both types of attackers to wipe the floor with you. Third, the number of actual dedicated mixed sweepers is actually low in number.
Mixed sweepers are generally used to hit a certain threat, such as putting Fire Blast on Garchomp to hit Skarmory, or Hidden Power Ice on Infernape to hit Gliscor. Mixed sweepers are, most of the time, mislabelled; a more appropriate name for them would be surprise counters, as the purpose of the "mixed" element is to eliminate a certain threat.
Say you have your trusty Mega Sableye, with stellar stats in both Defence and Special Defence. The best option is to dedicate that wall to one purpose. If my Sableye is fully invested in Sp. Defence, then sure a it may get KOed by two or three Physical attacks, but it won't be going down easy on the Special side. Whereas if you cut a large chunk of it's Sp. Defence to beef up it's Defense, you will find your Sableye lasting a little longer Physically, but losing quicker Specially. I'm not saying that a mixed wall is a bad thing, but that mixed wall should be leaning more to one side than the other.
Only a select few Pokémon with about equal offensive stats can even consider being a dedicated mixed attacker. Examples of these are Lucario, Infernape, Greninja, Garchomp, Genesect and Zoroark. The actual number of Pokémon that can hit with equal force from both sides is slim. Also, attackers are — much like walls — more inclined to fully invest in one side, either Physical or Special, and seldomly do they differ from that norm.
Just a few last notes: If you fear sending out the wrong wall, such as sending Physically defensive Heatran to wall Talonflame, only to find it using all-Special moves, don't fret. Players dumb enough to use things such as Special Talonflame and Physical Gengar are usually easily beaten; fall back on the appropriate counter, or just switch to your other wall. Secondly, there are some exceptions to the "One Stat to Rule Them All" rule. It's perfectly fine to run a Physical wall with Calm Mind (namely Mega Sableye), or a Special Wall with Bulk Up (Conkeldurr). Just don't fully invest in both stats; put the vast majority into one and the leftovers in the other.
I hope that this answered your question, and good luck finding some good walls!
Postscript: Just an unnecessary tidbit, if you find yourself have ing difficulties fighting a particualr wall, use a wallbreaker. These are Pokémon who, in effect, wall the wall (immune to/resist the opponent's attacks and status), can set up all over it, and then break it with a powerful boosted attack. If you find that a certain wallbreaker is thrashing you, devise a counter/check for it.
Post-postscript: Was that confusing enough for ya?