I wondered about that myself, so I ran some calculations!
Let's see the effect of specially defensive investment against a Modest nature Sylveon with ability Pixilate and item Choice Specs (252 EVs in SpAtt.) using Hypervoice on a Chansey. Few Pokémon pack as much power on the special side.
Let's say our Chansey (holding Eviolite) has 252 EVs in HP and no investment in SpDef (EVs or nature), this results in damage of 25.2 - 29.9% - guaranteed 4HKO.
Now, if Chansey has a Calm nature, 252 EVs in SpDef and 252 EVs in HP, you get: 18.4 - 21.8% - possible 5HKO.
Let's see the effect of physically defensive investement against a Jolly Mega-Kangashkan (252 EVs Att) using Return. If Chansey has 252 EVs in HP and no investement in Def (nature or EVs), this results in: 97.7 - 115% - 81.3% chance to OHKO.
If Chansey is Bold and has 252 EVs in Def and 252 EVs in HP, you get: 37.9 - 44.8% - guaranteed 3HKO.
The impact of physical defense investment on Chansey is massive. Chansey's base defense is so small that 252 EVs more than double the final stat. A bold nature coupled with eviolite further increase the stat to bearable levels. In comparison, investing in special defense is like a drop in the ocean. A wall is usually better off specializing in either physical or special attacks, but Chansey is on a class of its own.