You would still have +1 defense.
Power trick switches the users Attack and Defense stat. This does not include stat boosts. Here is a test to back this up as well.
>Power Trick switches the user's Attack stat with its Defense stat, however any boosts or reductions that have been applied to those stats are not switched.