not being funny or anything here, but most people who don't like the mute usually have never played with one. it only puts the guitar noticably out of key as you move up the neck. they sound ok with other instruments. and have a purcussive sound all of their own, that can sound a bit different to normal palm muting.
also, taking a mute off seems pointless. if you thought it looked dumb, then i could understand. but a mute does not interfere with anything. so mayaswell leave it