You might wanna try rubbing some pencil lead into the nut grooves, this lubricates it, so that the strings slide through and don't catch when you tune/ use the vibrato.
You might be alright going from 10s to 11s without having to get the nut filed, but if you do have to file the nut I'd recommend getting it done by a tech, rather than mauling it yourself and then having to get a tech to fit a new nut.
One thing I found with the Mustang vibrato is that you need toget the spring tension right or the strings will keep either going flat or sharp. A good setup makes a huge difference.