Ricky said: ok. just imagine that, whenever you play a string, the string behind the nut is holding tune too. if you slacken that part of the string, the playing part of the string will oscillate more widely
as you already said, the tension behind the nut is the same and therefore insignificant.
The nut is only serving to end the oscillating length of the strings, and the angle of the strings behind the nut doesn't make any difference to fret buzz. It can only possibly make a difference to noise from the nut.
I fail to understand how the tension/angle of the string behind/over the nut will affect the oscillation of the string, unless you've got a ridiculously wide nut slot.
I'm with springy on this one, it doesn't make sense.
A lower tension string won't oscillate wider than a higher tension string unless it's hit with a higher proportional force. That's as far as i understand the theory, but looking at a string vibrating states the opposite.
i.e. an 800Hz string hit with 20 newtons of force will oscillate to the same width as a 400Hz string hit with 40 newtons of force (if that's what the relationship is, I don't know)
Anyway, I think guitar strings oscillate primarily in one plane (from side to side) and not so much up and down, because they're hit from side to side.
All of which brings us back to the fret buzz... as the nut slot is cut tight to the string and it's a node of the oscillating length and the string behind the nut isn't oscillating, the angle over the nut can't make a difference to fret buzz, because the tension is equal with or without a string tree hence the width of oscillation is still going to be the same (depending on the force the string is hit with.
It simply doesn't make sense to say that the width of oscillation will be different with a different string/nut angle. (with equal tensions and initials displacements of the string)
I wish I could get my thoughts onto paper clearer than I can. aargh.