The only real comeback to a so's your face
. Makes just about as much sense, but it's a triple threat - it's offensive, confusing, and it draws attention to the stupidity of the phrase "so's your face".
When used, your opponent will be utterly confused. They will either stand with a bewildered expression on their face, or reply with a curt "What?"
At this point all you must do to finish now is go "Yeah! What now?" and walk off with a slight swagger. You win.
This works just about 100% of the time. I'm serious, try it.
Person 1: Alright, that's it. You're an unbelievable douche.
Person 2: Well, so's your face!
Person 1: Well, so's your nipples!
Person 2: ...
Person 1: What now, punk?
Person 1: Will you shut up? Santa does not exist.
Person 2: Your face should shut up!
Person 1: Your nipples should shut up!
Person 2: What?
Person 1: Yeah, that's what I thought!
(as you can see, it works best when you copy the same format the other person used so's your face in)