Basically an insult calling you and your father
ugly.
-(Explanation)-
You and your father look alike, and are genetically related, so (typically), if someone were to say “You
ugly, you yo daddy son.” They Are calling you
ugly initially, then linking that same adjective to your father.