Basically an insult calling you and your
father ugly.
-(Explanation)-
You and your
father look alike, and are genetically related, so (typically), if someone were to say “You ugly, you yo daddy son.” They Are calling you ugly initially, then linking that same adjective to your
father.