In the West of Scotland, I believe the term equates to someone that no one would associate themselves with else they get tainted with the same brush.
NOT a term of endearment.
It applies equally to both male and female... no sex discrimination there!
As children we would call someone a gadgie if they had a cold sore or spots and avoid them.
As teenagers it would apply to someone of the opposite sex who was not particularly attractive.
As adults I think we stopped using the term.