The term was originally High-Falutin Yella.
Well when the French ruled New Orleans there were many, many brothels and the highest priced was paid for the mixed African/European free prostitutes. It is well known that many slave owners raped their slaves and the children often escaped or were freed by the owners (how does one explain little mixed slaves running around the plantation)? This was not always a rule but many mixed slaves made up the population of free Africans in the colonies.
But. . . no matter how light, no matter how blue their eyes were, the High Yella's were still not allowed in the white community and in the black community with their perfumed skin, fine clothes and perhaps slightly uppity attitude the term High Falutin' Yella was coined.