During reconstruction after the civil war in the southern U.S. there were notherners who came down who were called scallywags and carpet baggers. The word scallywag meant troublemaker and carpet-bagger was an allusion to the bags carried by the norther buisness men who came just to profit. The use of both of these terms was in effect of any person seen as one of these type of people, regardless of race.

When you eat dirt for the first time and leave your family to build a shack in the woods