The word Cracker is in reference to the one who cracked the whip. Though the large majority were white slave owners, a few blacks got to use the whip too.
Calling a Black man a Cracker, is way worse than calling them a Nigger. Niggers back each other up, that's respectful, but a Black Cracker will sell them all out.
A Black Snitch.
"God-damn Black-Crackers done us all in!"