It is a common term, often used when 'grinding my gears' or 'annoying me' have already been used or aren't quite alternative and imaginative enough. Much like the chafing of tight underwear against one's thighs, the chafing of the mind is both annoying and unwanted. When someone's rubbing your brain and leaving a rash, just tell 'em how you feel.
Roy: "Hey mom can I have some money for the bus?"
Mom: "Fuk off Roy. You're really chafing my mind today."
the gradual deterioration of pocket fabric where a cellular device has been repeatedly stored, thus, causing excessive rubbing that results in an irritated, worn out patch on a pair of pants