A phrase people use as a disclaimer to make an excuse to say or do something rude or mean to you to prevent you from having a poor self-image or so there won't berepercussions and make you feel they still like you even though they probably don't.
Person A: Oh, the teachers are wasting too much time.
Person B: No!I have been working all day long and did not take any breaks yet...
Person A: Oh, "don't take it to heart", I did not mean you.