Definitions by David Charbonneau

heart is in the right place

one's intensions are good despite indications to the contrary

He often insults others by what he says but his heart is in the right place