To be fair is a phrase mostly used by people who make a living stealing oxygen from others. It serves no purpose other than to try to make their forthcoming statement sound more profound whereas it actually devalues everything that follows. Should be filed along with "Proactive", "At the end of the day" and "To be honest"