A person who has constantly accumulated irreputable damage throughout their entire life. Damage which cannot be easily repaired if at all. Also someone who has been beaten into submission by life's adversities or has given up on trying to overcome the obstacles.
A song by Metallica called, The Unforgiven might be about a broken man.
There's an old broken man living up the street who hides in his house all the time.