To really whoop someone or give them trouble. To attack. If you really bust someone up and thoroughly kick their tailfeathers, you
will have given them hell.
Since hell is a bad place full of suffering, to give someone hell is to create your own little part of it just for them.
It can also be a term for teasing or general razzing, ("he was really giving you hell at the
party, wasn't he?") as well as
winning in an athletic event. "We really gave that team hell."