The term allegedly comes from the Colonial Era in which tar was one of North Carolina's major products. It came from the workers who walked through the forrest accumulating the tar on their shoes.
A more colorful explanation comes from the Civil War in which North Carolinian soldiers were abandoned by other retreating soldiers to fight alone. Upon returning to the retreaters, the "Tar Heel Boys" told the other soldiers, "Next time we'll put tar on your heels to make them stick," insinuating that they won't be able to flee.
Dean Smith was the coach of the basketball Tar Heels and is regarded as the greatest coach in the history of college basketball. He compiled 879 career victories as well as 2 national championships.