Similar to "back in the day". Usage signifies that you have known said person for a long time; technically since you were little kids who were only as tall as the lower leg of an adult. Your height was "knee high".
Aaron: Hey man, that kid Sam is a real chiller. How do you know him?
Ben: He's been my boy for a while. I know that kid from knee-high