This is how Chicago mobsters describe business activities that are not illegal. Gangsters often invest ill-gotten monies into legitimate businesses, which, in turn, generate what they consider to be legitimate profit. Such business are referred to being "on the legit" (despite being an essential part of a larger money-laundering enterprise).
Guy #1: We'll fill the joint with high-line Spanish broads, and keep it on the legit.
What Chicago mobsters and wanna-bes say when referring to someone who is privy to or has inside knowledge of mob business. Such individuals are said to be "on the know." It means basically the same thing as "in the know" but only in reference to rackets.
Guy #1: Hey buddy, how about 'dem guyz on Grand Avenue that know da mare's kid?