The phrase used when a person is sweeping the floor and someone steps right into the pile of sweepings said person has gathered, accidentaly or deliberately, and scatters all of it back across the floor, causing the person to have to restart.
Idiot: Hey what's up man? (walking across the floor)
Sweeper: Oh, not much just sweeping the floor because my girlfriends coming over later.
Idiot: That's kool, well just let me get out of your hair.. (stepping on sweepings, scattering them)
Sweeper: Get off my sweep you bastard!!! Great now I have to start over. Just get out!
When you want someone to leave you alone, get out of your way, get out of your sight ETC, you say: "Get off my pitch!"
Boss: "This is the 15th time you've been late for work!"
Employee: "Sorry Sir! My dog had a really bad seizure and I had to rush him to the vet."
Boss: "I've had enough of your stupid excuses. You're fired! Now get off my pitch!!!"