A Jewish foreskin or Jew's foreskin is always off, therefore the expression is a very definite one about being off on one's way or just in a generally off condition.
It is time that i was on my way, so I am off like a Jewish foreskin. (or Jew's foreskin)