The word 'toerag' is believed to come from a rag used to wash the feet with, making it very lowly thought of.
Preceded more obscene (and thereby effective) insults such as tosser or wanker. It has fallen into disuse also because it invokes an element of the old class system, snootishness and looking down your nose at the lower classes.
Nah! I got her clouts off and her min smelled like a load of old toe rag so I left it.