A wanker is always used as someone who is ineffectual or useless. It does not substitute for asshole, bastard, dickhead etc. It´s roots are in the assumtion that someone who is so useless he cannot find another human being to have sexual relations with has to have sex with himself.
Hence the term meaning to be one who masturbates frequently.
Tony Blair is always pandering to the Americans...what a wanker!
He failed his driving test again the fucking wanker!