Derived from the phrase 'lunatic fringe', the small social subset that believes in completely bizarre concepts. Off the fringe is beyond this.
To do something totally balls-to-the-wall mad that will likely get you killed is to 'go off the fringe'
Steve went extreme masturbating yesterday. He jumped from a plane nude with a gallon of ky while listening to thrash metal and thought about gorillas in vinyl catsuits. That stuff is off the fringe.