The Grey Men have complete authority over all things. Even authority over the gods themselves. Grey men do not exist in just one realm/plane when one becomes a grey man they exist in all realms in all planes and in all of time from begining to end. What the grey men choose to do is done for their own good. The commander, of the Grey Men, has authority over the other grey men. The Commander appoints two generals who answer only to him, when the commander dies the generals are forced to fight each other, the winner becomes commander of the grey men, and the loser is forever trapped inside of his own mind to be eternally punished by having to feel all of the pain he has ever caused anyone and to feel all the pain that he has ever been through except that even the smallest amount of pain will make him wish that he could die but he will continue surviving and suffering. Once the commander makes someone a grey man they are written forever out of history, sometimes being written into fiction if they are lucky. Once a grey man dies he becomes nothing for he never existed other than when one of the generals loses the fight with the other general. except when the commander dies he is remembered.
King Aurthur and the Knights of the Round Table and Merlin eventually became grey men.
Atlantis exists in fiction because it was sunk by a grey man.
Avalon, Merlin's home was cast into fiction by Merlin becoming a grey man