A "Man's time" is time spent in a restroom of any kind while pooping. During such times, he is not to be disturbed by anyone, least of all a woman. Man's time is often accompanied by reading materials, including (but not limited to) pornography. Not to be confused with "Woman's time" which has something to do with the menstrual cycle.
"Dad? Are you jerking off to Barely Legal during your Man's time? I just had my first period and I need to use the can."