Having or making a finger baby is simply the act of fingering as performed by a lesbian to another. Lesbians are women and they do not have penises so they cannot actually make a baby. Rather, they finger each other, thus colloquially sex can be called making a finger baby, or often, making finger babies. See the "finger babies" entry.
"You're hot. I want to have your finger baby."
When you have finger to vaginal intercourse you make finger babies, because when you make a real baby you have penis to vaginal intercourse.
How was your date, did you make finger babies?
When you come on your hand, in which there's a cut on your finger thus resulting in a finger baby.
After Jason's fifth ejaculation of the day, he realized he had a good chance of having a finger baby 9 months later.