This is a phrase used when also pointing at the floor 'you dropped your gay card" the other person automatically looks at what your pointing to, without realising what you said. When they realise that they are looking for their imaginary gay card on the floor they feel very stupid, but very funny to everyone else.
Person 1. (points to the floor) Geoff you dropped your gay card!!
Geoff. (Looks to floor then feels foolish)