In Japanese, the word means, literally, "outside person." In Japanese it's spelled 外人. It is often used as a contraction of sorts for gaikokujin ("outside-country person"), meaning forienger. In its contracted form it can be an insult, but in recent years has been watered down by widespread use in and out of Japan. In actual usage, a better definition would be "non-Japanese" since Japanese people will use it in reference to non-Japanese even when they themselves are the foriegners in a country other than Japan.
Gaijin in Japan stick out like sore thumbs because they're usually a full head taller than everyone else.