Short for "English as Second Language", the term actually refers to *people* who use english as their second language. These people are often falsely thought of as morons, or generally a person of lower degree/intelligence. They often use their own version of English, mixing in some of their own words or their first language's grammar, especially when talking to other ESL, even more so when the other ESL's first language is the same as his own (when they are speaking English to each other for whatever weird reason, that is.)
Generally ESLs can understand English decently, especially when an ESL version is spoken by another ESL. Their first languages needn't be the same, but it would help even more.
Between the english-speaking community, however, it has a derogatory meaning.
Between 2 English-speaking persons (an arrogant one apparently) :
Damn, i got an ESL in my group. This is gonna suck...
Zip it, you'll be lucky if you contributed half as much as him, assuming he knows what's expected of him.