To aggravate someone.
The saying "Get Your Goat" supposedly originated from the early horse racing days, where goats were kept with nervous race horses to keep them calm before the race, and if you wanted to beat the competition you would get their goat causing their horse to become agitated and unable to race. (from: http://goathavenfarm.tripod.com/)
Don't let him bother you, he's just trying to get your goat.
The Urban Dictionary Mug
One side has the word, one side has the definition. Microwave and dishwasher safe. Lotsa space for your liquids.
Buy the mug