I live on Martha's Vineyard year round and I have to deal with all the tourists (by the way, some of whom are not all that bad) and then I get to live there through the desserted winters...and it is still my favorite place on earth. The natives are highly misunderstood. None of us are rich, we are scraping to get by because the Island is so expensive. We make our money through the tourism and we grin and bare it all for 4 months a year. The upisland beaches are gorgeous, and upisland in general is the place to be...avoid the crowds at all costs. But honestly, if you are looking for a place to hate on, look somewhere else. This is island is gorgeous and welcoming.
And yes, Nantucket has nothing on Martha's Vineyard.