Originally, adult sports involving a high level of danger and adrenaline that would not appeal to most people but attract a daring few: things like skydiving, bungee jumping, rock climbing, ice climbing, technical mountaineering, hang gliding. It generally did not include sports which were "extreme" in their early years but since became mainstream (e.g., downhill skiing or scuba diving), nor did it include children's and teenager's activities like BMX or skateboarding. Extreme sports originally meant adult and non-mainstream. Later (from the late 1990s on) the meaning changed to become synonymous with non-dangerous kiddie sports and the definition has more to do with age (and with marketing energy drinks, Mountain Dew, baggy pants, and punk and thrash music), much to the chagrin of older folks involved in dangerous sports who now would rather the term would go away.
Skateboarding? That's not extreme. If you want a real extreme sport try hot air ballooning.
An extreme sport is usually defined as one that requires extreme levels of endurance, balance, skill, co-ordination; also, ballistic reaction timings, spacial orientation abilities, and massive amounts of oxygen assimilation. Extreme sports are dominated by youth, age range 6 to mid 30's. The first recognized extreme sports, were officially documented by the Post Office commissioning of a stamp series highlighting the first three: Skateboarding, Snowboarding, and BMX bike riding. Since that time, several others have been added to include, Inline Rollerblading and Motocross.
Skateboarding is consided an extreme sport.