The areas north of NYC in New York State, with maybe the exception of Westchester County (depending on the context) due to the fact that it has a lot more in common with the areas to its south than to its north. Rockland County more often than not is considered to be Upstate NY (sorry.)
In rural New York State people will either use the NYC area or the Albany area as the marker for upstate. Some people from Orange County for example, say they live upstate and some vehemently oppose that designation, as Orange County is well south of Albany.
The word "upstate" has a particularly negative connotation and should be avoided. Few people like to be considered to be from "upstate", as it implies the antithesis of all that New York is about. "Upstate" is the N-word of geography and should be avoided at all costs.
Note: There is also a rare usage of "upstate" referring to everything outside of New York City, including Long Island and Westchester, just meaning New York State.
In Brooklyn, anything north of the Bronx is upstate.
In the Bronx, anything north of Yonkers is upstate.
In Yonkers, anything north of White Plain is upstate.
In White Plains, anything north of Peekskill is upstate.
In Peekskill, anything north of Poughkeepsie is upstate.
In Poughkeepsie, anything north of Albany is upstate.