Actually, it means that you live in the middle of nowhere - in the boonies - out in the boondocks - in an extremely rural or out-of-the-way area. Destination has nothing to do with it.
If the nearest place to buy gas for your car is an hour's drive away, you live in bumfuckegypt!