Origin of term is in question,but the one with most credence points to a combination and adaptation of terms. First, the muffs, a fashion accessory, most popular from the 1920's to the 1950's,for keeping the hands warm were most often made of rabbit or "bunny" fur. This fashion practice was adapted when the pullover hooded sweater became popular in the '50's and 60's when the pocket in front was also lined with fur from Peter Cottontail and his cousins.
The pullover sweatshirt, requiring some adjustment when donned, can call for the wearer to move the arms in a "hugging" motion when settling the pullover onto the body.
It can also feel as good as a hug when one is able to place one's cold hands into warm pockets!