Warm Features to Warm Them Up