The cops. It's correct that it means looking more than once can draw attention. But the actual phrase comes from someone saying "one time" to let the other passengers or friends know that they have now seen the cops and are claiming the first look, and no one else should now turn around because two or more people looking increase your odds of being stopped by more than 50%.
"One time, man. Don't turn around!"