To fondle them would be really weird, and to be caught fondling them would be downright embarrassing (and slightly hilarious...)
In J. K. Rowling's "The Tales of Beedle the Bard" page 60 (U.S. Edition), Albus Dumbledore says that it was rumored that his maiden aunt Honoria called off her engagement because she found her groom-to-be fondling some horklumps.
It has since become an inside joke on the Harry Potter podcast PotterCast, which comes out once a week from The Leaky Cauldron at LeakyNews.com.
"Hang on, babe, I'm fondling some horklumps."
"Nothing, dear. I'll be there in a minute."
Teehee. Fondling Horklumps.