Using "pool" as a verb implies everything one does when spending time in/around a pool. This includes (but is not limited to) swimming and diving, tanning, eating and drinking, listening to music, and playing games (e.g. Marco Polo).
"May I come over to pool later?"
"Hey, a bunch of us are getting together on Saturday to pool. You want to join us?"