A poorly constructed boat used by refugees.
It was also used to refer to inflatable boats back in the day. (old school)

A Puerto Rican. Like a wet back. But a Swimback needs to swim back to his damn island.