In eighteenth and nineteenth century England, it was considered a great offense to say "By God". Instead, men would say "BY Jove". Jove is an alternative name for the Roman god Jupiter. By saying "By Jove!", they could be saying "By God!", without committing a social offense.
"By Jove, Tom! William has a magnificent carriage!"