If you buy bananas on trash day, all the fruit flies will find their way into your home because of people's trash bins being moved about. Much like "Don't tempt fate", this is a phrase used to call someone out for their own foolish mistakes, or to warn them that if something can go wrong, it likely will go wrong.