The practice, state or ability of having more than
one sexual loving relationship at the same time, with the full
knowledge and consent of all partners involved.

This is the definition used by California polyamorist Morning Glory Zell, who coined the term in the early 1980's.

A form of relationship involving more than two people. The relationships may be symmetrical (like a triangle), or something more complex. The relationships may also be open or closed, straight or gay, or a mixture.
A purely heterosexual form of polyamorous marriage was once practiced among the Tibetans: each man had many wives, and each woman had many husbands. This raises the possible of your brother also being your husband-in-law-in-law.

In its most basic form, it is defined as honest consensual non-monogamy. While some decline to allow that Swinging is part of PolyAmory, the basic definition still applies. You will find, though, that there are as many definitions of PolyAmory as there are people practicing it. PolyAmory must have as a basic more than 2 people in the relationship. Not all participants need be involved with every other participant, nor is co-housing a requirement.