The
person or object referred to, for some reason, makes one lose one's resolve, rendering a
person easily influenced or coerced to doing something they're usually not predisposed to doing. In other
words, “kryptonite” in this context is a metaphor of “weakness”.