Attested in "True Grit" by Charles Portis, 1968, but most likely much older.
The derivation is probably from the
percussion "cap," a small metal cylinder open at one end with an amount of shock-sensitive
explosive, usually fulminate of mercury, used to set off the powder charge in a muzzle-loading firearm.
The modern derivation, however, is probably from the "cap gun," a toy firearm using paper "caps" containing Armstrong's mixture or a similar substance to provide the small
explosions.