Attested in "True Grit" by
Charles Portis, 1968, but most likely much older.
The derivation is probably from the percussion "cap," a small
metal cylinder open at one end with an amount of shock-sensitive explosive, usually fulminate of mercury, used to set off the powder charge in a muzzle-loading firearm.
The modern derivation, however, is probably from the "cap
gun," a toy firearm using paper "caps" containing Armstrong'
s mixture or a similar substance to provide the small explosions.