This is an older and not often heard slang term for an Insane asylum. (Better known as a "Mental Institution").

It is not related to the derogatory term "Cracker"
However it probably IS related to the phrase "Going Crackers".
The phrase "Cracker Factory" was in use by World War II, but is probably older.
Doc. Krabblefratz: "You've been hearing voices telling you the world is ending?
Joe Fruitcake: "Yup."
Doctor Krabblefratz: I think you're ready for the Cracker factory.

Keesha: "Yo dude. I heard you got a new job! What and where?"
Tom: "I'm an attendant in the Cracker factory! Those loonies are drivin' ME loony!

Jacob: That's my uncle Phil in that photo. I don't see him anymore.
Danielle: Why not? He dead or somethin'?

Jacob: Naw, he went nuts a few years back, he's in the Cracker factory now.

Sergey: Dude! I just saw three guys in armor with swords run past the house!
Pete: Awesome! Ya Know what THAT means?
Sergey: That I'm ready for the cracker factory?
Pete: No, dummy, it means that that hot new movie 'Knights in Kombat" Is gonna be showing down at the cinema today!

Screaming, naked, Deranged guy: Where are you @#$%ing taking me?!
Cop: You're gonna get an exclusive INSIDE tour of the cracker Factory.
by Joe Fenderbender May 18, 2015
Get the Cracker Factory mug.
A term used in Central America for a very sterotipical white couple who produce many offspring. Commen Cracker Factorys have 3-12 kids. Cracker Factorys generally are short on money and live in lower-class neiberhoods.
1. Man, that fucking Cracker Factory is moving to our hood.

2. Shit, once all those Cracker Factorys start moving in across the street, they're gonna paint the whole neiberhood white.
by Cory "The Artist" Green April 11, 2007
Get the Cracker Factory mug.
A white woman who has multiple children to collect money and food stamps from the state.
Candi is preggo again. She is a cracker factory!
by Cakes121282 December 28, 2011
Get the Cracker factory mug.
Rebellious or unusual. Characterized by non conformity.
"Hey, that punk kid is really making biscuits in the cracker factory."

"Wow, Rosa Parks was sure making biscuits in the cracker factory."
by Sean L. July 15, 2005
Get the making biscuits in the cracker factory mug.