Born in Dawson City, Yukon, of American parents, he was the boxing and wrestling champion of the United States Coast Guard during his military service, and he kept his burly physique. He toured with theater troupes and appeared on Broadway, before making his Hollywood debut in 1930. He initially played romantic leads, but later was mostly cast as the villain, likely due to his distinctive seemingly coal-black eyes that might be perceived as 'threatening'. He made over 150 films and dozens of TV episodes, as well as writing two plays. His long career in radio included starring in the series Dangerously Yours.
He is remembered for his roles as malevolent Injun Joe in The Adventures of Tom Sawyer (1938), Jonas Wilkerson, the brutal and opportunistic overseer, in Gone with the Wind and as Lamont Cranston, aka 'The Shadow', in the 1940 serial film The Shadow. He also portrayed Oberon in Max Reinhardt's 1935 film adaptation of Shakespeare's play A Midsummer Night's Dream.
From 1959 to 1961, he appeared with Patrick McVey in the 78-episode syndicated television police drama, Manhunt. Jory played the lead role of Detective Lieutenant Howard Finucane. McVey was cast as police reporter Ben Andrews.