The earliest quotation in the Oxford
English Dictionary for this sense of "drift" is dated 1526: "Harde it is . . . to perceyue the processe and dryfte of this treatyse."
Found on "Phrases dot org"
The speaker may be having trouble
communicating with their audience and not knowing how to better explain.
This is similar to "do you feel me?" or "do you get it?".
Otherwise there is a certain mentality and a certain consciousness along with what the speaker & the audience are generally known to
understand.