Jay was born in the New York City borough of Brooklyn to Shirley (Katz) and Samuel Potash. A member of a middle-class Jewish family, he grew up in Elizabeth, New Jersey. He rarely spoke publicly about his parents, but did share an anecdote: "My father oiled his hair with Brylcreem and brushed his teeth with Colgate," Jay recalled. "He kept his toothpaste in the medicine cabinet and the Brylcreem in a closet about a foot away. Once, when I was ten, I switched the tubes. All you need to know about my father is that after he brushed his teeth with Brylcreem he put the toothpaste in his hair." During an interview on the National Public Radio program Fresh Air with Terry Gross, Jay said that possibly "the only kind memory I ever had of my parents" was when they secretly hired one of his idols, magician Al Flosso, to perform at his bar mitzvah. Jay's grandfather, Max Katz, was a certified public accountant and amateur magician who introduced Jay to magic.