Sherman consequently turned to television, creating and producing the long-running quiz show I've Got a Secret. A tenure as the writer-producer of The Steve Allen Show followed, but when the series ended in 1961, Sherman found himself on the unemployment line. After signing a contract with Warner Bros., he released the parody collection My Son, the Folk Singer in 1962. To the shock of the recording industry, radio quickly picked up on the album despite Sherman's obscurity as a performer; according to legend, even President John F. Kennedy was spotted in a hotel lobby singing the cut "Sarah Jackman" (a parody of "Frere Jacques"), further boosting the record's popularity.

Ultimately, My Son, the Folk Singer topped the charts, and spawned a cottage industry of copycat releases. Nonetheless, Sherman remained the unquestioned king of the parody hit, and in late 1962, he returned with a follow-up, My Son, the Celebrity, which, like its predecessor, reached the number one spot. 1963's My Son, the Nut was even more successful, topping the charts for eight consecutive weeks on the strength of the Top Five novelty hit "Hello Muddah, Hello Fadduh," a summer camp-themed take on Ponchielli's 1876 composition "Dance of the Hours."

If, as legend dictates, President Kennedy helped establish Sherman as a star, he also inadvertently contributed to the comedian's drop-off in popularity: following Kennedy's assassination in November, 1963, the nation became serious and solemn, with little interest in the breezy fun offered by song parodies. Released in early 1964, Sherman's fourth album, Allan in Wonderland, reached only number 25 on the pop charts; issued later that year at the height of Beatlemania, the concurrent For Swingin' Livers Only! and Peter & the Commissar (recorded with Arthur Fiedler & Boston Pops) fared even more poorly, with the latter record failing even to crack the Top 40.

1965's My Name Is Allan was his last chart effort, reaching only number 88. Still, Sherman soldiered on, recording Live in front of a Las Vegas audience. After 1966's Togetherness, he was dropped by Warner Bros., effectively ending his career as a performer. After publishing an autobiography, A Gift of Laughter, Sherman died in California on November 21, 1973. He was just 48 years old.