Langhorne developed a distinctive economic style that acted as the response half of a call-and-response with singer/songwriters' vocals, often using rapid triplets of notes. The style arose partly as the result of a childhood accident in which he lost some fingers, which limited the range of techniques he could master to some extent, forcing him to concentrate on the accompanist role. When folk-rock came in, Langhorne used an acoustic guitar with a pickup, running it through a Fender Twin Reverb amp that he borrowed from guitarist (and fellow multi-instrumentalist) Sandy Bull. Influenced by Roebuck Staples of the Staple Singers, he would set up a tremolo effect in time with the song. The result was a sound, both acoustic and electric in color, well-suited to the period in which rock and folk music were combining.

Langhorne became a part of the New York folk scene in the early '60s, where he started out as an accompanist to folk singer Brother John Sellers, who worked as an MC at Gerde's Folk City club. As a result of his constant exposure at the club, he began sitting in with numerous Greenwich Village musicians and finding work as an accompanist both live and in the studio. One of his first recording sessions was for Carolyn Hester's first Columbia album in 1961, a session which also included a then-unsigned Bob Dylan on harmonica. Langhorne then played on the few tracks and outtakes from Dylan's second album to use accompanists, 1963's The Freewheelin' Bob Dylan, including the obscure non-LP rock single "Mixed Up Confusion."

Langhorne's biggest fame comes from just a few days of sessions in early 1965, for Dylan's Bringing It All Back Home album. Langhorne is heard throughout that LP, coming especially to the fore on "She Belongs to Me," "Love Minus Zero/No Limit," and "Mr. Tambourine Man." As spelled out in the liner notes to Dylan's box set Biograph, Langhorne is Mr. Tambourine Man. In the track commentary, Dylan is quoted as follows: "'Mr. Tambourine Man," I think, was inspired by Bruce Langhorne. Bruce was playing guitar with me on a bunch of the early records. On one session, (producer) Tom Wilson had asked him to play tambourine. And he had this gigantic tambourine. It was like, really big. It was as big as a wagon-wheel. He was playing, and this vision of him playing this tambourine just stuck in my mind. He was one of those characters...he was like that. I don't know if I've ever told him that." For all the impression Langhorne apparently made on Dylan, he didn't record with him again (other than on the soundtrack of Pat Garrett & Billy the Kid), though he did play live with him at least once, for a 1965 appearance on Les Crane's television show.

Langhorne was much more than an interesting footnote in Dylan's career, though. In the mid- to late '60s he was in the studio all of the time, adding particularly important contributions to the two Vanguard albums by Richard & Mimi Fariña. He made other notable appearances on Tom Rush's first electric album, Take a Little Walk With Me; John Sebastian's first album; Joan Baez's Farewell, Angelina; and numerous other LPs. He also produced Ramblin' Jack Elliott's first major-label album, 1968's Young Brigham. By the early '70s his session work was becoming less frequent, though he continued over the next few decades to work in soundtracks, as a live accompanist, and co-running a recording studio with Morgan Cavett.