Cline was born in an isolated community in southern West Virginia. As a youth, he was inspired by Fiddlin' Arthur Smith from the Grand Ole Opry. He began his professional career at age 15 when he and brother-in-law/cousin Ezra Cline founded the Lonesome Pine Fiddlers in 1938 and played on the radio in nearby Bluefield. He and his talented brother Charlie were also playing with the Sunny Mountain Boys. Occasionally, Cline did studio work for such musicians as Jimmy Martin and Bobby Osborne, Rex and Eleanor Parker, and Hobo Jack Atkins.

In 1963, he left the Fiddlers for the Stanley Brothers. Following the death of Carter Stanley, Cline became Ralph Stanley's only fiddle player and played on every succeeding Stanley album. On his solo instrumental albums, Cline combined his own vocal sound effects, including sounds of barking hounds and braying mules, with traditional bluegrass songs, finally singing in earnest beginning in 1972. He penned many of his vocal numbers, humorous ditties focusing on family members or himself. He continued to write and perform through the early '80s until he retired in 1993.