Johnny Williams played bass with such swing-era musicians as Coleman Hawkins, Billie Holiday, Teddy Wilson, Johnny Hodges, and Tab Smith, among others. Williams studied violin in his youth, before switching to tuba. He played in a marching band and at the Palace Theater in Memphis. Williams then moved to Atlanta with the intention of attending college. Instead, he became a professional musician, taking up the double bass and joining the Graham Jackson Orchestra in 1930. In the early '30s, he played in territory bands and with pianist Billy Kyle in Philadelphia and vocalist Tiny Bradshaw in Baltimore. He moved to New York in 1936 and worked with Lucky Millinder's Mills Blue Rhythm Band. He also played with pianist/bandleader Claude Hopkins, then joined saxophonist Benny Carter's band around 1937. Soon after, Williams recorded the first of many sides with Holiday, and also worked with trumpeter Frankie Newton and tenor saxophonist Hawkins. He left Hawkins and joined Louis Armstrong, staying with the trumpeter until 1942 and working with him occasionally thereafter.