Joe Williams worked as a singer and bouncer in Chicago in the late 1930s and early 1940s. He began singing professionally as a soloist in 1937. He sometimes sang with big bands: from 1937 he performed with Jimmie Noone’s Apex Club Orchestra, and also toured with Les Hite in the Midwest. In 1941 he toured with Coleman Hawkins to Memphis, Tennessee. In 1943 he performed in Boston with the Lionel Hampton Orchestra. He toured with Hampton for several years but never achieved breakthrough success. He sang with Red Saunders at the Club DeLisa in Chicago in 1945, and in 1946 was in New York with Andy Kirk. In the late 1940s Williams was ill and performed little. By October 1950 he was again at the Club DeLisa with Red Saunders, where Count Basie heard him. From 1954 to 1961 he was the singer for the Count Basie Orchestra. Joe Williams rose to national prominence with Basie, who nicknamed him “The Number One Son”. “Every Day I Have the Blues”, recorded in 1955, was one of his many hit recordings. After leaving the Basie band, Williams had a successful career as a soloist at festivals, in clubs and on television. He and Basie remained on good terms and he regularly appeared with the Basie orchestra. He toured and made recordings with many other musicians, including Harry “Sweets” Edison in 1961–62, Junior Mance between 1962 and 1964, George Shearing in 1971, and Cannonball Adderley between 1973 and 1975. He went on a long tour from Egypt to India with Clark Terry in 1977, and toured Europe and the United States with Thad Jones and the Basie Orchestra in 1985. He also worked with his own combos, which between 1970 and 1990 usually included the pianist Norman Simmons, and often had Henry Johnson on guitar.

Joe Williams sang with the Basie orchestra in two films, Jamboree in 1957 and Cinderfella in 1960. He sometimes worked as an actor, and in 1985 took the rôle of “Grandpa Al” Hanks in Bill Cosby’s popular Cosby Show. Williams appeared several times on Sesame Street in the 1980s and early 1990s. In later life Williams often worked in hotels and clubs in Las Vegas, but also sang at festivals and worked on cruise ships. He toured again with the Basie Orchestra, this time under the direction of Frank Foster, who had succeeded Thad Jones as leader of the band. Williams sang with the former Ellington Orchestra drummer Louie Bellson in Duke Ellington’s jazz suite Black, Brown and Beige; in about 1993 or 1994 he again toured with George Shearing. Williams worked regularly until his death in Las Vegas on March 29, 1999, at the age of 80.