Born in New York City, Sullivan was a law student at New York University and Temple University. He fell into acting when in college playing semi-pro football. During the later Depression years, Sullivan was told that because of his 6 ft 3 in (1.9 m) stature and rugged good looks he could “make money” simply standing on a Broadway stage. This began a successful career on Broadway, movies and television.  Sullivan’s first appearance on Broadway was in I Want a Policeman in 1936. He later appeared in The Caine Mutiny Court-Martial on Broadway. One of Sullivan’s most memorable roles was playing a movie director in The Bad and the Beautiful opposite Kirk Douglas. Sullivan starred opposite Bette Davis in the 1951 film Payment on Demand. In 1950, Sullivan appeared in the film A Life of Her Own. His debut in film came in 1943 in Lady in the Dark.  Sullivan replaced Vincent Price in the role of Leslie Charteris’ Simon Templar on the NBC Radio show The Saint. Sullivan only lasted two episodes before the show was cancelled.

In the 1953-1954 television season, Sullivan appeared with other celebrities as a musical judge in ABC’s Jukebox Jury. Sullivan’s first starring television role was a syndicated adaptation of the radio series The Man Called X for Ziv Television in 1956-1957, as secret agent Ken Thurston, the role Herbert Marshall originally portrayed before the microphone. In the 1957-1958 season, Sullivan starred in the adventure/drama television series Harbormaster. He played a commercial ship’s captain, David Scott, and Paul Burke played his partner, Jeff Kittridge, in five episodes of the series, which aired first on CBS and then ABC under the revised title Adventure at Scott Island.  Sullivan appeared again with Bette Davis, on stage, in 1960. Davis and her husband Gary Merrill were touring the US in a theatrical staging of selected prose and poetry of Carl Sandburg, but their marriage was failing, and Sullivan substituted for Merrill.  In 1960, Sullivan played frontier sheriff Pat Garrett opposite Clu Gulager as outlaw Billy the Kid in the NBC western television series The Tall Man (although the series ran for seventy-five half-hour episodes, the one in which Garrett kills Billy was never filmed). In the same year Sullivan had one of his best roles, albeit in a B-Western, as the charming and likeable villain in Seven Ways from Sundown.

In 1965 he appeared in a pinch-hit role for Raymond Burr as Attorney Ken Kramer in the Perry Mason episode,[8] “The Case of the Thermal Thief.” Sullivan appeared in Sam Peckinpah’s Pat Garrett & Billy the Kid (1973) as John Chisum, but his scene was excised from the release print (though later restored to the film). He had a featured role in the 1976 miniseries Rich Man, Poor Man Book II. In additional to The Tall Man, Sullivan also starred in the television series The Road West, which aired on NBC on Monday, alternating with Perry Como), during the 1966-1967 season. Sullivan played the role of family patriarch Ben Pride.  Sullivan guest starred in many series, including The Ford Show, Starring Tennessee Ernie Ford, The DuPont Show with June Allyson, The Reporter, That Girl,” The Love Boat, Little House on the Prairie, The Streets of San Francisco, and McMillan & Wife. He starred in many Hallmark Hall of Fame specials including a highly acclaimed production of “The Price” opposite George C. Scott. Sullivan was consistently in demand for the entirety of his career. His acting career spanned romantic leading man roles to villains and finally to character roles. In his later years, Sullivan had roles in the films, Oh, God! with George Burns and Earthquake. In 1965-66 he guest starred on Twelve o’clock High as Lt. Gen Max Gallagher, father of Colonel Joe Gallagher in the episode “Grant Me No Favor”.  Sullivan has two stars on the Hollywood Walk of Fame: one at 1500 Vine St. for his work in television, and another at 6160 Hollywood Blvd. for motion pictures.

Sullivan was a Democratic Party activist and an advocate for the mentally disabled. He had three children. Sullivan was married and divorced three times. Marie Brown, a Broadway actress, was mother to both Jenny and John Sullivan. On June 25, 1959, he was divorced by Gita Hall, model and actress, the mother of his daughter Patricia Christine, who gave him six grandchildren via her marriage to composer-musician Jimmy Webb. However, the couple reconciled in 1961 before the divorce became final. His third marriage to Desiree Sumara produced no issue.  Sullivan died of respiratory failure on June 6, 1994, in Sherman Oaks, California.