Browne was the fourth son of a Baptist minister, Sylvanus S. Browne, and his wife Lovie (née Lovie Lee Usher). Born in Woodbury, New Jersey, Browne first attended historically black Lincoln University in Pennsylvania, where he became a member of Omega Psi Phi fraternity and graduated with a bachelor’s degree in 1946. During World War II, he served in Italy with the Negro 92nd Infantry Division and organised the Division’s track and field team. After the war Browne undertook postgraduate work at Middlebury College, Columbia University and at the University of Florence. A middle-distance runner, he won the Amateur Athletic Union 1,000-yard national indoor championship in 1949. He occasionally returned to Lincoln University between 1946-52 to teach English, French and comparative literature. Upon leaving academia he earned a living for several years selling wine for Schenley Import Corporation. In 1956, he quit his job with Schenley to become a full-time professional actor.

Despite the apprehensions of his friends, Browne managed to land the roles of soothsayer and Pindarus in Julius Caesar, directed by Joseph Papp for New York City’s first Shakespeare Festival Theater. More work with the Shakespeare Festival Theater followed, and he voiced an off-screen part as camera operator, J.J. Burden, in The Connection (1961), his first movie role. In The Cowboys (1972), in a role as a camp cook, he leads a group of young cowhands avenging the death of John Wayne’s character in the movie. Browne was determined not to accept stereotyped roles that had routinely been offered to black actors. He also wanted to do more than act and narrate. In 1966, he wrote and made his directorial stage debut with A Hand Is On The Gate starring Cicely Tyson, James Earl Jones, and Moses Gunn. A lifelong bachelor who coveted his privacy, in the turbulent decades of the civil rights revolution Browne avoided participation in public protests preferring instead to be “more effective on stage with metaphor…than in the streets with an editorial”. His stage success brought him to the attention of producer Leland Hayward, and in 1964 he began a regular stint as a cast member on Hayward’s satirical NBC-TV series That Was the Week That Was. Starting in the late 1960s, Browne increasingly became a guest star on TV on both comedy and dramatic shows like Mannix, All in the Family, Good Times, Sanford and Son, The Cosby Show, A Different World, and dozens of other shows. He also was a regular on Soap where he played Saunders, the erudite butler from 1979–81, replacing Robert Guillaume who went on to his own show Benson. Browne later guest-starred on Benson with Guillaume. His appearances on The Cosby Show won him an Emmy Award in 1986 for his guest role as Professor Foster.

He and fellow actor Anthony Zerbe toured the United States with their poetry performance piece, Behind the Broken Words, which included readings of poetry, some of it written by Browne, as well as performances of comedy and dramatic works. Browne found additional success performing in the plays of August Wilson, both on Broadway and the Pittsburgh Public Theater. He was described as having “a baritone voice like a sable coat”, speaking the King’s English with a strong mid-Atlantic accent. To someone who once said Browne sounded “too white”, he replied, “I’m sorry, I once had a white maid.” Four years before his death, Browne narrated a series of WPA slave narratives in the HBO film, Unchained Memories (2003). Browne died of stomach cancer in Los Angeles on April 11, 2007, aged 84.