Groh was born in Brooklyn, the son of Mildred and Benjamin Groh. He had a sister, Marilyn. He attended Brooklyn Technical High School, then enrolled at Brown University in Providence, Rhode Island, where he graduated Phi Beta Kappa with a degree in English literature. He performed with the American Shakespeare Theatre, then went to Great Britain to attend the London Academy of Music and Dramatic Art on a Fulbright scholarship, and served in the United States Army from 1963 to 1964. On his return to New York City, he studied at The Actors Studio. He made his television debut in silent walk-on parts in two episodes of the Gothic daytime soap opera Dark Shadows on ABC in 1968, but did not garner fame until Rhoda. Groh co-starred in the sitcom Rhoda in which he played Joe Gerard, a New York City building demolition company owner who met and married Rhoda Morgenstern, the best friend of Mary Richards from CBS’s The Mary Tyler Moore Show. The show premiered September 9, 1974 and Joe and Rhoda married in the seventh episode. The network gave the marriage much advance publicity, and the episode proved a ratings blockbuster, having drawn some 50 million viewers to become one of television’s most-watched single programs. In season three, however, the couple separated, and they later divorced. According to Valerie Harper, Groh was written out of the show when the producers decided that Rhoda worked better with its star as a single woman. “We all felt very bad about David not continuing,” she said. The two remained lifelong friends. However, this change was not embraced by the show’s audience and ratings began to decline sharply. Rhoda was eventually canceled in the fall of 1978. Groh starred in his own series, the short-lived Another Day, and went on to make his Broadway theatre debut in Neil Simon’s Chapter Two.

From 1983 to 1985, Groh played D.L. Brock in the ABC soap opera General Hospital, leaving the show to appear in the off Broadway play Be Happy for Me (1986). The New York Times drama critic Frank Rich found Groh “completely convincing as the brash gold-chain-and-bikini-clad Lothario”. Other New York City theater credits include Road Show (1987), and The Twilight of the Golds (1993). On television, Groh appeared in guest roles on such series as Buck Rogers in the 25th Century, L.A. Law, Baywatch, Law & Order, Murder, She Wrote, Melrose Place, The X-Files, and JAG. His film career includes appearances in Two-Minute Warning (1976), Smash-Up on Interstate 5 (1976), Victory at Entebbe (1976), A Hero Ain’t Nothin’ but a Sandwich (1978), The Dream Merchants (1980), The Return of Superfly (1990), Get Shorty (1995), and several independent films. Groh died on February 12, 2008 of kidney cancer in a Los Angeles hospital at the age of 68.