Carter was born on March 11, 1917, in Caryville, Florida. While an infant, his mother moved north to Newark, New Jersey and later East Orange, where he was raised and attended Barringer High School in Newark and then graduated at age 16 from East Orange High School after having skipped two grades. He earned his undergraduate degree in political science from Lincoln University (Pennsylvania) and his law degree from Howard University School of Law in 1940, both on scholarship. Carter earned his LLM from Columbia Law School in 1941, after writing an influential master’s thesis that would later define the NAACP’s legal strategy on the right to freedom of association under the First Amendment to the United States Constitution. He then joined the United States Army Air Corps a few months before the United States entered World War II.

In 1944, upon completion of his wartime service in the United States Army Air Corps, Carter went to work at the NAACP Legal Defense and Educational Fund and in 1948 became a legal assistant to Thurgood Marshall. In 1945, he became an assistant special counsel at the LDF. Carter was a lead attorney on Sweatt v. Painte and presented part of the oral argument to the Supreme Court in Brown v. Board of Education. He also worked on many important civil rights cases, including Sipuel v. Board of Regents of Univ. of Okla..

In 1956, Carter succeeded Thurgood Marshall as the general counsel of the NAACP. Over the course of his tenure, Carter argued or co-argued and won twenty-one of twenty-two cases in the United States Supreme Court.  Among the most important cases Carter worked on after Brown was NAACP v. Alabama (1958), in which the Supreme Court held that the NAACP could not be required to make its membership lists public. This removed a tool of intimidation employed by some southern states after Brown was decided, and put into practice the insights into the First Amendment that Carter had gleaned when still a student at Columbia Law School.

In 1968, Carter resigned from the NAACP, along with his entire legal staff, in protest of the firing of NAACP employee Lewis Steele for a critical article he published in The New York Times Magazine. In his autobiography, Carter writes that the NAACP board’s decision to fire Steele over the article was aimed at him, as “an effort to exert control over the general counsel’s office and bring [Carter] in line.”  In recognition of his civil rights achievements, Fordham University School of Law gave Carter an honorary juris doctor degree in November 2004.  In 2004, he was awarded the NAACP’s Spingarn Medal.

On June 15, 1972, Carter was nominated by President Richard M. Nixon to a seat on the United States District Court for the Southern District of New York vacated by Thomas F. Croake. Carter was confirmed by the United States Senate on July 21, 1972, and received his commission on July 25, 1972. He assumed senior status on December 31, 1986, serving in that capacity until his death.  Carter was a co-founder of the National Conference of Black Lawyers (NCBL). He has served as a member of innumerable committees of the bar and the court, and has been associated with a very wide array of educational institutions, organizations, and foundations. He has written extensively about discrimination in the United States, particularly school segregation, and of his longtime friends and colleagues, Thurgood Marshall and Charles Hamilton Houston.  Carter was a member of Alpha Phi Alpha fraternity.  He died of complications of a stroke in 2012.