During the Nazi era, he acted in 37 films and directed four. After the Nazis seized power in 1933, Rühmann did not speak openly about German politics, but instead kept himself as neutral as possible. He never stated a word against or towards the Nazis in the press, although he had been a supporter of democracy. In 1938, he divorced his Jewish wife, who married a Swedish actor and, before World War II broke out, travelled to Stockholm and, as a result, survived the Holocaust. The divorce caused Rühmann to be accused of wanting to secure his career; however, the marriage had probably already fallen apart and some sources say, that he wanted to protect his wife with the divorce. His second wife Hertha Feiler, whom he married shortly after, had a Jewish grandfather, a fact that caused Rühmann problems with the Nazi cultural authorities. Heinz Rühmann retained his reputation as an apolitical star during the entire Nazi era. During the war years, Rühmann increasingly let himself be co-opted by the Third Reich. His role as lead actor in the comedy Quax, der Bruchpilot was supposed to distract the populace from the war. In 1941, under the direction of Reichsfilmkammer president Carl Froelich, Rühmann played the title role in Der Gasmann, about a gas meter reader who is suspected of foreign espionage. In 1944, the premiere of Die Feuerzangenbowle was forbidden by the Nazi film censor for “disrespect for authority”. Through his good relationships with the regime, however, Rühmann was able to screen the film in public. He brought the film to the Führerhauptquartier Wolfsschanze for a private screening for Hermann Göring and others. Afterward, Göring was able to get the ban on the film lifted by Adolf Hitler. A nostalgic comedy of mistaken identities, the film was probably the most popular film of his career and later became a cult hit among college students. As a “state actor”, the highest title for an actor during the Nazi era, Rühmann was not drafted into the Wehrmacht. He did have to take the basic training to become a military pilot, but for the Third Reich, Rühmann was more valuable as an actor and he was spared having to take part in the war effort. In August 1944, Joseph Goebbels put Heinz Rühmann on the Gottbegnadeten list of indispensable actors.

Heinz Rühmann was a favorite actor of Holocaust diarist Anne Frank, who pasted his picture on the wall of her room in her family’s hiding place during the war, where it can still be seen today. The enormous range of Rühmann’s popularity during the Nazi era is illustrated by the fact that he was also a favorite actor of Adolf Hitler and his propaganda minister Joseph Goebbels. Heinz Rühmann had a difficult time resuming his career after the war, but by the mid-1950s, the former comedian had established himself again as a star, only this time as Germany’s leading character actor.[citation needed] In 1956, Rühmann starred in the title role of the internationally acclaimed picture Der Hauptmann von Köpenick (The Captain of Köpenick), the true story of a Prussian cobbler, Wilhelm Voigt, who dressed up as an army officer and took over the town hall in Köpenick. In the days of the German Empire, the army had an exalted status and Voigt embarrassed the army officers and civil servants who obeyed him without question. Rühmann was also the leading man in the 1960 film version of The Adventures of the Good Soldier Schweik, after the novel by Czech author Jaroslav Hašek. He also played the role of Father Brown in three German films during the 1960s. In 1965, Rühmann was brought to Hollywood by producer Stanley Kramer for a supporting role as a German Jew in his all-star movie Ship of Fools.

His wife Hertha Feiler died in 1970 and Rühmann married his third wife Hertha Droemer in 1974. In his later years he also worked as a recitator for German television. His last film was Faraway, So Close! (1993) by Wim Wenders, where he played an old fatherly chauffeur named Konrad. Rühmann died in October 1994, aged 92 years. He was buried in Berg-Aufkirchen, Bavaria. His popularity with German audiences continues: In 1995, he was posthumously awarded the Goldene Kamera as the “Greatest German Actor of the Century”; in 2006, a poll voted him No. 1 in the ZDF TV-show „Unsere Besten – Favorite German Actors“.