Allen was born in the town of Jamesport, Missouri. His father, William Allen, was among the 30 people who originally incorporated Jameson Missouri in 1879 and the doctor who delivered Allen lived in James. However, William Allen also had strong ties to Jamesport where he was town clerk, collector, and constable. Biographies of Allen usually refer to his birthplace as Jamesport. His family later moved to Independence, Missouri.  Allen coached at William Chrisman High School (then known as Independence High School) in Independence, Missouri, the University of Kansas, Baker University, Haskell Institute, and Warrensburg Teachers College in Warrensburg, Missouri.  Allen began classes at the University of Kansas in 1904, where he lettered three years in basketball under James Naismith’s coaching, and two years in baseball. In 1905 he also played for the Kansas City Athletic Club.

At Kansas he was a member of Phi Kappa Psi Fraternity. Allen launched his coaching career at his alma mater in 1907, but took a hiatus after graduating in 1909 to study osteopathic medicine at Kansas College of Osteopathy. Known as “Doc” to his players and students, he was reputed to be a colorful figure on the University of Kansas campus, coaching all sports and becoming known for his osteopathic manipulation techniques for ailing athletes. Allen was a legend in the field of treatment of athletic injuries and benefited a long list of high-profile performers. He also had a successful private osteopathic practice, and many he treated, the famous and otherwise, contend he had a “magic touch” for such ailments as bad backs, knees and ankles. He said he applied the same treatments to “civilians” as he did to his athletes.  His forceful, yet reasonable, disposition helped him become the driving force behind basketball becoming accepted as an official sport in the Olympics in 1936. Allen later coached in the 1952 Summer Olympics, leading the United States to the gold medal in Helsinki, Finland.

He coached college basketball for 50 seasons, and compiled a 746–264 record, retiring with the all-time record for most coaching wins in college basketball history at the time. During his tenure at Kansas, Allen coached Dutch Lonborg, Adolph Rupp, Ralph Miller and Dean Smith, all future Hall of Fame coaches. Among the Hall of Fame players he coached were Paul Endacott, Bill Johnson, and Clyde Lovellette. He also recruited Wilt Chamberlain to Kansas, and even coached former United States Senate Majority Leader Bob Dole. Allen Fieldhouse, the basketball arena on the campus of the University of Kansas, is named in his honor. A banner that hangs in the rafters of Allen Fieldhouse reads: “Pay heed all who enter, beware of the Phog.” Phog Allen was enshrined as part of the inaugural class in the Basketball Hall of Fame in 1959.  Allen also created the National Association of Basketball Coaches, which went on to create the NCAA tournament.