Graham led the team supporting Sir Howard Davies's Airports Commission and worked on the London Olympics.
He joins from the department for transport, where he worked on the government's high-speed rail strategy.
Osborne said: "I am delighted to appoint Phil Graham as CEO of the National Infrastructure Commission.
"The NIC will provide expert, independent advice to the government on the most pressing infrastructure challenges facing the country.
"Phil's role as CEO will be vital in overseeing this work."
In October the chancellor announced the creation of the NIC to provide analysis of the long-term infrastructure needs of the country.