Up to 1,000 staff will be employed at HS2 Ltd's offices in Birmingham's Colmore Business District, including engineers and procurement specialists.
The first phase of the line is expected to be completed by around 2026 and will reduce London to Birmingham rail times by 32 minutes.
Connections to Crewe will open in 2027, before the lines to Leeds and Manchester are finished in 2033.
The opening of the headquarters coincides with the final week of petitions relating to phase one being heard by the Commons' select committee examining the project.
Mr McLoughlin said: " HS2 will rebalance our economy, helping make the Midlands an engine for growth.
"The official opening of the headquarters here in Birmingham is proof that HS2 is already boosting jobs and skills, and will continue to do so as we head towards construction.
"This week we have taken another step towards putting spades in the ground as the Commons select committee has been hearing the final petitions on the Phase One Bill.
"I would like to pay tribute to members of the committee, as well as those who have brought forward petitions. We have listened to residents and communities and I am pleased that in many cases we have been able to make the changes they have been calling for."
HS2 Ltd chairman Sir David Higgins said: " The arrival of HS2 in Birmingham will play a vital role in boosting jobs, skills, economic growth and regeneration across the city.
"Britain's second city is at the heart of the HS2 network."
Officials hope the HS2 Phase One hybrid Bill will achieve Royal Assent by December, with construction due to start next year.