London's mayor said it was "absurd" to blame multinationals for minimising tax bills using legal loopholes and insisted the fault laid with successive governments.
Chancellor George Osborne is expected to come under pressure in Parliament over Google's controversial £130m deal with the taxman.
Labour counterpart John McDonnell said the public would be "sceptical" about the "relatively small" settlement and called for the National Audit Office to launch an inquiry.
In his column for the Daily Telegraph, Johnson said he largely agreed with calls for loopholes and dodges to be axed and said multinationals must hand over more to UK coffers.
He wrote: "It has never seemed fair that some of these companies - no matter how wonderful the service they provide - should be paying so much less in tax than the high-street tea rooms and bookshops they have pulverised.
"It would be a good thing, both for the UK finances and for the image of these great companies, if they paid more."
Johnson said the "problem does not lie with the firms", insisting "we should recognise that the fault in the whole affair lies with our national arrangements - our own system for not getting a fair whack from the tech giants."
He added: "George Osborne has made progress. The Google payback is a start. We now need to go further. We want, need and deserve these companies somehow to pay more tax in the UK."
Google reached the agreement with HM Revenue and Customs over taxes it has owed since 2005 and will also start to pay tax "based on revenue from UK-based advertisers, which reflects the size and scope of our UK business."
It comes after years of criticism of multinational firms over their arrangements in the UK and across Europe.
The internet giant's sales were valued at £3.8bn in Britain during 2013 but it paid just £20.4m in UK taxes that year.
Between 2006 and 2011 the company's revenue in the UK hit around £12.6bn but its corporation tax payments for the period totalled £11.2m.
Our BQ Bulletin emails will land in your inbox at 7.30am, Monday to Friday, with a mix of the latest local business news, national news, and features to inspire you. Sign up here!
Click here to read our privacy statement