Writing in The Times in the 90s when he was a columnist, said he would return the noose out of "respect for democracy".
He wrote: "Hanging may seem barbarous, but the greater barbarity lies in the slow abandonment of our common law traditions.
"Were I ever alone in the dock I would not want to be arraigned before our flawed tribunals, knowing my freedom could be forfeit as a result of political pressures. I would prefer a fair trial, under the shadow of the noose."
He added that abolishing the death penalty "has led to a corruption of our criminal justice system, the erosion of all our freedoms and has made the punishment of the innocent more likely."
Gove has not not repeated the call to re-establish capital punishment since.
As Justice Secretary he has been tasked with scrapping the Human Rights Act and replacing it with an alternative that is independent of European courts.
The move would bleakly leave us standing alone with Belarus and Kazakhstan - the only other countries in Europe that have chosen to forsake similar laws.
Belarus "suppresses virtually all forms of dissent and uses restrictive legislation and abusive practices to impede freedoms of association and assembly", while in Kazakhstan freedom of assembly is strictly controlled and a restrictive law on religious freedoms remains in force, according to Human Rights Watch.
Gove replaces Chris Grayling, who becomes Leader of the House of Commons.
In other reshuffle news, George Osborne's significance in the new administration was reinforced by his appointment as First Secretary of State - the highest ranking Cabinet minister and effectively number two to the PM.
The award of the title, previously held by William Hague, will fuel expectations that Cameron will not appoint a deputy prime minister to replace Nick Clegg.
Cameron must now draw up a Queen's Speech for the State Opening of Parliament on May 27, which is likely to include measures to drive through the £30 billion austerity package - including £12 billion of welfare cuts - which Osborne says is needed to eliminate the state deficit by 2017/18.
At the same time he will seek to press ahead with plans for the renegotiation of Britain's membership of the European Union in time for an in/out referendum in 2017.
The PM later confirmed Harper was now the Government's Chief Whip.
Eric Pickles, who was Communities Secretary in the last Parliament, said he had not yet heard from Cameron about whether he would be re-appointed or moved.
He told the BBC Radio 5 Live Pienaar's Politics programme: "I wait by my phone for the great man - he can do what he wants."
Downing Street said that among the calls of congratulation had been one from Danish prime minister Helle Thorning-Schmidt - whose husband Stephen Kinnock was elected a Labour MP in the General Election.
"They agreed to meet soon for talks on EU and bilateral issues," a No10 spokesman said.