Danny Alexander has hit back at defence secretary Philip Hammond and the Ministry of Defence for resisting cuts to its budget.
"In a department where there are more horses than tanks there is room for efficiency savings," the chief secretary to the Treasury told Sky News on Friday.
According to the Ministry of Defence, as of 1 March 2013, there were 494 military working horses maintained by the armed forces.
By contrast the Army is said to have around 200 tanks - although each one is probably more effective in combat than a horse.
In a parliamentary written answer earlier this year, defence minister Mark Francois said horses played an "important role working alongside our armed forces".
"Horses are used for ceremonial purposes throughout the year, such as Changing the Guard, and at annual events such as Trooping the Colour and the State Opening of Parliament. This forms an important part of our national heritage and helps to raise the profile of our armed forces," he said.
Alexander's comments came after the head of the Army warned that any gap between military resources and planned capabilities caused by spending cuts "could become quite dangerous, quite quickly".
Chief of the General Staff General Sir Peter Wall said imposing further efficiencies while the impact of previous deep spending cuts on the armed forces were still being absorbed would be "very disruptive".
The Ministry of Defence under pressure to deliver its share of the £11.5 billion spending cuts across government demanded by George Osborne in his spending review.