With some students paying more than £78 per 50 minute lecture, Willetts said he was "surprised" the time devoted to lectures had fallen.
In a new report, the Conservative minister suggests that the system has become "lopsided" with institutions focusing more on research than on teaching students.
He said that given the growth in the number of students it was "surprising" that the time devoted to teaching had fallen, and argued that the "pendulum has swung too far away".
Willetts' report, for the Social Market Foundation (SMF) think tank, looks at the UK's higher education system in the 50 years since the Robbins Report was published.
That report, led by Lord Lionel Robbins, was published in 1963 and its recommendations led to major reform, including the expansion of universities.
Willetts argues that over the years, universities have focused primarily on research "because that is where the funding and prestige came from."
But in the last 20 years there has not been the same incentives on universities to focus on teaching as there has been on research.
"The pendulum has swung too far away from teaching," the minister says.
Last year, The Huffington Post UK spoke to several graduates about whether they thought their degree was worth the money.
Dom Travers studied biology at Bristol University and has been working in bars since graduating. "My advice would be not to assume getting a good degree from a good university means you're guaranteed a job," he told us.
"I regret not making more of my degree and taking the approach that I had the rest of my life to try really hard with jobs in the holidays."
In the universities open when the Robbins Report was published in 1963, there was a 55:45 split in favour of teaching.
Official data shows that across all the institutions open in 2011/12, the split was 69:31 in favour of teaching.
But Willetts says that these figures mask big differences, and new analysis shows that for the universities that existed in 1963, the split is now 40:60, in favour of research.
"Our analysis suggests that despite decades of change, the pre-Robbins universities appear to be more focused on research now than at any other time," the report says.
"Given the growth in the number of students, even at these older institutions, it is surprising that the proportion of academic time devoted to teaching them appears to have fallen from 55% to 40%".
Willetts later adds: "Looking back we will wonder how the higher education system was ever allowed to become so lopsided away from teaching."
He insisted that changes to funding - including the move to triple fees to a maximum of £9,000 with students paying it back after they graduate and are in work - are an opportunity to "push a real cultural change back towards teaching".
One of the main aims of the Government's higher education reforms is to "place students back at the heart of universities", Mr Willetts says.
He insists that changes such as allowing institutions to recruit as many students as they like with at least an A and two B grades at A-level and giving students more information about courses, job prospects and costs are incentives to improve teaching.
"Students aren't merely buying a degree, as they might a holiday," Willetts says.
"They are engaging in something inherently worthwhile and also investing in their future. The paradox is that unleashing the forces of consumerism with more information for prospective students and funding following their choices is the best way of bringing back traditional academic focus on high-quality teaching."
SMF director Emran Mian said: "David Willetts provides a compelling case for expanding higher education.
"This is our best bet for equipping the UK economy with the skills it needs and improving social mobility."
Sonia Sodha, head of public services at Which?, said: "Today's students are working for fewer hours, are set less work and are receiving less detailed feedback. This raises questions about the quality of some university courses.
"We welcome the Government's proposal to provide more useful information on the academic experience but they must also make sure the Quality Assurance Agency is monitoring long-term trends.
"With soaring tuition fees, it is increasingly important that students are getting value for money from their course."