The challenges are endless - land, local communities, planning, materials. But there are some levers that are not only jaw-droppingly simple - they are also extremely effective.
One of these is money. When Tom Cruise shouted "show me the money" in Jerry Maguire he could easily have been a housing professional in a meeting with Treasury. Because whatever it is you want to do, it is easier to do it with financial support than without it.
But if you did scream "show me the money" to a Treasury official then the chances are that, just before they ejected you, they would indeed show you the money. Because no one should be in any doubt that Government spends a lot of money on housing people.
For instance, Government seemingly loves to pay money on rents. That can be the only explanation of creating a system that squeezes supply in a world of rising demand. Over the last 20 years, spending on housing benefit has risen from £16.6billion to £25.1billion.
This is during a time of austerity, and often is used to support people who are working but who still cannot afford to house themselves adequately without the support of the state. In many cases these people are housed, sometimes well, often badly, by private landlords - in fact, the amount of housing benefit going to private landlords has doubled over the last decade. This delivers poor value to taxpayers. Compared to a social rented home, a privately rented home costs the state (or more accurately you and I) an additional £21 every week.
In addition to spiralling rents, Government also has a plethora of schemes to help people into home ownership - whether that is Help to Buy or Starter Homes.
So we have a Government that is forever spending more on rents and schemes for those who want to buy a home. But that doesn't help those working renters spending half of their incomes on rent and still requiring support from the benefits system.
The question we need to ask then is what doesn't government choose to support?
Well the simple answer is what we have traditionally called social housing. I don't care too much about labels but to be clear, what we are talking about is state paying money to build houses that are good quality, secure and genuinely affordable for people who need the most. That used to happen a lot. It doesn't any more. In 2010 Government stopped investing public money in homes for social rent. Construction of these homes ground to a shuddering halt. In 2010/11 just under 36,000 social rented homes were started. The next year work started on just 3000.
This is something that housing associations care about passionately. It is because housing associations care that we continue to build social housing without any support from government. Not one penny. And this is something that we are rightly proud of as a sector. But we are also absolutely clear that in a world of rising demand it simply isn't enough. The report that we publish today shows clearly that it is not enough.
Fundamentally, what we choose to spend our resources on - both as individuals and as a society - tells us a lot about the sort of people we are and the sort of society we aspire to be.
The message of the report is very simple. Smarter investment can lead to better outcomes. The case is compelling. Show us the money for genuinely affordable rented homes and housing associations will get on with the job of building them. Maybe it isn't so complicated after all.
Rob Warm is Head of Member Relations at the National Housing Federation