Social housing is now – probably more that at any time in its 120 year history – the focus for controversy. Over the first 80 to 90 years of that period it was seen overwhelmingly as part of the solution (indeed sometimes the only solution) to this country’s housing problems. There was widespread confidence that the provision of more social housing, predominantly but not exclusively built and managed by local councils, was the right way forward.

Over the past 30 to 40 years however, a growing number of voices have questioned whether social housing hasn’t itself become part of the problem. The number of new social homes being built during this period has been way below the levels constructed in the immediate post-war era, and the sale of council housing under the right to buy has significantly diminished the stock. So social housing, provided by councils and housing associations combined, now represents only one fifth of the country’s housing stock compared with one third that were owned by councils alone just 30 years ago. This decline is not universally seen as unfortunate. On the contrary some commentators have even proposed an end to the provision of social rented housing altogether.

Paradoxically, the level of demand for social housing remains very high. In the midst of the most severe downturn in the housing market for decades, the number of people applying to go on council waiting lists has risen dramatically.

So how do we explain this conundrum of large scale demand, but growing doubts about the appropriateness of the response? The answer lies partly in the changed nature of the social housing tenure, and the characteristics of those living in social housing, and partly in the wider economic and social environment in which it is now operating.

Put crudely, social housing represented an aspiration for a substantial proportion of the population for its first 70 to 80 years. It offered higher quality accommodation than most available alternatives and tenants were selected in such a way that many of the poorest and most disadvantaged were excluded.

Over the past 40 years or so, this has changed fundamentally. Social housing is now widely perceived as a ‘residual’ sector that houses only those who are unable to afford what are seen as more attractive alternatives and hence disproportionately accommodating those without work or with a range of disadvantages. At the same time the shortage of affordable accommodation, exacerbated by steep increases in house prices and a persistent undersupply of new homes, means that social housing remains the only viable option for larger numbers of people. But only a relatively small proportion of those on waiting lists get housed. While allocation policies continue, understandably, to give priority to those in greater need, the concentration of the poor and disadvantaged in social housing is inevitably perpetuated and social mix becomes ever harder to achieve. All of this reinforces the process of social separation and stigmatisation of social housing.

These are of course generalisations and as such inevitably over-simplify the much more complex reality in which most residents of social housing developments live. Indeed to a considerable extent the polarisation of opinion in the past 40 years about social housing has contributed to unhelpful caricatures and stereotypes of the tenure. Particularly with the benefit of substantial recent investment in the ‘decent homes’ programme and neighbourhood renewal, many council and housing association tenants feel great pride in their homes and the areas in which they live. They rightly deplore the stigmatizing processes associated with criticisms of the alleged failure of social housing.

Not only do these stereotypes misrepresent the more complex and richer patterns which exist in the real world, they also contribute to two misguided and dangerous policy responses. On the one hand there are those who advocate the end of security of tenure for social housing, seeing it as a transitory tenure to help people through adversity, but not as a basis for long-term occupancy. Take away security of tenure, they say, and this will free up many more tenancies that can be let to people in immediate need – a housing equivalent of the ‘bed blocking’ analysis of the situation affecting hospital beds in the NHS.

What those who advocate this approach fail to recognise is that it will simply intensify the ‘problem’ which they see with social housing – an exclusive concentration of unemployed and disadvantaged people locked in dependency. What chance is there of creating a better social mix, let alone a community spirit and a sense of commitment to the area, if the reward for anyone getting a job or a better income is the receipt of a notice to quit? “Time to move on to make way for someone in greater need” is a recipe for perpetual residualisation of the estate. Ironically the advocates of this policy would be the first to object if their entitlement to occupy a home of their own were to be arbitrarily threatened by some higher authority. “An Englishman’s home is his castle” is the policy they apply to themselves, but wish to deny to their less fortunate fellow citizens.

But equally misguided is the belief that the way forward is the resumption of large scale council home building programmes. Yes, we do need an expanded supply of social and affordable housing and indeed of other housing tenures, but these should not be built in mono-tenure estates. One of the curious characteristics of 20th century housing policy was the adoption of what can only be described as social apartheid. In the 30 years that followed the end of the second world war, we saw the largest scale housebuilding programme in the country’s history. Politicians promised to build ever more homes – and they delivered. Harold Macmillan built 300,000 new homes a year in the 1950s; Harold Wilson built 400,000 homes a year in the 1960s.

These new homes were almost exclusively owner-occupied or council, but they rarely if ever were built together. Each had their own separate location, as if it were unthinkable for people of different tenure to live next door to each other. Yet that has been the norm for most of the 5000 or so years of recorded world history. Not only did people of different economic and social status live in the same street or village in medieval England, they often lived in the same home, with a few animals as well thrown in for good measure! These patterns persisted until the 19th century. Of course wealthier people enjoyed much grander and more comfortable accommodation than others, and within the home ‘upstairs’ and ‘downstairs’ were clearly delineated. But the rule that people of different economic status should live in entirely separate geographical areas did not become the norm until the mass housing programmes of the mid 20th century.

And a very unhappy change of policy it has proved. While council housing remained an aspiration for large sections of the better-off working class, the disastrous consequences of this social apartheid were not fully felt. But once council housing became a residual sector accommodating a disproportionate number of the poorest and most disadvantaged members of society, the dire results of this social separation came home to roost.

That is why the return to mixed tenure communities is such a crucial issue, and why this Fabian pamphlet fulfils such a valuable role in the debate about the future of housing policy. Unlike too many other recent contributions from think tanks which show little or no appreciation of the real-world realities, this publication offers a thoughtful and thorough analysis of the problem of residualisation and social segregation, and the damaging impact this has on people’s life chances. James Gregory also recognises that the problem goes far wider than simply housing policy. Solutions depend on a range of well-integrated responses, addressing education, health, employment, crime and other issues as well as housing.

To an extent we have begun to learn this lesson. The fact that the responsible government department is now known as Communities and Local Government, rather than Housing and Local Government as it was in the immediate post-war era, is an indication of the change in thinking. Housing and planning policy has championed the cause of mixed tenure and balanced communities for more than a decade. And even among the housebuilders, who for most of the 20th century were intransigent about the need to segregate owner-occupation from social housing to retain its value (the infamous Cutteslowe Walls referred to on page 19 of this pamphlet providing a characteristic example), there is growing recognition that well designed mixed tenure developments can succeed socially, economically and environmentally.

In the current recession, funding from the Homes and Communities Agency for social and affordable housing as part of mixed tenure developments is likely to prove a highly attractive proposition to the homebuilding industry. Our challenge is to ensure that this commitment doesn’t evaporate when recovery comes. But even more challenging, as this pamphlet recognises, will be the task of turning round existing mono-tenure estates which suffer from stigmatisation and social segregation. There are good examples of success which demonstrate the scope for transformational change even in very deprived areas. Sensible housing allocations policies which do not reinforce concentrations of unemployed and disadvantaged people in one area have a role to play.

So too does good management and maintenance. Too many ‘sink’ estates started life as model developments but have been allowed to deteriorate. Hence the tendency of older, long established tenants of council housing estates to reminisce nostalgically about how lovely the place was when they first moved in, often contrasting this very unfavourably with its current state. Once the rot sets in and estates get a bad name, the task of returning them to normality is much harder. So active management and prompt and efficient maintenance services have a key role to play.

However, the overwhelming evidence is that the turnaround of a problem estate cannot be accomplished by housing policies alone. When there is widespread worklessness and unemployment, active intervention to assist people to acquire the skills and confidence necessary to secure lasting employment will be vital. Similarly measures to tackle problems of crime, drug abuse and anti-social behaviour may be essential, as well as new opportunities and facilities for youngsters who in the past had too few chances for constructive play or leisure activity. The good news is that an increasing number of more progressive councils and housing associations are recognising the need for comprehensive approaches that incorporate a range of different elements to break down the sense of social separation and isolation that can affect some stigmatised estates.

Building on the perceptive and thorough analysis of the problems this pamphlet concludes with a series of recommendations on actions which can help achieve the objective of normalising areas which have become stigmatised and isolated from the mainstream of society. As with any comprehensive set of recommendations some are more likely to succeed than others, but in the best Fabian tradition this pamphlet sets a clear direction based on a sound evidence base and invites us to respond. There can be few more important political challenges in the coming years and ’In the Mix’ deserves the closest attention from those responsible for defining the policies on which a Labour government should fight the forthcoming election.

Nick Raynsford is MP for Greenwich and Woolwich