America has everything it needs for success in the twenty-first century with one exception: a critical mass of thinkers, analysts and policy entrepreneurs who can help unleash the creative potential of the American people and build the new government and policy structures that will facilitate a new wave of private-sector led growth. Figuring out why so many of our intellectuals and experts are so poorly equipped to play a constructive role — and figuring out how to develop the leadership we currently lack — may be the most important single thing Americans need to work on right now.Regular readers of these posts know that I think that the world is headed into a tumultuous period, and that the United States is stuck with a social model that doesn’t work anymore. Thanks to the wonders of the internet, I don’t need to reproduce those arguments here; readers interested in the gathering storms can look here to see what I mean, and readers curious about the failure of the Blue Social Model can get started here.There’s a lot of work ahead to enable the United States to meet the coming challenges. I’m reasonably confident that we remain the best placed large society on earth to make the right moves. Our culture of enterprise and risk-taking is still strong; a critical mass of Americans still have the values and the characteristics that helped us overcome the challenges of the last two hundred years.
Guild members.But when I look at the problems we face, I worry. It’s not just that some of our cultural strengths are eroding as both the financial and intellectual elites rush to shed many of the values that made the country great. And it’s not the deficit: we can and will deal with that if we get our policies and politics right. And it’s certainly not the international competition: our geopolitical advantages remain overwhelming and China, India and the EU all face challenges even more daunting than ours and they lack our long tradition of successful, radical but peaceful reform and renewal.No, what worries me most today is the state of the people who should be the natural leaders of the next American transformation: our intellectuals and professionals. Not all of them, I hasten to say: the United States is still rich in great scholars and daring thinkers. A few of them even blog.But the biggest roadblock today is that so many of America’s best-educated, best-placed people are too invested in old social models and old visions of history to do their real job and help society transition to the next level. Instead of opportunities they see threats; instead of hope they see danger; instead of the possibility of progress they see the unraveling of everything beautiful and true.Too many of the very people who should be leading the country into a process of renewal that would allow us to harness the full power of the technological revolution and make the average person incomparably better off and more in control of his or her own destiny than ever before are devoting their considerable talent and energy to fighting the future.I’m overgeneralizing wildly, of course, but there seem to be three big reasons why so many intellectuals today are so backward looking and reactionary.First, there’s ideology. Since the late nineteenth century most intellectuals have identified progress with the advance of the bureaucratic, redistributionist and administrative state. The government, guided by credentialed intellectuals with scientific training and values, would lead society through the economic and political perils of the day. An ever more powerful state would play an ever larger role in achieving ever greater degrees of affluence and stability for the population at large, redistributing wealth to provide basic sustenance and justice to the poor. The social mission of intellectuals was to build political support for the development of the new order, to provide enlightened guidance based on rational and scientific thought to policymakers, to administer the state through a merit based civil service, and to train new generations of managers and administrators. The modern corporation was supposed to evolve in a similar way, with business becoming more stable, more predictable and more bureaucratic.Most American intellectuals today are still shaped by this worldview and genuinely cannot imagine an alternative vision of progress. It is extremely difficult for such people to understand the economic forces that are making this model unsustainable and to see why so many Americans are in rebellion against this kind of state and society – but if our society is going to develop we have to move beyond the ideas and the institutions of twentieth century progressivism. The promises of the administrative state can no longer be kept and its premises no longer hold. The bureaucratic state is too inefficient to provide the needed services at a sustainable cost – and bureaucratic, administrative governments are by nature committed to maintain the status quo at a time when change is needed. For America to move forward, power is going to have to shift from bureaucrats to entrepreneurs, from the state to society and from qualified experts and licensed professionals to the population at large.This doesn’t mean that government becomes insignificant. The state will survive and as social life becomes more complex it will inevitably acquire new responsibilities – but it will look and act less like the administrative, bureaucratic entity of the past. The professional, life-tenured civil service bureaucrat will have a smaller role; more work will be contracted out; much more aggressive efforts will be made to harness the power of information technology to transfer decision making power from the federal to the state and local level. All this change runs so deeply against the grain for many American intellectuals that they have a hard time seeing it whole, much less helping make the reforms and adjustments these changes demand.Second, there are the related questions of interest and class. Most intellectuals today still live in a guild economy. The learned professions – lawyers, doctors, university professors, the clergy of most mainline denominations, and (aspirationally anyway) school teachers and journalists – are organized in modern day versions of the medieval guilds. Membership in the guilds is restricted, and the self-regulated guilds do their best to uphold an ideal of service and fairness and also to defend the economic interests of the members. The culture and structure of the learned professions shape the world view of most American intellectuals today, but high on the list of necessary changes our society must make is the restructuring and in many cases the destruction of the guilds. Just as the industrial revolution broke up the manufacturing guilds, the information revolution today is breaking up the knowledge guilds. Guild methods are too expensive given society’s rapidly increasing need for the services they provide; we must drastically raise productivity by re-imagining the way our society makes and distributes the services that, currently, the guilds and the learned professions provide.Guilds are not very good at mass production, and our need for the services they produce has become so great that only a much more efficient production process can serve. Health care, education and legal services are all economic sectors where prices have been rising more rapidly than the overall rate of inflation. These professions must be fundamentally restructured; a Marxist would speak at this point about the proletarianization of the petit bourgeois intellectual professions.Fortunately for the rest of society if not for the guilds, developments in IT and telecommunications now make it possible to reduce costs dramatically in the learned professions. Outsourcing and automation between them can transform the production and delivery of these services. Moreover, the process of disintermediation will enable many Americans to dispense with the expensive services of the professional classes. Basic legal services and advice can increasingly be found, free or at very low cost, on the internet. Many Americans have substituted tax software for accountants; more and more activities once performed by highly paid professionals will be performed by computers and the internet.Ultimately one suspects that services once reserved for elites will be available for the masses, just as the industrial revolution enabled mass ownership of goods that had once been the preserve of small elites. The effect will not only be to raise living standards for most people by improving their access to useful services. It will also be to transfer power and authority from the provider of such services to the consumer. When my grandfather was a doctor, his patients mostly did what he told them to do. He was the expert and there was no rival source of information — especially as many of his patients had little or no formal education and in some cases struggled to write their own names.Today, well-educated patients (many of whom have college and advanced degrees and must routinely master complicated bodies of knowledge in their own work) check with the internet and search the archives of web-based support groups to challenge their doctors’ prescriptions. Increasingly, people will seek and acquire more control over the decisions that shape their lives. People not only want to be more affluent in the future than they are today; they want to be more powerful, less beholden to the men in white suits.Third, there’s training. America today has many technical intellectuals – people like doctors, engineers, and others who are able to carry out complex tasks – and we are extraordinarily rich in specialist intellectuals who have a deep knowledge of a particular subject. Our educational and professional systems are set up to train and support the large numbers of people needed to fill these roles. We are much less effective at teaching and supporting people who are able to master the essentials of many complex subjects, integrate the insights from this kind of study into a coherent social or political vision, and communicate what they have learned to a broad general lay audience. The more complex a society and the more rapidly it is changing, the more need it has for multi-disciplinary, synthesizing intellectuals who are focused on communicating serious ideas to a large audience. Otherwise, a gap grows between the technical and specialist intellectuals and the values and ideas of society at large.There’s another, equally serious problem. In most of our learned professions and knowledge guilds today, promotion is linked to the needs and aspirations of the guild rather than to society at large. Promotion in the academy is almost universally linked to the production of ever more specialized, theory-rich (and, outside the natural sciences, too often application-poor) texts, pulling the discourse in one discipline after another into increasingly self-referential black holes. We suffer from ‘runaway guilds': costs skyrocket in medicine, the civil service, education and the law in part because the imperatives of the guilds and the interests of their members too often triumph over the needs and interests of the wider society.Almost everywhere one looks in American intellectual institutions there is a hypertrophy of the theoretical, galloping credentialism and a withering of the real. In literature, critics and theoreticians erect increasingly complex structures of interpretation and reflection – while the general audience for good literature diminishes from year to year. We are moving towards a society in which a tiny but very well credentialed minority obsessively produces arcane and self referential (but carefully peer reviewed) theory about texts that nobody reads. Political science is becoming more mathematical and dogmatic – while fewer and fewer Americans understand the political foundations and ideas behind American institutions. Similar problems unfortunately exist in many disciplines. Academic discourse becomes more self-referential and remote from public concerns; the public discussion suffers from the absence of the intellectual rigor and historical perspective that serious students and thinkers can bring to it. (The natural sciences are in much less bad shape as the process of empirical verification imposes a certain necessary honesty on the intellectual process, but those who try to connect the sciences to the world of philosophy, policy, theology and politics suffer many of the same problems as intellectuals in the humanities and social sciences.) At the same time, the edifice of academic studies is becoming so expensive and top heavy that except at a relative handful of very wealthy institutions the whole system of tenured teaching appointments looks steadily less sustainable.We can see the same unhappy pattern in knowledge-based American institutions beyond the groves of academe. The mainline Protestant churches have a hyperdeveloped theology, an over-professionalized clergy – and shrinking congregations. The typical American foundation is similarly hyperdeveloped in terms of social and political theory, over professionalized in its staff – and perhaps thankfully has a declining impact on American society because its approaches are increasingly out of touch. With the New York Times in the lead, American journalism was moving in this direction until the rapid onset of financial problems began to force change.So there you have it. The foundational assumptions of American intellectuals as a group are firmly based on the assumptions of the progressive state and the Blue Social Model. Those who run our government agencies, our universities, our foundations, our mainstream media outlets and other key institutions cannot at this point look the future in the face. The world is moving in ways so opposed to their most hallowed assumptions that they simply cannot make sense of it. They resist blindly and uncreatively and, unable to appreciate the extraordinary prospects for human liberation that this change can bring, they are incapable of creative and innovative response.For the sake of the country’s position in the world, for the sake of our economic development , for the sake of American democracy and for the sake of our intellectuals themselves, this needs to change.Right now, too many intellectuals try to turn this into a left/right debate rather than one about the past and the future. There is a liberal case for the radical overhaul of our knowledge industries as well as a Tea Party one. People who want to extend government protections to more groups need to be thinking how government can be radically restructured so it can be more effective at a lower cost. People who want more education to be available for the poor need to think about deep reform in primary and secondary education, and they need to think up ways to reduce the spiraling costs of university education. Those who like the public services provided in troubled blue states like New York, Illinois and California need to redesign state government and find alternatives to the tenured civil service bureaucracies built one hundred years ago. Those who want more access and more equal access to education, to legal services and to medical care need to think about how we can use technology to radically restructure the way we organize and deliver these services — and the more you care about the poor the less you can care about the protests of the guilds.IIn a society like ours, the future is always unexpected, always surprising. The emerging American future will both fulfill and confound the expectations and hopes of people from all different political backgrounds. Because American society is undergoing a chaotic process of accelerating change, no one can really know what will be needed tomorrow – what ideas and what institutions will be useful as we move forward into the unknown.