It’s not news to anyone that the United States, and the world, have an energy problem. But we’ve had this problem for so long without the roof falling in on us that it has become a kind of psycho-political background noise. Administrations going back to Richard Nixon’s have talked about energy independence, or at least significantly reducing dependence on foreign sources of oil and gas, and yet no one ever seems to address the problem on anything like the scale required. And so the problem has only gotten worse.

Since the roof has not fallen in, optimistic observers in these libertarian-leaning times like to point to the market as our inevitable savior. The market is surely a powerful mechanism: It, not any government policy, is responsible for our ability to get more production out of less energy than was possible in the middle 1970s, and production has responded to price signals in ways that have brought more oil and gas into the market. Nevertheless, the market alone cannot save us from the problem we face now. Thinking otherwise will turn us into the proverbial frog in the slowly heating pot. If we trust only in the market and ignore the wider social ingenuity of which we are capable (and which enabled us to establish the market in the first place), then we’re cooked.

The crux of the oil problem, as an economist might put it, is a market imperfection with very large and intransigent cost externalities. Rather than try to define in lay terms what this means, let a simple illustration suffice: Despite a surge in demand, world oil production has not been able to rise above the peak it reached in May 2005, leading to prices above $100 per barrel. The reason for this is that the severe decline of “easy oil” is imminent.

Easy oil is petroleum that is low in sulfur, near the surface, geographically near markets and exportable. Nearly half of all such petroleum deposits worldwide have been produced—a fact documented by numerous expert sources. Sufficient quantities of good petroleum substitutes, such as natural gas or electricity from other energy sources, are far more expensive, and neither crude discoveries in remote or underwater areas, nor reserves of “heavy oil” in Canada and elsewhere, are any cheaper. At present levels of demand, therefore, most oil economists agree that the global economy will run through the remaining deposits of easy oil much faster than the world consumed earlier ones, resulting in abruptly higher and unpredictable price levels.

This analysis is not hypothetical, despite what some critics contend. The U.S. experience proves it. In 1970, much to the surprise of those who believed that market incentives and a little more drilling would yield sufficient quantities of commercial-grade petroleum, U.S. oil production peaked. It has never recovered, despite huge price increases between 1973 and 1979 and additional intense drilling.

When U.S. oil peaked, Saudi production acted as a price cushion on the world market. Today, only economic recession, initially at least, can provide such a damper on price. The reason is that demand, set against the falloff in new oil coming into the market, has virtually eliminated the margin of excess supply necessary to stabilize prices at moderate levels. For many years, oil experts could point to “spare capacity” in Saudi Arabia and a few other large producers as leverage against price spikes. Of course, spare capacity is only useful as leverage if the relevant governments are willing and able to act as the “swing producer” to stabilize price.

However, as I predicted in 1977, Saudi Arabia used a very different strategy to preserve its price leadership and overall price stability.¹ When the price dipped temporarily as happened between 1981 and 1984, price-cheaters within the cartel took market share away from Saudi Arabia. My model predicted that Saudi Arabia, the low-cost producer, would flood the market with oil, dropping price precipitously, and driving these exporters out of the market so as to re-establish the Saudi market share. The Saudis did this twice, in 1985 and again in the 1990s, thus disciplining the cartel and reclaiming its role as price leader in the world oil market.

Those days are over. The Saudis need revenue to finance their lavish projects and princely appetites, and they now lack much spare capacity in any case. Since no other major producer has significant spare capacity either, the next U.S. administration should expect much greater price volatility.

Higher prices will make the recovery of “hard” oil more economical, and there is plenty of oil in remote, hard-to-extract and hard-to-transport places. So we’re not running out of oil; it’s just going to get a lot more expensive. But price is no small matter, for a prolonged period of sharply higher world energy prices is likely to be accompanied by economic and political turmoil. The outbreak of food riots in more than two dozen countries in recent months is just a glimpse of what lies in store. If the petrochemical inputs in agriculture do not arrest one’s attention, the diversion of food crops to create biofuels will.

We need to be clear about the scope of the problem. It is not just about the price of retail gasoline at the pump, or the cost of heating oil. It is not just about the stagflation potential of sharply more expensive energy inputs in the economy as a whole. Indeed, every aspect of foreign and domestic policy undertaken by the next administration will feel the weight of the oil problem. Consider that the cost of imported oil accounts for about a third of the gargantuan U.S. trade deficit. The trade deficit, in turn, is largely responsible for the decline of the dollar against the euro, causing the cost of imports to rise substantially and setting the stage for post-recession inflation.

That is not all. Since oil is demarcated in dollars, a weak dollar in turn leads oil exporters to raise prices to maintain the purchasing power of their substantial dollar reserves. This raises the cost of agricultural inputs, contributes to the sharp rise in food prices, and dooms the economies of developing countries whose oil import bill is going through the roof, dwarfing any increases in foreign aid and remittances they may be receiving. Since Europeans can now buy dollars to purchase oil at far more advantageous rates than they could before, the oil-dollar nexus helps the EU economy at the expense of our own. High oil prices also help the Russians, the Venezuelans and the Iranians even as they hurt us. In these ways and others, a significantly weaker dollar undermines American influence abroad and hinders the conduct of U.S. foreign policy.

The oil problem also affects U.S. foreign policy more directly. Seventy percent of the world’s exportable oil and natural gas is located in the Persian Gulf. Providing security for the oil wells, pipelines and supply lines in the Gulf has become the singular responsibility of the United States, stretching U.S. air and naval capability thin elsewhere. Risks of supply disruption add a “security increment” to the price of crude oil. The United States therefore ends up paying for energy security twice. Maintaining order in Iraq, the locus of the second-largest oil reserves in the world, is an obvious principal burden. It is not too much to say that the paramount focus of U.S. foreign policy since 1990, involving two wars and an expensive and frustrating occupation of Iraq, has been the oil problem; yet nothing fundamental has been done to mitigate it.

Without a solution to the oil problem, the challenge of global warming will also go unanswered. Though the science is still uncertain, it’s a good bet that much of the heat-trapping gas that causes global warming stems from the burning of gasoline, diesel fuel and heating oil. Motor vehicles generate 72 percent of the nitrous oxides and 52 percent of the reactive hydrocarbons responsible for most air and water pollution. Sulfur oxides from coal-burning electric utilities account for most of the rest. This is a problem that goes beyond the price of hydrocarbon fuels to the question of more environmentally friendly substitutes for them—which brings us to the issue of energy research and development.

Languishing Energy Research

It is not as though the decline of easy oil and the rise in energy prices is any surprise. Economists and others have been warning about this for years. Energy research, development and demonstration (RD&D;) was supposed to fix this problem. It might have, too, but energy RD&D; as a percent of total RD&D; in constant dollars, including the contributions of both the private and public sectors, is only one-fifth of its 1979 levels. One could be forgiven for wanting to accuse the Reagan, George H.W. Bush, Clinton and George W. Bush Administrations of scandalous, criminal neglect of a core national security concern. But in truth the matter is more complicated than that.

Since 1973, huge price volatility has sent all the wrong signals to firms and governments alike, undermining incentives to undertake energy research and development. As soon as the inflation-adjusted price for oil tumbled in the early 1980s, much promising energy research was broken off. Each year since 1979, the United States has spent less on energy RD&D; as a percent of total RD&D.; The blame lies primarily with shortsighted Executive Branch leaders and feckless legislators. But private-sector energy RD&D; has matched the deterioration in public-sector spending. The shocking truth is that total public and private energy research and development expenditure in the United States today is the rough equivalent of the research and development expenditure for a single large pharmaceutical firm (about $3.5 billion). In contrast, we have spent more than $100 billion per year for five years to pay for the war in Iraq. Indeed, the data show that prior Administrations have treated the energy problem far less seriously than they did the race to put a man on the moon.

What the U.S. political system has managed to muster as a response to the problem in recent years is simply embarrassing. The December 2007 energy legislation mandates efficiency increases for appliances, lighting, insulation and corporate average fuel economy of 35 miles per gallon by 2020, a 40 percent increase. There’s nothing wrong with that, and new mandated efficiencies will help us incrementally. But the bill also sets a U.S. production standard for ethanol at 36 billion gallons by 2022, a nearly sixfold increase, including a requirement that a little more than half would come from feedstocks other than corn.

The Federal subsidy to the ethanol industry is flat out bad policy, and it is bad policy that has been connived out of Washington in the service of unadulterated corporate greed. The facts are unarguable. Even if every single kernel of corn grown in America were used for ethanol, that would satisfy only about 12 percent of current American energy needs. And the energy returned on energy invested for corn-based ethanol is low, meaning little if any net energy is produced once corn subsidies and energy expenditures from farming, transportation and distribution are factored in. The “infant industry” argument that the ethanol-from-corn industry will somehow mature, become more efficient, and shake off heavy government subsidies has not a shred of evidence to support it. Moreover, given that foreign-produced ethanol, particularly from sugar cane as opposed to corn, is far more efficient and can be imported far more cheaply than what can be produced at home, why then have we raised trade barriers to importing it? If you guessed “corporate welfare”, you would be right.

Not only is corn-based ethanol a phantom energy source, its environmental and other drawbacks are immense. Corn production guzzles fertilizers, pesticides and herbicides, thus adding to run-offs and environmental damage. Higher corn production has set back the clean-up of the Chesapeake Bay by five years, greatly extended the anoxic “dead zone” at the mouth of the Mississippi River, and polluted ground water supplies just about everywhere. By chasing quick returns, farmers are increasingly ignoring crop rotation, and when they use stalks and leaves for ethanol production instead of turning them over back into the soil, the soil erodes much faster. As corn prices climb, the cost of meat, eggs, milk and other foods increases sharply, as well.

Supporters claim that ethanol will save the equivalent of half the U.S. oil imports from the Persian Gulf by 2022. This is a “truth” that only the oiliest lobbyists and advertisers could utter with a straight face. The actual estimated amount of consumption savings will be about 1.1 million barrels a day, or about 9 percent of the 2006 aggregate volume of U.S. oil imports. Here’s why: America imports more than 12 million barrels per day of petroleum, with Canada being the largest single source of these barrels, followed by Mexico, Venezuela and others. Only a small fraction comes from the Middle East—thus making the ethanol lobby’s claim true as stated but false as intended. That statement could be genuinely true by 2022 only if the number of imported barrels from the Persian Gulf did not increase as cheaper oil disappeared elsewhere—a highly unlikely prospect.

The December 2007 energy legislation will do nothing to significantly alter U.S. dependence on Middle Eastern oil. Fifteen years after its implementation, the United States will likely be importing a larger amount, and a larger percentage, of oil from the Middle East. The bill does nothing to reduce natural gas imports, either. The December legislation can be likened to a tiny Band-Aid slapped over a gaping, self-inflicted gunshot wound.

Goals for January 2013

The United States is not going to ethanol its way out of the energy bind, nor will ethanol help America avoid the broad economic, social and political effects of global energy price shocks. America will not find enough new “easy” oil to do so, either, and it is certainly not going to seize for itself the oil supplies of major Middle Eastern or African countries (as if that would be a cost-effective fix anyway). The only way we can get out of the pot is by innovation, by transcending the present terms of technical and economic reality.

The energy industry is the largest industry in the world, but research investment in it is the smallest of any major industry. Only an effective program of energy research and development can assure a safe port against gargantuan and unsettling future waves of price increases and the security liabilities it will lift up in its wake. Energy innovation can take many forms, however. It can mean improving existing techniques of energy production or lowering demand by means of enhanced efficiency. It can mean fundamentally changing the nature of energy production and distribution through new modes of energy generation. Whatever the form, energy RD&D; must roll back the level of U.S. oil imports as expeditiously and as massively as possible.

In that effort, the next U.S. administration should take a skeptical attitude toward the goal set by the current Administration to reduce oil imports in a full-employment economy by 20 percent in 15 years.² There is nothing wrong with the mandate as stated, but the next president, national security advisor and secretary of energy must avoid four mistakes. First, they must realize that the fastest way to kill a promising RD&D; program is to underfund it in its youth. Second, they must avoid sleights-of-hand. Set against the context of the worldwide decline of easy oil, even higher market prices for oil could occur so rapidly that a 20 percent reduction comes about because of international market pressures rather than anything achieved through presidential initiative. Third, they must not be rigid in their assessments. Demand in Asia and the Third World could rise much faster than anticipated, changing entirely the base on which these mandates rest. And fourth, they must not ignore geopolitical detail. America can make significant progress in the reduction of overall imports without lowering the percentage of imported oil coming from the Middle East.

Having avoided these mistakes, what should the goals of the next administration be?

Three are critical. First, the new administration must take a wide view of the problem and potential solutions to it. It should think beyond encouraging energy technology innovation as such. Advances in materials science relating to the efficient transmission and storage of energy, for example, could be critical to solving the problem. It should also recognize that reducing oil and gas imports is important for reasons beyond geopolitics. The potential effects on the environment and on global health could also have major economic and strategic implications.

Second, the next administration must provide a more reliable commitment to funding if it wants universities and corporations to devote their own resources, personnel and programs to this project. The United States has always succeeded in “big science” when the institutional template connecting government, universities and business is in good working order. Trusting the market alone isn’t good enough, and it never has been. Government, above all, has to be a consistent player. A good example of what not to do comes from the current Administration: Despite listing energy research as a “high priority national need” and pointing to the 2005 energy bill as proof of its sincerity, in that very bill the Bush Administration reduced the research budget by 11 percent. When budgets tighten, Congress, true to the shortsightedness that adorns it always, tends to cut research first. This sort of thing has got to stop. No energy RD&D; program can survive that kind of budgetary zig-zagging.

The third objective ought to be to gradually increase RD&D; funding tenfold by 2018 from the current $1 billion allocation. We are a rich country. We can afford it. What we cannot afford is to be miserly. Only a high level of funding will enable all research facilities to expand in a systematic and committed fashion so as to get serious results by the end of this period. Even at $10 billion per year, research funding in the energy industry would still be below the 2.6 percent average for research funding across all U.S. industries.³ Again, this is a purse-string we cannot afford to cinch too tightly.

Getting There from Here

It is not enough to devise sound goals for energy policy. We need a strategy to achieve them. Fortunately, one element of that strategy is in hand, if barely.

When analysts and engineers confront a problem such as this one, they face certain structural uncertainties. Since we cannot know in advance how quickly and in what form innovations in energy technology will come about, we cannot know exactly how to invest our resources to achieve energy cost reduction and rapid commercialization. But we have learned how to parse similar uncertainties in other policy fields. That is why the one truly innovative aspect of the December 2007 energy bill was the creation of ARPA-E, the new energy-research arm of the Department of Energy.

ARPA-E is based on DARPA, the Defense Advanced Research Projects Agency. For decades, DARPA ensured that the United States had the lead in defense-related science and technology. ARPA-E can do the same for energy-related technology. There are differences between the models, but the similarities are more important. ARPA-E encompasses what we have learned about effective science policy since the days of the Manhattan Project. Its founding is the most heartening development in U.S. energy policy in more than forty years.

But that does not mean that the hard part is over. The Bush Administration has not made ARPA-E a funding priority, and this is a mistake. We must be wary of election-year, feel-good initiatives that lack the money to back them up. And we must recognize the difficulties a new administration faces in shifting research-funding priorities while parrying the jabs of an entrenched bureaucracy. In light of all this, here is a prospective four-year plan for the next president.

Year 1 Agenda: Funding and Organization. Three criteria must guide the funding initiatives of the first year. First, the next administration must identify a dedicated source of income independent of the general budgetary process and the annual goodwill of Congress. Second, these funds must come from a source that does not compete with research funding for other science and technology needs. Third, the funding for energy RD&D; ought to come from energy users, especially petroleum users, since the costs at issue are associated directly with petroleum. Even more specifically, energy RD&D; funds ideally would come from the importation of petroleum since that is the petroleum at risk, not the one-third or so of all crude that is domestically produced. In general, two sources of funding seem to meet these criteria. One is a tax on gasoline. The other is a small tariff on imported oil.

An increase in the gasoline tax is, as economists argue, a transparent and efficient solution. It is readily collected along with other taxes on gasoline. But politicians know that such an increase is also a highly visible and unpopular solution, even if it is only a few cents a gallon. They have proven cowardly almost without exception in their reluctance to ask for new taxes on gasoline at the pumps, especially when petroleum prices are high. In any event, a gasoline tax is not ideal, for it does not distinguish between domestic and foreign oil production. For that reason it will probably always draw the potent wrath of the domestic oil lobby.

A better approach would be to levy a tariff on imported oil, dedicated solely to energy research. Unlike a gasoline tax, this instrument is perfectly focused on the origin of the policy problem, namely, our external energy vulnerability. That focus gives the research tariff an important advantage: It is means tested, in effect. When the level of oil imports declines through the contributions of energy research and development, the tariff diminishes (even an ad valorem tariff diminishes), perhaps even disappears. Because it is an import tariff, the oil exporting countries will help American consumers absorb the incidence of the tariff and the cost of the research. (Yes, they may complain; in response, we can tell them it’s part of our anti-global-warming effort.) And domestic oil producers are unlikely to oppose such a tariff: Not only does it not hurt them in terms of price, it probably gives them a small price benefit.

Year 2 Agenda: Project Assignment. By the second year of the next administration, the institutional structure must be in place to select, manage and implement new research on energy technology. Here the operation of ARPA-E is critical. Those who have designed ARPA-E understand that it must proceed differently than DARPA, an organization with a single client and a mandate stressing technical performance criteria rather than cost. ARPA-E will have many clients throughout the private sector, energy firms of diverse size and purpose, all in competition. And for the most part, technical performance criteria will be less important than cost considerations.

One of the first tasks of ARPA-E must be to assess the overall efficiency and technical promise of each ongoing or proposed project. That will not be easy in a world of proprietary information and litigiousness. Still, some centralization of data and administration is essential. And yet it is the decentralization of decision-making that promotes the diversity and multiplicity of inputs and participation by firm, government laboratory and university—as DARPA has shown.

This is another way of saying that energy RD&D; in the new administration will face a tension between hierarchy and diffused authority. That tension can arise over decisions about big demonstration projects or heavy concentration of research and development funding in a single area. Yet energy RD&D; is not at all like the Manhattan or the Apollo projects that were so characteristic of Big Science. Matching discovery with innovation in energy should not involve establishing an elaborate and costly institutional platform, and scientists, not politicians, must make the decisions regarding the plausibility of innovation and research. Otherwise, the organization will fail. A two-layer administrative hierarchy between budgets and projects, involving project manager and scientist (part of the essence of the DARPA model) is the way to go.

ARPA-E will face another challenge: establishing a balance between the need to identify “niche picks” for RD&D; support and the need to promote radical invention broadly in the energy field. Since research money is scarce even at $10 billon per year, ARPA-E managers must avoid the temptation to go with a few niche picks at the expense of radical, out-of-the-box thinking that could transform the entire energy field. As we have learned from studying the early stages of major innovation in the personal-computer, software and pharmaceutical industries, we must even fund solo scientists, engineers and inventors associated with universities. Choosing niche picks prematurely is like attempting to choose “national champions” in the corporate world: It almost never works.

Finally, by the second year of the RD&D; program, links to transportation and technologies affecting global warming ought to be a centerpiece of the agenda. The administration will have an opportunity to make unparalleled progress not just on the direct costs of the oil problem but also on the indirect external costs affecting automobile use and the environment. Indeed, in some ways these indirect costs are easier for government to attack, since they lie outside the normal market.

Year 3 Agenda: Demonstration Projects. With funding, institutional structure and a research program in place, the administration’s attention ought to turn to establishing demonstration projects. Such projects are crucial to the successful commercialization of innovation because they bridge the gap between the laboratory and the workplace. In particular, demonstration projects are essential when the price of oil is volatile, thus discouraging the private sector from investing in high-cost, high-risk but still promising forms of energy production or application.

The Department of Energy has not always picked winners in selecting demonstration projects. The list of failures is impressively long: the Great Plains coal gasification project; the Clinch River Breeder Reactor; synthetic-fuel projects such as Solvent Refined Coal; Central Solar Power Tower; and various shale projects. Commercialization failed in these cases for several reasons. Under public pressure for immediate results, the Department of Energy began some projects before the technology was mature enough to justify a demonstration project. The gap between private-sector and government mentalities, especially in hierarchical departments like Defense or Energy, often got in the way of technology transfer. Lacking expertise and authorization, the Department of Energy was unable to use new finance-assistance mechanisms such as tax credits, guarantees of purchase or loan guarantees. In particular, the Department couldn’t get the costs down far enough to achieve immediate application by the private sector. While Congress was often supportive of the projects, pork-barreling and efforts to locate a demonstration project in a favored district, combined with restrictions on intellectual property rights and excessive concern about litigation, tended to impede efficient choices. Part of the problem, too, was that Congress simply lost interest in energy when prices temporarily dropped, thus reducing funding and undercutting the effectiveness of some of the project operations.

The next administration can learn from these mistakes, but it can also learn from successes, few though they may be. The Combustion Research Facility at Sandia National Laboratory and other government laboratories led to profitable collaboration, showing that the private sector sometimes found government-sponsored basic research more useful than elaborate demonstration projects that failed to meet commercial cost criteria.

This example also suggests that we can re-imagine the nation’s national laboratory system, arguably underfunded and underutilized since the end of the Cold War, as a key part of an energy technology effort. For example, the labs could specialize in collaborations in which the private sector offers new financial instruments for taking advantage of ARPA-E project successes in return for accelerated access to the latest technology. Labs would thus effectively function as finance-and-applications middlemen between ARPA-E projects and the market—just as industrial engineering companies help industry adapt technological capacities to specific uses.

Year 4 Agenda: Internationalization. Once the new administration has funding, organization and administrative structures in place—as well as a couple of carefully planned demonstration projects that build on promising areas of basic research—it should reach out to the international community. There are two primary reasons for doing so.

First, a number of governments may possess expertise that could be usefully pooled or exchanged with that of the United States. U.S. participation in ITER, the fifty-year, joint fusion project in Cadarache, France, involving the EU and six other countries, is exemplary.

Second, some non-proprietary technology and expertise ought to be transferred to governments in China, India, Brazil, Indonesia and Bangladesh. With large populations and comparatively low income levels, these societies form the vanguard of sharply accelerating energy demand. Technology that would help preempt this demand could be of great benefit globally by reducing energy consumption as a percentage of GDP.

It would also be possible to press China, India and other big emitters of pollutants to partially switch from coal to less environmentally damaging forms of energy production, or to help them burn coal in a more environmentally friendly way. The next administration can thus use greater international cooperation in energy production to make progress on global warming. It makes sense, however, for the United States to demonstrate that it has turned the corner in energy policy before reaching out to others. Only when others see that the American frog has leapt out of the pot will they have a genuine incentive to leap out, too.