Economic “Experiments”

After my last post, some friends stopped by my office with a few questions: “If, as you say, we are conducting a big experiment in spending, will the experiment produce evidence that finally answers the question of whether or not fiscal stimulus works? Why can’t we just compare the economy’s performance during periods of stimulus with its performance during normal times?  Why mess with military spending as Barro and Redlick do, when what we want to know is whether stimulus spending works, not military spending?” (This latter question gets at Harry Moroz’s point too).   

Here is my attempt at an answer:

Let’s start by imagining the ideal conditions to test for the effect of a stimulus. Suppose the distribution of stimulus money were determined not by the political process, but by a scientist. This scientist would probably randomly assign units of observation two groups: a “treatment” and a “control” group. He would use a coin or some other random process to select some regions to receive money and some regions to receive none. Ideally, he would do this over the course of several years, distributing money both during boom and bust periods to see if the economy responded differently. Then, he would compare various measure of well-being (growth rates, unemployment rates, etc.) in times and places that received stimulus (the treatment group) with comparable measures in times and places that did not receive stimulus (the control group). 

Unfortunately for the scientist (fortunately for the citizen), stimulus money isn’t doled out this way. Instead, politicians make some attempt to target the expenditure of stimulus money to hit times and places that are in need (as my colleague, Veronique de Rugy has shown, they aren’t always very good at hitting their target). But this means that it becomes very difficult for the economist to assess, empirically, the impact of fiscal stimulus.

Why? Because economies in times and places that are in need tend not to grow at the same pace as more normal economies. As standard economic theory teaches us, market-based economies have natural recuperative properties. For example, if aggregate demand suddenly falls, causing a contraction, a chain of events is set in motion that helps sow the seeds of recovery. Spending will fall, lowering prices and increasing savings. The lower prices cushion some of the blow, allowing consumers’ dollars to go farther than before and allowing them to spend more than they otherwise would. As saving increases, interest rates fall and business investment picks up. As these processes work their way through the system, the economy begins to heal. Economists famously argue about how effective this process is, but few would deny that there is some truth to this story.

But knowing that this process happens to at least some degree, we can’t simply compare economic growth in times and places that receive stimulus with that of times and places that don’t. Otherwise, instead of picking up the effect of stimulus, we may just end up measuring the natural recuperative abilities of the market economy. Nor, more generally, can we compare economic growth in times and places where governments spend a great deal of money with economic growth in times and places where governments spend little. This is because there is strong reason to believe that causation runs the other way too: when the economy is humming, state and federal coffers are flush with cash and tend to spend more and when times are lean, states have no choice but to cut back spending.

The problem is analogous to that of understanding the impact of police patrols on crime. We would like to measure crime rates in times and places where patrols are sent with crime rates in times and places where patrols are not sent. But, like politicians distributing stimulus funds, police captains don’t randomly pick the areas where they send their patrols. Instead, they try to target patrols to the places and times where they are needed. Thus, a naïve look at the data shows that places with more police patrols tend to have more crime! This clearly doesn’t make sense, but it is what the data show.    

Which gets us to the question: why study military spending when we are interested in stimulus spending? The answer is that it helps solve the statistical problems I mention above. I won’t get into the technical details of two-stage least squares regression techniques (I’d prefer you finish reading the post), but here is the basic gist of the strategy: start by finding some phenomenon that is correlated with the treatment (the treatment here being cops or government spending) but uncorrelated with the outcome of interest (in this case, crime rates or economic growth). If you can find such a phenomenon, you can use it to study the pure, unbiased effect of the treatment on the outcome.

In the case of police and crime ­­­­­­­­­Steven Levitt came up with an ingenious phenomenon to help unravel the real relationship. He accurately surmised that elections might induce elected officials to increase the number of patrols on the street. And since elections are not directly related to the underlying crime rate, this allowed him to obtain an unbiased estimate of the effect of patrols on crime. As you probably guessed, this unbiased estimate showed that, indeed, more police patrols actually lead to less crime.

So what about stimulus? As I mentioned in my previous post, Robert Barro and Charles Redlick use military spending to assess the impact of stimulus spending on economic growth. Military spending is positively related to overall government spending. But it turns out that it isn’t related (positively or negatively) with economic downturns. Thus, it makes an ideal phenomenon to assess the impact of stimulus. As I mentioned, Barro and Redlick found that stimulus spending isn’t stimulative.

Similarly, Lauren Cohen, Joshua Coval, and Christopher Malloy, make clever use of another phenomenon to assess the impact of government spending on economic activity. They rely on the fact that government spends more in Congressional districts whose members are chairs of powerful committees than in districts whose members are just rank and file. Like Barro and Redlick, they find that government spending isn’t stimulative.   

I suspect that right now some clever economist is working on a study of the current stimulus that relies on a technique similar to these. I sincerely hope that it will bring us closer to a consensus on the effect of stimulus. If Barro and the others are correct, we can’t afford to keep throwing good money after bad.