Tag Archives: Government Accountability Office

How Complete Are Federal Agencies’ Regulatory Analyses?

A report released yesterday by the Government Accountability Office will likely get spun to imply that federal agencies are doing a pretty good job of assessing the benefits and costs of their proposed regulations. The subtitle of the report reads in part, “Agencies Included Key Elements of Cost-Benefit Analysis…” Unfortunately, agency analyses of regulations are less complete than this subtitle suggests.

The GAO report defined four major elements of regulatory analysis: discussion of the need for the regulatory action, analysis of alternatives, and assessment of the benefits and costs of the regulation. These crucial features have been required in executive orders on regulatory analysis and OMB guidance for decades. For the largest regulations with economic effects exceeding $100 million annually (“economically significant” regulations), GAO found that agencies always included a statement of the regulation’s purpose, discussed alternatives 81 percent of the time, always discussed benefits and costs, provided a monetized estimate of costs 97 percent of the time, and provided a monetized estimate of benefits 76 percent of the time.

A deeper dive into the report, however, reveals that GAO did not evaluate the quality of any of these aspects of agencies’ analysis. Page 4 of the report notes, “[O]ur analysis was not designed to evaluate the quality of the cost-benefit analysis in the rules. The presence of all key elements does not provide information regarding the quality of the analysis, nor does the absence of a key element necessarily imply a deficiency in a cost-benefit analysis.”

For example, GAO checked to see if the agency include a statement of the purpose of the regulation, but it apparently accepted a statement that the regulation is required by law as a sufficient statement of purpose (p. 22). Citing a statute is not the same thing as articulating a goal or identifying the root cause of the problem an agency seeks to solve.

Similarly, an agency can provide a monetary estimate of some benefits or costs without necessarily addressing all major benefits or costs the regulation is likely to create. GAO notes that it did not ascertain whether agencies addressed all relevant benefits or costs (p. 23).

For an assessment of the quality of agencies’ regulatory analysis, check out the Mercatus Center’s Regulatory Report Card. The Report Card evaluation method explicitly assesses the quality of the agency’s analysis, rather than just checking to see if the agency discussed the topics. For example, to assess how well the agency analyzed the problem it is trying to solve, the evaluators ask five questions:

1. Does the analysis identify a market failure or other systemic problem?

2. Does the analysis outline a coherent and testable theory that explains why the problem is systemic rather than anecdotal?

3. Does the analysis present credible empirical support for the theory?

4. Does the analysis adequately address the baseline — that is, what the state of the world is likely to be in the absence of federal intervention not just now but in the future?

5. Does the analysis adequately assess uncertainty about the existence or size of the problem?

These questions are intended to ascertain whether the agency identified a real, significant problem and identified its likely cause. On a scoring scale ranging from 0 points (no relevant content) to 5 points (substantial analysis), economically significant regulations proposed between 2008 and 2012 scored an average of just 2.2 points for their analysis of the systemic problem. This score indicates that many regulations are accompanied by very little evidence-based analysis of the underlying problem the regulation is supposed to solve. Scores for assessment of alternatives, benefits, and costs are only slightly better, which suggests that these aspects of the analysis are often seriously incomplete.

These results are consistent with the findings of other scholars who have evaluated the quality of agency Regulatory Impact Analyses during the past several decades. (Check pp. 7-10 of this paper for citations.)

The Report Card results are also consistent with the findings in the GAO report. GAO assessed whether agencies are turning in their assigned homework; the Report Card assesses how well they did the work.

The GAO report contains a lot of useful information, and the authors are forthright about its limitations. GAO combed through 203 final regulations to figure out what parts of the analysis the agencies did and did not do — an impressive accomplishment by any measure!

I’m more concerned that some participants in the political debate over regulatory reform will claim that the report shows regulatory agencies are doing a great job of analysis, and no reforms to improve the quality of analysis are needed. The Regulatory Report Card results clearly demonstrate otherwise.

The CAP Act: A Glass Half-Full?

Aaron Merrill is “very pessimistic” about the new “CAP Act” proposal. I’m mildly pessimistic.

First, a few things that I think are good signs:

  1. Unlike PAYGO, the law isn’t just about making sure spending is paid for. It is an attempt to actually limit spending. Also unlike PAYGO, the law targets existing, not just new programs.
  2. In the event that Congress can’t agree on where to cut, the act would trigger automatic, evenly distributed cuts across all categories of spending. This is good. Across-the-board cuts are sufficiently unpleasant to make legislators want to prioritize. But since the cuts will be evenly-distributed, Congress won’t have a strong incentive to scuttle the bill altogether. The last time something like this was tried (the Gramm-Rudman-Hollings (GRH) Act of 1985), a number of programs were exempted from the automatic cuts. These included: Social Security (which was actually in surplus at the time), veterans’ pensions, the earned income tax credit, the president’s compensation, the postal service, welfare payments, and (for the most part) Medicare. This meant that the cuts had to be concentrated on a relatively few programs. David Primo writes (p. 111) that: “At one point, GRH authorized the [office of management and budget (OMB)] to make cuts of over 30 percent to both defense and unprotected discretionary spending.” Faced with this option, Congress found it much more palatable to simply repeal GRH.
  3. It is smart to give OMB—an executive branch agency—the authority to execute the automatic cuts. The goal here is to tie legislators’ hands so that they don’t have to police themselves. GRH initially gave Congress’s Government Accountability Office the authority to execute the automatic cuts. But when a court struck that provision down, the fallback provision was for a joint committee of Congress to execute the cuts. As Primo explained (p. 111), this meant that Congress had “to pass legislation each time it wanted to trigger a sequester.” You can imagine how eager they were to do that. Ultimately, GRH was amended so that the OMB would make the cuts. And to me, that makes sense.   
  4. The CAP Act would permit legislators to avoid the cuts if they can muster a supermajority in both houses. This, too, is a step in the right direction, though I share Aaron’s concern that it is not enough. Normal legislation, of course, can always be repealed with 60 votes in the Senate and a simple majority in the House, plus the President’s signature. By requiring a supermajority in both chambers, this raises the hurdle a bit. But I would be much happier if it were a higher bar (90% of lawmakers?). Of course, even a 90% hurdle could be overcome by repealing the law. In the end, a Constitutional Amendment may be the only way to really bind.    
  5. The cuts are gradual which means that they are more likely to happen. As the GRH experience shows, spending reduction plans that call for cuts that are too dramatic often end up being repealed or ignored.

But it isn’t all sunlight and rainbows. I still have some concerns:

  1. As Aaron points out, the bill is enforced via a “budgetary point of order.” As my friend Jim Musser explained to me, the House Rules Committee can and often does waive all points of order when they consider certain pieces of legislation. In this case, this evidently means that a simple majority of a committee can basically suspend this rule whenever it so chooses.
  2. The reductions are not enough to solve the problem. They want to get spending down to its historical average, but I’d point out that the historical spending average is greater than the historical revenue average and this is a formula of unsustainable debt accumulation.
  3. The reduction in the growth of spending doesn’t start until 2013 and even then, the glide path is so shallow that after 10 years, the national debt will still be well above 100 percent of GDP (assuming revenue remains at its 10 year average).
  4. It is possible that the CAP Act could—perversely—act as an excuse to spend up to the limit. 20.6% may have been the norm for the last 40 years, but I don’t think it is the ideal by any stretch.