OMB's Proposed Risk Assessment Guidance
NAS issues its review report
12 Jan 2007 in Regulatory Science, Regulatory Policy, Information Quality, People & Institutions
On January 11, the National Academy of Sciences released its review of OMB's proposed risk assessment guidance.
OMB's proposal is here. The NAS committee report is available for a fee from National Academies Press, and can be read one page at a time for free. There is also a press release.
A few news organizations have published stories about the report, including AP, the Washington Post, the New York Times, and BNA's Daily Report for Executives, a popular subscription-based newsletter.
Of these three, the AP account is the most balanced. The Post's Rick Weiss indulges in hyperbole in lieu of reporting. The Times report by Cornelia Dean contains a significant factual error in the lede. OMB has not withdrawn the proposed guidance; the NAS committee recommended that it do so and issue a different guidance in its place.
Neutral Source managing editor Richard Belzer was quoted in the BNA account by Pat Phibbs-Rizzuto as follows:
Richard Belzer, a consultant on regulatory affairs and former OMB analyst, was disappointed by the academies' report.
"I predict that OMB will respond to the report by returning to its roots in economics and regulatory policy. The [John] Graham Era is over," Belzer said, referring to the former head of OIRA, who issued the draft bulletin.
Here is the full text of our initial comments to BNA. Where useful, links have been inserted to relevant documents:
The Committee supports what OMB tried to do -- improve quality -- but concluded that the OMB authors did a sloppy job of it. Taking that conclusion at face value, it's unfortunate that the Committee did not offer much constructive advice. It's a nonstarter to suggest that OMB should delegate its statutory responsibilities back to the agencies. It's also a bit silly to "strongly" advise agencies with deeply divergent authorities, cultures and interests to develop common technical guidance. They've had decades to do that, and have little to show for it. The Committee clearly doesn't like OMB's proposal, but it just as clearly, it doesn't have any better ideas.
I predict that OMB will respond to the report by returning to its roots in economics and regulatory policy. The Graham Era is over. In economics lies OMB's equilibrium and its dominant, longstanding expertise. OMB has had guidance on benefit-cost analysis since at least 1990. This guidance has gone through several revisions, but Circular A-4 isn't very different from the 1990 edition. Where risk assessment is used as an input to regulatory analysis, OMB will expect agencies to perform it in ways that yield risk estimates compatible with benefit-cost analysis and the 2002 Information Quality Guidelines.
Excerpts from the NAS report:
"The committee strongly recommends that federal agencies addressing similar hazards or risks work together to develop common technical guidance for risk assessment; that would help to achieve the appropriate consistency among agencies in risk assessment practices" (p. 4)
The committee offers OMB the following recommendations to consider in developing a new risk assessment bulletin.
-
After withdrawing the current bulletin and before proceeding further, OMB should produce a description of current agency risk assessment practices and resources and the likely effects (both benefits and costs) of changing those practices.
-
Before mandating substantial changes in agency risk assessment practices, OMB should ensure that sufficient funds and staffing are available on a continuing basis to support the agencies in their risk assessment responsibilities. Adequate staffing and funding are prerequisites to the kind of risk assessment envisioned in the bulletin.
-
OMB should ensure that any government-wide risk assessment bulletin takes full account of and makes allowance for variations among agencies with respect to the types of risk assessments they engage in, the resources they have to devote to risk assessments, and their proficiency in risk assessment generally.
-
Any guidance on risk assessment should provide a definition of risk assessment that is compatible with previous NRC documents and guidelines of other expert organizations; does not include information documents or individual components of risk assessment, such as hazard or exposure assessment; preserves the clear conventional distinctions between risk assessment and risk management; and refers to a process, not a document.
-
OMB should develop goals for risk assessment that emphasize the central objective of enhanced scientific quality and the complementary objectives of efficiency and consistency among agencies evaluating the same or similar risks. The goals should support the production of risk assessments that provide clear, relevant, and scientifically sound information for policy-makers.
-
OMB should develop general principles for risk assessment that are fully consistent with the recommendations provided by previous committees of NRC and those of other expert organizations. The committee recommends that the affected federal agencies develop their own technical risk assessment guidelines that are consistent with the OMB general principles.
-
The committee strongly recommends that discussion of uncertainty and variability, presentation of risk results, definition of adversity, and other similar topics be reserved for the technical guidance to be developed by the agencies.
