An unusual example of risk assessment appeared today in the news: the disclosure of an unclassified summary of the November National Intelligence Estimate (NIE) concerning Iran’s nuclear weapons program. The document, which for obvious reasons is not transparent and reproducible, nevertheless is remarkably clear about the uncertainties which underlie its estimates.
We’re not competent to provide an informed analysis of the content of the NIE. We focus on its information quality aspects: Is the information objective? Does it clearly distinguish the assessment of risk from the risk management views of either its authors or its audience? Here is our take:
- The introduction clearly describes the process used to develop the NIE, and who is responsible for it. The text is prepared by analysts within each respective agency and reviewed by the respective agency head before distribution to other agencies. Differences are subsequently reconciled where that is feasible, and retained in the NIE otherwise.
- The NIE is subjected to a high-level peer review comprised of the heads of the respective agencies. Thus, they are responsible for the content, including the exercise of technical judgment.
- The Scope Note gives a clear description of what questions were addressed and what questions were not.
- The text box describes what its “estimative language” (that is, semi-quantitative probability words) are intended to mean, and how to interpret statements about the level of confidence in these probability estimates. It does not provide bright lines to separate any pair of categories except for distinguishing between categories that more (or less) likely than not.
- The NIE always leaves open the possibility of error, even among key judgments deemed to be remote (at the low end) or almost certain (at the high end).
We do not detect any text in the NIE that can fairly be said to prescribe a policy viewpoint. On this margin, the NIE is among the very best risk assessments we have ever seen. In the environmental health area, for example, risk assessments are infused with policy views such that it is often impossible to determine where science ends and policy begins. (This is a separate issue from the problem that individual experts can look at exactly the same data and reach different subjective judgments about scientific or technical question, such as the likelihood that Iran will develop a nuclear weapon by any specific date.)
One might wish for greater precision (and policy officials including the president surely do), and that might be contained in the actual report, which remains classified. In any case, excess precision is incompatible with information quality principles. It implies greater confidence than actually exists and thus is inherently misleading. Human health risk assessments often are based on data of similar quality, but estimates are typically reported quantitatively, often the output of impressively complex models, with two or three significant figures.
OFFICE OF THE DIRECTOR OF NATIONAL INTELLIGENCE
The Director of National Intelligence serves as the head of the Intelligence Community (IC), overseeing and directing the implementation of the National Intelligence Program and acting as the principal advisor to the President, the National Security Council, and the Homeland Security Council for intelligence matters.
The Office of the Director of National Intelligence is charged with:
- Integrating the domestic and foreign dimensions of US intelligence so that there are no gaps in our understanding of threats to our national security;
- Bringing more depth and accuracy to intelligence analysis; and
- Ensuring that US intelligence resources generate future capabilities as well as present results.
Since its formation in 1973, the National Intelligence Council (NIC) has served as a bridge between the intelligence and policy communities, a source of deep substantive expertise on critical national security issues, and as a focal point for Intelligence Community collaboration. The NIC’s key goal is to provide policymakers with the best, unvarnished, and unbiased information—regardless of whether analytic judgments conform to US policy. Its primary functions are to:
- Support the DNI in his role as Principal Intelligence Advisor to the President and other senior policymakers.
- Lead the Intelligence Community’s effort to produce National Intelligence Estimates (NIEs) and other NIC products that address key national security concerns.
- Provide a focal point for policymakers, warfighters, and Congressional leaders to task the Intelligence Community for answers to important questions.
- Reach out to nongovernment experts in academia and the private sector—and use alternative analyses and new analytic tools—to broaden and deepen the Intelligence Community’s perspective.
National Intelligence Estimates (NIEs) are the Intelligence Community’s (IC) most authoritative written judgments on national security issues and designed to help US civilian and military leaders develop policies to protect US national security interests. NIEs usually provide information on the current state of play but are primarily “estimative”—that is, they make judgments about the likely course of future events and identify the implications for US policy.
The NIEs are typically requested by senior civilian and military policymakers, Congressional leaders and at times are initiated by the National Intelligence Council (NIC). Before a NIE is drafted, the relevant NIO is responsible for producing a concept paper or terms of reference (TOR) and circulates it throughout the Intelligence Community for comment. The TOR defines the key estimative questions, determines drafting responsibilities, and sets the drafting and publication schedule. One or more IC analysts are usually assigned to produce the initial text. The NIC then meets to critique the draft before it is circulated to the broader IC. Representatives from the relevant IC agencies meet to hone and coordinate line-by-line the full text of the NIE. Working with their Agencies, reps also assign the level of confidence they have in each key judgment. IC reps discuss the quality of sources with collectors, and the National Clandestine Service vets the sources used to ensure the draft does not include any that have been recalled or otherwise seriously questioned.
All NIEs are reviewed by National Intelligence Board, which is chaired by the DNI and is composed of the heads of relevant IC agencies. Once approved by the NIB, NIEs are briefed to the President and senior policymakers. The whole process of producing NIEs normally takes at least several months.
The NIC has undertaken a number of steps to improve the NIE process under the DNI. These steps are in accordance with the goals and recommendations set out in the SSCI and WMD Commission reports and the 2004 Intelligence Reform and Prevention of Terrorism Act. Most notably, over the last year and a half, the IC has:
- Created new procedures to integrate formal reviews of source reporting and technical judgments. The Directors of the National Clandestine Service, NSA, NGA, and DIA and the Assistant Secretary/INR are now required to submit formal assessments that highlight the strengths, weaknesses, and overall credibility of their sources used in developing the critical judgments of the NIE.
- Applied more rigorous standards. A textbox is incorporated into all NIEs that explains what we mean by such terms as “we judge” and that clarifies the difference between judgments of likelihood and confidence levels. We have made a concerted effort to not only highlight differences among agencies but to explain the reasons for such differences and to prominently display them in the Key Judgments.
This National Intelligence Estimate (NIE) assesses the status of Iran’s nuclear program, and the program’s outlook over the next 10 years. This time frame is more appropriate for estimating capabilities than intentions and foreign reactions, which are more difficult to estimate over a decade. In presenting the Intelligence Community’s assessment of Iranian nuclear intentions and capabilities, the NIE thoroughly reviews all available information on these questions, examines the range of reasonable scenarios consistent with this information, and describes the key factors we judge would drive or impede nuclear progress in Iran. This NIE is an extensive reexamination of the issues in the May 2005 assessment.
This Estimate focuses on the following key questions:
- What are Iran’s intentions toward developing nuclear weapons?
- What domestic factors affect Iran’s decisionmaking on whether to develop nuclear weapons?
- What external factors affect Iran’s decisionmaking on whether to develop nuclear weapons?
- What is the range of potential Iranian actions concerning the development of nuclear weapons, and the decisive factors that would lead Iran to choose one course of action over another?
- What is Iran’s current and projected capability to develop nuclear weapons? What are our key assumptions, and Iran’s key chokepoints/vulnerabilities?
This NIE does not assume that Iran intends to acquire nuclear weapons. Rather, it examines the intelligence to assess Iran’s capability and intent (or lack thereof) to acquire nuclear weapons, taking full account of Iran’s dual-use uranium fuel cycle and those nuclear activities that are at least partly civil in nature.
This Estimate does assume that the strategic goals and basic structure of Iran’s senior leadership and government will remain similar to those that have endured since the death of Ayatollah Khomeini in 1989. We acknowledge the potential for these to change during the time frame of the Estimate, but are unable to confidently predict such changes or their implications. This Estimate does not assess how Iran may conduct future negotiations with the West on the nuclear issue.
This Estimate incorporates intelligence reporting available as of 31 October 2007.
A. We judge with high confidence that in fall 2003, Tehran halted its nuclear weapons program1; we also assess with moderate-to-high confidence that Tehran at a minimum is keeping open the option to develop nuclear weapons. We judge with high confidence that the halt, and Tehran’s announcement of its decision to suspend its declared uranium enrichment program and sign an Additional Protocol to its Nuclear Non-Proliferation Treaty Safeguards Agreement, was directed primarily in response to increasing international scrutiny and pressure resulting from exposure of Iran’s previously undeclared nuclear work.
- We assess with high confidence that until fall 2003, Iranian military entities were working under government direction to develop nuclear weapons.
- We judge with high confidence that the halt lasted at least several years. (Because of intelligence gaps discussed elsewhere in this Estimate, however, DOE and the NIC assess with only moderate confidence that the halt to those activities represents a halt to Iran’s entire nuclear weapons program.)
- We assess with moderate confidence Tehran had not restarted its nuclear weapons program as of mid-2007, but we do not know whether it currently intends to develop nuclear weapons.
- We continue to assess with moderate-to-high confidence that Iran does not currently have a nuclear weapon.
- Tehran’s decision to halt its nuclear weapons program suggests it is less determined to develop nuclear weapons than we have been judging since 2005. Our assessment that the program probably was halted primarily in response to international pressure suggests Iran may be more vulnerable to influence on the issue than we judged previously.
B. We continue to assess with low confidence that Iran probably has imported at least some weapons-usable fissile material, but still judge with moderate-to-high confidence it has not obtained enough for a nuclear weapon. We cannot rule out that Iran has acquired from abroad—or will acquire in the future—a nuclear weapon or enough fissile material for a weapon. Barring such acquisitions, if Iran wants to have nuclear weapons it would need to produce sufficient amounts of fissile material indigenously—which we judge with high confidence it has not yet done.
C. We assess centrifuge enrichment is how Iran probably could first produce enough fissile material for a weapon, if it decides to do so. Iran resumed its declared centrifuge enrichment activities in January 2006, despite the continued halt in the nuclear weapons program. Iran made significant progress in 2007 installing centrifuges at Natanz, but we judge with moderate confidence it still faces significant technical problems operating them.
- We judge with moderate confidence that the earliest possible date Iran would betechnically capable of producing enough HEU for a weapon is late 2009, but that this is very unlikely.
- We judge with moderate confidence Iran probably would be technically capable of producing enough HEU for a weapon sometime during the 2010-2015 time frame. (INR judges Iran is unlikely to achieve this capability before 2013 because of foreseeable technical and programmatic problems.) All agencies recognize the possibility that this capability may not be attained until after 2015.
D. Iranian entities are continuing to develop a range of technical capabilities that could be applied to producing nuclear weapons, if a decision is made to do so. For example, Iran’s civilian uranium enrichment program is continuing. We also assess with high confidence that since fall 2003, Iran has been conducting research and development projects with commercial and conventional military applications—some of which would
also be of limited use for nuclear weapons.
E. We do not have sufficient intelligence to judge confidently whether Tehran is willing to maintain the halt of its nuclear weapons program indefinitely while it weighs its options, or whether it will or already has set specific deadlines or criteria that will prompt it to restart the program.
- Our assessment that Iran halted the program in 2003 primarily in response tointernational pressure indicates Tehran’s decisions are guided by a cost-benefit approach rather than a rush to a weapon irrespective of the political, economic, and military costs. This, in turn, suggests that some combination of threats of intensified international scrutiny and pressures, along with opportunities for Iran to achieve its security, prestige, and goals for regional influence in other ways, might—if perceived by Iran’s leaders as credible—prompt Tehran to extend the current halt to its nuclear weapons program. It is difficult to specify what such a combination might be.
- We assess with moderate confidence that convincing the Iranian leadership to forgo the eventual development of nuclear weapons will be difficult given the linkage many within the leadership probably see between nuclear weapons development and Iran’s key national security and foreign policy objectives, and given Iran’s considerable effort from at least the late 1980s to 2003 to develop such weapons. In our judgment, only an Iranian political decision to abandon a nuclear weapons objective would plausibly keep Iran from eventually producing nuclear weapons—and such a decision is inherently reversible.
F. We assess with moderate confidence that Iran probably would use covert facilities— rather than its declared nuclear sites—for the production of highly enriched uranium for a weapon. A growing amount of intelligence indicates Iran was engaged in covert uranium conversion and uranium enrichment activity, but we judge that these efforts probably were halted in response to the fall 2003 halt, and that these efforts probably had not been
restarted through at least mid-2007.
G. We judge with high confidence that Iran will not be technically capable of producing and reprocessing enough plutonium for a weapon before about 2015.
H. We assess with high confidence that Iran has the scientific, technical and industrial capacity eventually to produce nuclear weapons if it decides to do so.
|2005 IC Estimate||2007 National Intelligence Estimate|
|Assess with high confidence that Iran currently is determined to develop nuclear weapons despite its international obligations and international pressure, but we do not assess that Iran is immovable.||Judge with high confidence that in fall 2003, Tehran halted its nuclear weapons program. Judge with high confidence that the halt lasted at least several years. (DOE and the NIC have moderate confidence that the halt to those activities represents a halt to Iran’s entire nuclear weapons program.) Assess with moderate confidence Tehran had not restarted its nuclear weapons program as of mid-2007, but we do not know whether it currently intends to develop nuclear weapons. Judge with high confidence that the halt was directed primarily in response to increasing international scrutiny and pressure resulting from exposure of Iran’s previously undeclared nuclear work. Assess with moderate-to-high confidence that Tehran at a minimum is keeping open the option to develop nuclear weapons.|
|We have moderate confidence in projecting when Iran is likely to make a nuclear weapon; we assess that it is unlikely before early-to-mid next decade.||We judge with moderate confidence that the earliest possible date Iran would be technically capable of producing enough highly enriched uranium (HEU) for a weapon is late 2009, but that this is very unlikely. We judge with moderate confidence Iran probably would be technically capable of producing enough HEU for a weapon sometime during the 2010-2015 time frame. (INR judges that Iran is unlikely to achieve this capability before 2013 because of foreseeable technical and programmatic problems.)|
|Iran could produce enough fissile material for a weapon by the end of this decade if it were to make more rapid and successful progress than we have seen to date.||We judge with moderate confidence that the earliest possible date Iran would be technically capable of producing enough highly enriched uranium (HEU) for a weapon is late 2009, but that this is very unlikely.|
1 For the purposes of this Estimate, by “nuclear weapons program” we mean Iran’s nuclear weapon design and weaponization work and covert uranium conversion-related and uranium enrichment-related work; we do not mean Iran’s declared civil work related to uranium conversion and enrichment.