Analytic confidence

Analytic confidence is a rating employed by intelligence analysts to convey doubt to decision makers about a statement of estimative probability. The need for analytic confidence ratings arise from analysts' imperfect knowledge of a conceptual model. An analytic confidence rating pairs with a statement using a word of estimative probability to form a complete analytic statement. Scientific methods for determining analytic confidence remain in infancy.

In an effort to apply more rigorous standards to National Intelligence Estimates, the National Intelligence Council includes explanations of the three levels of analytic confidence made in estimative statements.

Analytic confidence beginnings coincide with the cognitive psychology movement, especially in psychological decision theory. This branch of psychology did not set out to study analytic confidence as it pertains to intelligence reporting. Rather, the advances in cognitive psychology established a groundwork for understanding well calibrated confidence levels in decision making.

Early accounts of explaining analytic confidence focused on certainty forecasts, as opposed to the overall confidence the analyst had in the analysis itself. This highlights the degree of confusion among scholars about the difference between psychological and analytic confidence. Analysts often lessened certainty statements when confronted with challenging analysis, instead of proscribing a level of analytic confidence to explain those concerns. By lessening certainty levels due to a lack of confidence, a dangerous possibility of misrepresenting the target existed.

The Intelligence Reform and Terrorism Prevention Act of 2004 establishes some guidelines for conveying the analytic confidence in an intelligence product. The summary document states each review should include, among other things, whether the product or products concerned were based on all sources of available intelligence, properly describe the quality and reliability of underlying sources, properly caveat and express uncertainties or confidence in analytic judgments, and properly distinguish between underlying intelligence and the assumptions and judgments of analysts.

...
Wikipedia