many robots

Information design smartyman Edward Tufte on the recent leaked PowerPoint slide trying to summarize the current situation in Iraq. I've quoted the whole thing because his site doesn't have anchor tags. My highlights in green.

Here are some preliminary comments on the slide "Iraq: I&W of Civil Conflict." My previous
one-word critique ('lousy") wasn't very thoughtful, and here's another try at an analysis.

It appears that "I&W" means "Indications and Warnings." Replacing the acronym in the slide title does pep up meaning to outsiders: "Iraq: Indications and Warnings of Civil Conflict," but maybe it wouldn't fit on the slide.

Only this single slide was leaked (by the military? by DoD?), and so maybe some of the analytical problems are better handled on accompanying slides. Maybe.

Doing competent political analysis, epidemiology, nation-building, and war planning (all of which they're trying to do) in a chaotic situation is impossible, and not much good social science and epidemiology can be expected in chaos and from a military entangled in Iraq. In real-time chaotic situations, the data-collection is going to be sloppy because people have more important things to do. (Recall, for example, the gross errors in counts of 9/11 deaths, as the count went from 6,000 to 2,800 in a few weeks.) And what's taking place is in profoundly different cultures and in different languages from those of the non-local military in Iraq. But sloppy data does not justify analytical sloppiness in reporting. In fact, sloppy data requires greater analytical precision of thought.

The slide reports performance data--a list of phrases, with each phrase accompanied by a measure of performance. This is what the tables in the sports section, mutual fund page, and weather page of newspapers do very well. Those designs are much better for reporting performance data than the slide format here. In sports and stock market tables, each phrase is accompanied by multiple measures of performance, often over varying time-periods. All that won't fit on the slide; this suggests that we should use better reporting method than PP, instead of abbreviating the evidence to fit the slide. As the millions of readers of sports tables each day demonstrate, people can easily manage large tables of information. Thus those being briefed in the military should ask: Why are our presentations operating at 2% of the data richness of routine tables found in the sports section? Let the viewers read and explore through a range of material; different eyes will search for different things in the evidence. The metaphor should be the cognitive style of the sports section (or weather or financial newspaper pages) not the cognitive style of PowerPoint.

There is no cloud of uncertainty or error history associated with the editorializing color. At times, such color codings suggest an excess of certainty.

The Iraq slide above provides some relevant but thin and overly short-run time-comparisons: 2 arrows on the left showing "change since last week," and the "Index of Civil Conflict (Assessed)", which sort of compares "Pre-Samarra" with "Last week" and "Current". And there's a potent time-comparison in words: ". . . violence at all-time high, spreading geographically."

To get more time comparisons on the 14 "Reads" and "Additional Indicators," 14 sparkline time-line histories for the last year (week by week, if available) would be useful as a overall but detailed summary. This would reduce the snapshot tone of the 14 reads and indicators. In our thread Sparklines: theory and practice, there are (at the top of the thread) data tables with sparklines that report daily and longterm financial data; one such table shows 14,000 numbers, many of them accurate to only 2 digits (not much for financial data) under the philosophy of "Try to be approximately right rather than exactly wrong." The short-run weekly jitters and non-reports need to be smoothed out to see (and compare with)the long-run trends. Weekly data cooperate with the notorious recency bias, whereby way too much weight is given to the most recent piece of data, just because it is recent. These weekly reports should be in the context of longer run information to reduce the chances that analysis will be dancing around only with today's news.

The list style, surely one damn thing after another here, is merely descriptive and thus preliminary to policy analysis. That analysis might have been done on the other slides or maybe this report is merely meant as a scorecard. If it is a scorecard, it is grossly impoverished compared to sports, weather, and financial tables.

The current fashion (it, too, shall pass) in government is the stoplight style (green, yellow, red), which tends to dequantify data. With categories of this sort, there's always a concern with how the breaks among categories are chosen and with the meanings of the categories. It will often be better to provide some evidence or numbers, and then an editorial-judgmental color about the number.

The slide contains odd uses of the color-words: for example, a green dot indicating "routine" next to the exciting phrase "unorganized spontaneous mass civil conflict". Shouldn't "routine unorganized spontaneous mass civil conflict" be red-critical? After Hiroshima, would Nagasaki get the routine green dot for nothing different than what happened three days earlier? It looks like weekly wiggles get too much attention, and longrun levels of seriousness too little attention on this slide, as chaos becomes routine week by week and bit by bit. Monthly rather then the sketchy weekly reports might be better for policy analysis. Or at least provide a monthly aggregations over a period of many months (even the entire war) in a scorecard along with the weekly incidents.

The leaking of the slide makes a point about the differences between the government's secret analysis and the public reports by the Administration, a common theme of the insider books on Iraq policy-making (most recently Colin Powell's book). At some time, "reality must take precedence over public relations," as Richard Feynman remarked about the shuttle Challenger accident.

A good many comments by our contributors are on-point but are not taken into account here.

Our thread PowerPoint does rocket science--and better techniques for technical papers has many contributions that are relevant to the Iraq slide. In particular, note the measurements, definitions, and comparisons to standards in the customer scorecard in the "Report of the South Central Connecticut Regional Water Authority" (near the end of the long thread).

Finally, the over-riding metaphor of all this--"the war on terror"--is a big conceptual problem. Once it's a war, then it almost necessarily invokes large-scale military action and searching for a locale (some place, any place, Iraq) for large-scale military action. But terrorists are more like the Mafia or gangs than they are like armies. Perhaps a better metaphor is that terrorism is largely a law-enforcement problem (requiring focused and clandestine local action, informants, endless detective work, detailed knowledge of the local languages and cultures).

Among the grand truths about human behavior, surely the principles of "the unintended consequences of purposive social action" and "it's more complicated than that" are among the top five. Sometimes unintended consequences are largely virtuous or benign (as in market allocation of goods and services if externalities are mitigated) and sometimes the unintended consequences are appalling. That's because it's more complicated than that.

-- Edward Tufte, November 1, 2006

So it seems that at least some of the folks in our government don't think everything in Iraq is going great as the people most willing to stand up aren't waiting for us to stand down.



In a Many Robots exclusive I've received from an anonymous source the Presidential summary edition of this slide as included in his daily intelligence briefings.