What "nine annual and long-term outcomes" would you use to measure America's public diplomacy apparatus? State has apparently found them.
The American concept of "public diplomacy" is a strange one. As Americans, we seek a return on our investments. It's in our blood. If there is no clear payback, then there's no clear value and there's no reason to continue. Public diplomacy is no different as we, unique to perhaps the rest of the world, view it as discrete cylinder of excellence that must be measured to prove its worth. Numerous reports as well as historic and recent prominent officials have noted, public diplomacy is presented as something that lacks a domestic constituency and thus support for its programs must be somehow explained.
Operating under the quaint restrictions of distinguishing audiences (foreign and domestic) and an inappropriate focus on means (over effect), the Department of State has implemented a business intelligence system that seeks to measure the effectiveness of its public diplomacy products. The May 2006 GAO report U.S. Public Diplomacy: State Department Efforts to Engage Muslim Audiences Lack Certain Communication Elements and Face Significant Challenges (GAO-06-535) described the adoption of segmented ROI on an element of America's national security that every other country sees as part of overall national efforts.
The department also plans to institutionalize the use of the “logic model” approach endorsed by GAO and others, which could have a significant impact on the department’s program design, implementation, and evaluation efforts. The logic model calls for program managers to define their key inputs, activities, outputs, outcomes, and impact. The head of the Public Diplomacy Evaluation Council has briefed field staff on the logic model using the illustration in figure 4.
The "logic model" is apparently online. From Government Computer News this month:
Government agencies often have an acute case of performance anxiety. Tight budgets force managers to do more with fewer resources, and mandates such as the Government Performance and Results Act demand mechanisms for gauging the performance of tax dollars, products and services, irrespective of cost.
Business intelligence programs can make it easier to gather the statistics and other information for answering the “how are we doing” question for both financial and non-financial performance metrics, then manage the plans and projects that are designed to improve performance.
BI’s raw material is structured data, though the newest software also works on the unstructured data in word processor files and Web pages.
This twofold value is readily apparent in a Business Objects application running in the State Department’s Public Diplomacy office.
Cherreka Montgomery, the office’s performance measurement officer, said it is the first application at State to track performance-based budgeting at the level of program outcomes, not just dollars. “This has never been done for Public Diplomacy,” she said.
The previous system examined 900 performance measures that were slanted toward shortterm outputs, not long-term outcomes. The new Business Objects application reduced the hodgepodge to 15 measurements, nine of which are annual and long-term outcomes, such as surveys that gauge how well foreign audiences understand the United States.
Starting with a dashboard populated by manual extraction from an SPSS database, the agency next plans a much more ambitious performance-based budgeting system, that it will deploy it to seven posts this spring, Montgomery said.
Question: what are your thoughts on what what are the "15 measurements, nine of which are annual and long-term outcomes"?
Think they'll tweak the system to learn from the Chinese Comprehensive National Power?
Thinking graphically, is this how public diplomacy should be perceived? A scientific determination leading to a widget pulled from the shelf, packaged, and sent off to the world. Appropriate? Why or why not?