Ipsos custodes

24/06/2013 § 3 Comments

The Dominion Post had a spread on the latest concern about the US NSA over the weekend, entitled ‘Prism: Why We Should Worry’.

Before I get to my main point, one digression. If we start from the idea that words matter, what do we call Prism? ‘Spying’ evokes the Cold War and country-based activities that don’t or shouldn’t involve ‘civilians’. ‘Intelligence activity’ begs the question: it assumes that knowledge is being produced before that has been proven. ‘Eavesdropping’ makes it sound like a nosy neighbour, picking up snatches of conversations. ‘Hoovering up metadata’ is closer to the truth, and does lead to the easy-to-use ‘HUM’ acronym. Of course, ‘hoover’ as a synonym for ‘vacuuming’ is culturally specific, so might not work, but the resonance with J. Edgar works, too.

Digression done.

One of my favourite types of economic projects is the cost-benefit analysis. We get to go into an organisation, assess what they have been doing, and come up with some advice on what works and what doesn’t. Done well, we challenge people’s notions about what they have been accomplishing and whether it was worthwhile.

It is not unusual to find limited evidence that a programme or part of a programme has been effective. By effective, I’m not imposing an outside criterion; I mean ‘effective’ by the organisation’s own standards of what they are trying to do. Nothing takes more time than explaining to people that this great project with documentation and logic and funding and internal stakeholders has been a bust. One project I looked at had impact reports about all the uptake by external people who had been involved, but conversations with those actual people suggested that involvement and impacts had been minimal. Another project had a clear logic laid out about how people would move through stages from awareness to assessment to adoption, and there was no evidence in their own files that anyone had actually been progressed through the stages. And on and on.

The internal people really believe in their programmes. They believe in the mission and goals as well as the implementation. They are surrounded by other people filtering information to reinforce the idea that the programme works. Anything that hints at success is seized upon as evidence of the great things they are doing. Anything that suggests otherwise is explained away: the external party, the context, the timing, lots of things can explain failure without implicating the programme itself. Even data — such numbers from their own databases — are rejected as ‘not telling the full story’.

And that is what I read in the articles about Prism: people inside the programme convinced of the value of what they do, no different from everyone else. We’ve stopped 50 attacks! We’ve saved countless lives! It works!

That’s the value of an outside assessment, a cost-benefit or cost-effectiveness analysis or other approach. These watchmen are, after all, only human. Like most other humans, they believe in their own virtue and sincerity, and thus in the rightness of their actions.

Quis custodiet ipsos custodes? Economists, that’s who should do it.

Advertisements

Tagged: , ,

§ 3 Responses to Ipsos custodes

  • So long as I get to define who gets to count as economist for purposes of your last line, I’m totally cool with it. But others might not be.

    There’s at least one NZ-based outfit who I’m sure could produce a cost-benefit analysis claiming that GCSB putting spy cameras in everyone’s bedrooms would prevent $50 billion in social costs. Or would cause $50 billion in social costs. Depending on who commissioned them to do it….

  • dragonfly says:

    I don’t think you need to be an economist or do a formal cost-benefit analysis to see that the limited knowledge of features of terrorists (limited because these people are very rare), combined with the resulting huge number of false positives from a data-mining algorithm, will mean that such a program cannot possibly be cost-effective.

    Eric, you’re not being mean again are you? 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

What’s this?

You are currently reading Ipsos custodes at Groping towards Bethlehem.

meta

%d bloggers like this: