January 27, 2018

COME BACK, ADMIRAL POINDEXTER, ALL IS FORGIVEN:

Intelligence Community Looking At Crowdsourcing For Predicting Geopolitical Events (Mary Louise Kelly, , January 26, 2018, All Things Considered)

NPR's Mary Louise Kelly speaks with Seth Goldstein, a program manager for Intelligence Advanced Research Projects Activity, about the $200,000 prize offered to anyone able to demonstrate accurate forecasting of a geopolitical event via crowdsourcing. [...]

GOLDSTEIN: So this program was about improving on what was then a very limited state of the art in crowdsourced forecasting. We achieved that in that program. We beat the existing state of the art by greater than 50 percent. You all did a story on it.

KELLY: Yeah.

GOLDSTEIN: And taking that method as the state of the art, IARPA is launching this Geopolitical Forecasting Challenge by giving those who are interested access to a data stream based on the method that was so successful there.

KELLY: So here's my question - why does a crowd of individuals stand a better chance of forecasting something accurately than one really smart person sitting there with a classified security clearance who has access to all of the information that the intelligence community has?

GOLDSTEIN: The problem with expert judgment generally is that it's difficult to know in advance which expert is going to make a correct forecast on any particular event.

KELLY: Inevitably, somebody might get it right, somebody might get it wrong.

GOLDSTEIN: Somebody might get it one time and they might get it wrong another time. The idea behind crowdsourcing is that if you assemble a reasonably sized crowd, a large crowd of hundreds or even thousands of people making judgments, the idea of there being any particular directional bias in some aggregate of that judgment is reduced. It's not to say experts can't make accurate forecasts. It's to say if you had to choose a method, this might serve you better.

KELLY: It seems like there's a conundrum inherent in here for you. If crowdsourcing by ordinary civilians can out-forecast, you know, the smartest minds of the intelligence community, if this project succeeds, do you risk putting yourself out of business?

One of W's big mistakes was making more stuff secret, in the wake of the massive failures of 9-11, instead of open-sourcing it all.

Posted by at January 27, 2018 9:13 AM

  

« JOBS WHITE PEOPLE WON'T DO: | Main | WHO WILL RID ME OF THESE TURBULENT POLICE?: »