Why has collective intelligence become such a big deal? With the rise of social media (wikis, social bookmarking sites and socially driven news and content aggregation sites), it seems that everyone wants to get on the bandwagon.

The principle behind collective intelligence is that a conclusion reached in collaboration with and from competition among multiple individuals will be more intelligent than any conclusion reached by an individual, no matter how smart.

Before we can harness the power of collective intelligence, we have to understand a few things.

1. What is collective intelligence?
2. Why do we need collective intelligence?
3. How do we harness collective intelligence?
4. How do we make sure we don’t get collective stupidity?

I recently interviewed Stephen Buckley from MIT’s Center for Collective Intelligence. He used the example of global climate change to explain the necessity of collaboration:

“One of the problems that we have identified, and want to help people study, is global climate change. This is a problem that really spans a whole lot of disciplines. There is the scientific aspect to it, and then there are the economic and political aspects of it. Now that we have classified the existence of the problem, how do we navigate the scientific challenges, the organizational, economic and political challenges, still has to work itself out.”

As Buckley points out, some believe collective intelligence is a magical concept that can be applied to any problem, no matter how simple or complex. And then others believe collective intelligence efforts (like Wikipedia) will fail because the collective ultimately becomes a mob and leads to collective stupidity.

“There are some instances of entries in Wikipedia that aren’t perfect, and those tend to be the entries around which there are controversies. For example there was an article recently, about campaign organizations for two opposing politicians in Connecticut, running for the Senate, were manipulating each other and in fact sometimes deleting them completely, in order to gain advantage.”

Buckley stressed that in order to make collective intelligence work, we need to understand the circumstances that naturally lend themselves to collaboration and collective intelligence. Then we need a set of incentives that can motivate people to participate.

The incentives, of course, will vary but so far it seems incentives don’t even have to be monetary. Most people that participate on Wikipedia do so because they feel the need to make that information freely available to others. The motivation in computer programming (Linux development) is slightly different. Most people contribute to the open source operating system because they have a sense of purpose and as mentioned by Buckley, under the engineering culture, the strongest motivational tool is to design something that will be the object of adulation for one’s fellow engineers.

Incentives can even be built into systems through rewards for positive contributions as well as punishing negative ones. There are several examples of this already in practice: Amazon and eBay feedback systems, Digg and Reddit ranking systems, and so on.

Abuse is one of the biggest problems open source sites face. If someone can interact with a site and be completely anonymous, there is a higher chance that people will be destructive, than in a case where one’s identity is public knowledge.

“Just from experience, when people can be anonymous, and unaccountable for their actions, they tend to be more disruptive then they are when they have to establish their identity. A prime example of this is: Driving to work today, there’s a guy in his car, I don’t know who he is, he gives me the finger and cuts me off. If I bumped into the guy in the street he wouldn’t stand in front of me and give me the finger. Just because he can simply drive away, he behaves badly.”

Wikipedia, for example, is a project that by in large provides good information, but has its limitations because of annonymous abuse.

“There are some instances of entries in Wikipedia that aren’t perfect, and those tend to be the entries around which there are controversies. For example there was an article recently, about campaign organizations for two opposing politicians in Connecticut, running for the senate, were manipulating each other’s and in fact sometimes deleting them completely, in order to gain advantage.”

This does not mean that Wikipedia is a failure, rather that Wikipedia is not perfect, and more specifically, it is not a good tool for conflict management — where there is subjectivity with regards to the content, Wikipedia fails as a tool for content management to harness the wisdom of the crowd.

MIT deals with the problem of abuse by making people request to participate in their project which officially launched a only a month ago. Although they haven’t denied access, the requirement works as a filter to weed out people who have completely malicious intentions.

Understanding the questions helps us to harness this collective intelligence while keeping in mind its limitations.

Muhammad Saleem is a Netscape Navigator and writes on his own blog The Mu Life where he studies the social bookmarking phenomenon.