Saturday, July 22, 2006

The Problems with Digg.com


Don't get me wrong Digg.com is very popular but popularity does not equal intelligent content or even good content. I've heard different people make claims that crowds can make good, intelligent decisions. That is obviously subjective depending on who is actually in the crowd making these decisions. A crowd of people with PhDs is going to get very different results as opposed to a crowd of high school drop outs. So the theory of the crowd mentality known as "Group Think" is flawed from the start. No one knows the actual intelligence level of the members of Digg, their age or educational level. This makes the site as scary as the content on Wikipedia. Mobs of morons through sheer numbers can make ridiculous or incorrect content popular and worse believed to be correct.

Pseudo-intellectuals (Defined)
1. a person exhibiting intellectual pretensions that have no basis in sound scholarship.
2. a person who pretends an interest in intellectual matters for reasons of status.

Pseudo-Intellectuals can become dominant on these sites since they live to further their status. The lesser educated mobs need someone to follow which creates this absurd hierarchy of the blind leading the blind. Which is what a few have learned to manipulate through sensationalism, popularity and aggressive determination. These "content controllers" apparently control almost all of the content on Digg.com.

Top 100 Digg Users Control 56% of Digg's HomePage Content

When folks think of Digg, they're often misled into believing that the content seen on the homepage is representative of what a wide base of Internet users think is news-worthy and important. The numbers tell a different story - that of all stories that make it to the front page of Digg, more than 20% come from a select group of 20 users.

I don't fault them, I fault the Digg.com system. Why should educated people, knowledgeable in a certain field waste their time arguing with people that may be 13 years old on a topic that the 13 year old has no understanding of? The point is they shouldn't have to. 13 year olds don't write encyclopedias... oh wait maybe they do (Wikipedia), heh. Seriously why is someone 13 years old given the same content control as a PhD holder? Do we really want content delivered to us by mindless mobs of people or even worse the ones who know how to manipulate the mob? Well maybe marketers do so they can follow popularity trends. But as for deep, intellectual content it is just not possible for it to exist on these sites long term. Simply because the few who are really capable of delivering good, accurate content can so easily be out voted by the mindless mobs. Thus they do not stick around and the Pseudo-Intellectuals survive with their herd in tow. There is no "wisdom of the masses" on Digg.com only the Mob Mentality.

Notoriety
Digg's initial claim to fame of breaking a story about Paris Hilton says it all. In this Interview Kevin explains:
It was around February or March of 2005 that we really started to see the site take off. One of the things that pushed us into the mainstream was the Paris Hilton cellphone hack, in which her cellphone images and data were stolen and posted online. Someone posted the story on digg late one night and because of the popularity of the story it was quickly dugg onto the homepage. We were then indexed by Google and Yahoo that night under the keywords "paris hilton cellphone hack" and when we woke up the next morning, we saw our site was crawling along… because there were hundreds of thousands of people that were typing those keywords in - and the first site that would come up was digg. So it was a really strange incident at the time, but it showed us the power of using the masses to break news stories.
This makes Digg.com nothing but a mob ruled online tabloid.

Bias
Kevin Rose is only 29 and so far has what looks to be a highly successful startup in Digg.com, yet his attitude of liking things "idie and cool" (or what he perceives to be either) allows his site to take on ridiculous biases. Take the Technology categories on the left of Digg.com. You have categories for Apple and Linux/Unix yet no category for Microsoft or Windows?


So shunning 95% of your audience makes sense? Maybe if you are delusional and really believe all those Firefox users that you have helped convert to the Religion are running Firefox on Linux and not Windows. Lets come back to reality:

Yes that is right Open Source fanboys, Linux has a ridiculous 0.38% market share even with the constant bombardment of related stories to the Digg.com front page. On any given day you are almost guaranteed multiple stories relating to Linux or Open Source. Take Today for instance, you have 3 out of 15 top stories mentioning Linux:



I mean what is this the new Slashdot? It sure looks that way:

If the Slashdotters have their way, there will be only one way of thinking for the future of Digg.com = Open Source. And anything that stands in their way will be burried. You have been warned.

Gaming the System
Gaming Digg is really easy. Digg content appears on the front page when they are Dugg roughly 30-50 times. You simply need that many people coordinated or access to that many individual IP addresses. You can now gain a large control over the content on Digg. You have the ability to promote hand picked stories to the front page and bury content as you wish. What happens when botnet owners get around to writing specific software to utilize their army of individual IP addresses to create Digg accounts and then strategically promote content? Scary stuff, well for Digg anyway.

Spam Control or Censorship?
Theories that sites that get "Dugg" and rely on advertising revenue get hurt due to excessive bandwidth bills in comparison to lost ad revenue are flawed. Digg.com is highly syndicated and whether the initial Digg mob generates ad revenue is irrelevant to the the masses of users incoming from other sites that link to the same article due to syndication. Kevin Rose in a recent interview even mentions that steps are necessary to control spamming on Digg.com. Which means that getting "Dugg" to the front page can obviously be profitable if so many people are even trying it. But this anti-spamming system can also be abused to get legitimate sites blacklisted. Kevin claims 95% of the spam control is user driven. Which means if enough of the mob doesn't like you, you are done. Try submitting www.FirefoxMyths.com to Digg and see what happens to your account. It appears the whole http://mywebpages.comcast.net domain is banned due to this. Fair? Only to someone as biased as Kevin Rose and his staff. I wonder how long before Popular Technology.net gets censored on Digg.com? It shouldn't be too far off.

Followup Links
I will be spending time tracking down all the good stories I can to compile a list here that criticizes Digg, stay tuned...

Digging The Madness of Crowds
Digg Army: Right in Line
Digg Corrupted: Editor's Playground, not User-Driven Website
Responding to Kevin's Non-Response Post
Digg.com Acts Like A Bully
Summing collective ignorance
Digg, spam, and most popular lists
Digg and the So-Called “Wisdom of Mobs”
Suspicious Digging
Blog Sensationalism
Why is Digg Censoring an Article on Defeating Censorship?
Buried Story Shenanigans At Digg
Gaming Digg: the KoolAidGuy saga

8 comments:

James said...

Andrew,

Interesting stuff. I've just started using DIGG with my blog a couple days ago and am curious to see how it turns out.

Your fascination with PhDs amuses me, however. (Full disclosure: I have one.) Given that most of the content on DIGG and similar sites is media reports and the like, I'm not sure why a PhD would have any particular advantage in evaluating them.

Further, most of the studies of Wikipedia I've seen shows it just as accurate as Encyclopedia Brittanica, which has long been the gold standard. While EB is written by experts, it's not updated that often and is therefore dated. And even brilliant scholars make mistakes. As a general rule, 13-year-olds and high school dropouts lack the interest to work on Wikis, leaving informed amateurs and actual experts alike in the forefront.

Andrew said...

James,

I used PhDs as an example of educational excellence. Simply reaching that level required individual research and scholarly discipline. Something I have seen grossy lacking on both Digg.com and Wikipedia. Someone with a PhD is much more likely to submit and identify meaningful content to these sites and less likely to fall for sensationalized unsubstantiated posts.

I have yet to see an evidence that proves otherwise my experience with sites like Wikipedia.

I believe you are being far too generous with thinking that informed amateurs and actual experts (both few and far between) have a large say in the majority of content on these sites. While brilliant scholars can make mistakes their mistakes in no way relate to the sheer ridiculous content or changes to content (Wikipedia) that can make it to these sites.

Your ideals I too held of these sites for a very, very brief time.

da5id said...

Here is another article to add to your list:

http://digg.com/software/An_Outsiders_View_of_Digg_There_is_a_Flaw_in_the_System

Andrew said...

Yes it happened and continues to happen to Wikipedia. Digg is only going to amplify the problem since popularity is more important than Intelligence. But overall these sort of "social" sites are relatively new in scope and popularity.

Unknown said...

James, after joining Digg, I have to say your dead on the money. Is there no hope for intelligence over ignorance on the web? or will the truth continually be buried by some high school drop out who can "sound" like they know what their talking about.

Unknown said...

Your really obsessed with PHDs and stuff, if you really thing that this matter, think of the bloggers, they've also amateurs and may or may not have a PHD. Also, many of the great things that existed weren't made by guys holding PHDs, even many things outside the computer universe weren't made by PHD guys but by simple normal people who cares about them.

Andrew said...

No you failed to read...

"I used PhDs as an example of educational excellence. Simply reaching that level required individual research and scholarly discipline. Something I have seen grossy lacking on both Digg.com and Wikipedia. Someone with a PhD is much more likely to submit and identify meaningful content to these sites and less likely to fall for sensationalized unsubstantiated posts."

While it is true those without higher education have done great things this is directly related to research and information sorting, something someone with a Ph.D. has had to do to achieve that level of education.

But what we are seeing is not even content from simply college educated people but any idiot with a computer. I have met far to many people who simply "believe" they know what they are talking about and lack the basic education to understand why they really do not.

w9596 said...

This article is absolutely true, I find that many social bookmarking and SEO techniques are just ruining the way the Internet is meant to work. It should be best content for a search getting the traffic, not this stupid game playing.