Like HowStuffWorks on Facebook!

How Digg Works

        Tech | Social Networks

Digg Technology
Overview of Digg.com technology infrastructure
Overview of Digg.com technology infrastructure

For a Web site that receives more than 230 million page views a month (May 2008), Digg's technology framework is pretty streamlined. As with any proprietary system, Digg's technical department doesn't just put it all out there for everyone to copycat. But there are bits and pieces to be gleaned. The entire setup is based on LAMP (Linux, Apache, MySQL, PHP/Perl/Python), a programming model that ties together server applications running, say, PHP, with something like a MySQL database. It allows multiple languages to converse over a system architecture with minimal translation hold-ups.

In the Diggnation podcast recorded on June 14, 2006, Kevin Rose estimated the total number of servers in the area of 75. A 2008 post in the Digg technical blog stated that the company has between 1.8 and 2.5 times the minimum number of servers it needs to keep Digg running. According to the post, no one at Digg really knows how many servers the company actually has [source: Digg].

In an e-mail interview with HowStuffWorks, former Digg CEO Jay Adelson identifies the servers as Penguin Computing and Sun servers and says the Web servers are running PHP and the Debian Linux. operating system, and the database servers are running MySQL database management software. Digg actually doesn't have to store that much since it deals almost exclusively in text, but Adelson reports that the current setup is infinitely scalable. In a Mad Penguin interview in December 2005, Adelson says that Digg is "doubling the infrastructure every month to keep up with the demand." The basic infrastructure looks something like this:

Within this system, users submit and Digg stories and utilize all of the other features available on Digg.com. Digg doesn't use any cookies, only server-side storage, so all of your user data (past Diggs, friends, comments, etc.) are saved in your Digg profile, which is saved in the Digg database. You need to actually log in when you get to the site, which in practice may act as an initial security measure to ensure user validity for each visit. There are a number of legitimacy checks, which Digg calls "karma checks," built into the system at different points:

  • When a user attempts to submit an article, the system checks that the submission is legitimate -- that it's not spam (auto-submitted by a computer) and that the user is valid (not banned or otherwise limited). According to Adelson, the karma system also takes into account a variety of factors that includes certain "unique properties to digg's critical mass of users."
  • At the point when a submitted article is up for promotion to a category homepage or the front page -- which is determined by a number of factors, including the number and velocity of Diggs -- the system checks to make sure the Diggs are valid. One check involves looking for fraudulent accounts created only for the purpose of promoting a story. According to Adelman, "Our karma system knows the difference between users created to just digg a story and a user who has interacted with the site." The system also looks for auto-Digging, the computerized Digging of a story to fraudulently promote it to the homepage.

­

­Adelman confirms that there is zero editorial control happening at Digg behind the scenes, whether in submission, promotion or burying (the removal of a story from the Digg queue). There is no censorship of submissions beyond letting a user turn on a profanity filter that blocks curse words. And Digg manages "buries" the same way it manages everything else -- with a proprietary algorithm. The system runs a "de-promotion algorithm" that determines when a reported story is ready to disappear from the main site pages.

All of this sounds very democratic and forward-thinking, with Digg moving us further down the path of the populist Web that turns regular Joes into entrepreneurs, reporters, editors, stock traders and encyclopedia contributors. But a bit of a hubbub in mid-2006 called Digg's utterly user-driven nature into question -- at least in the minds of a select and verbal few. In the next section, we'll look into the user response to Digg.


More to Explore