How could an algorithm spot trolls on the Internet?

Is an algorithm viable, or enough?
The roadblock with using an algorithm to identify trolls is that it could target innocent users who exhibit some of the behaviors that are also common flags for troublemakers. For now, human moderation is still needed.
The roadblock with using an algorithm to identify trolls is that it could target innocent users who exhibit some of the behaviors that are also common flags for troublemakers. For now, human moderation is still needed.
© galaxy67/iStockphoto

Using an algorithm like this to automate troll banning is unlikely in the immediate future. It's in the academic research phase and not available as a usable software package at the moment. And at 80 percent accuracy, one in five of the users it identified would be innocent. The researchers found that being overly harsh or quick to ban or censor users tended to increase antisocial behavior, and even worsened the writing of posters who initially had higher text quality [source: Cheng]. There are also other types of trolling, including more subtle varieties, such as posting vexingly naive questions, that this algorithm wouldn't be able to detect.

Therefore it's still important for a human moderator to review the situation before anyone is banned. But a tool like this could help moderators spend their time more efficiently by flagging potentially disruptive users ahead of time. The findings can be used to develop subsequent studies and future troll-detection and moderation tools.

In the meantime, we can keep trying other measures. Some have suggested instituting user pseudonyms that stay the same over multiple sites [source: Schwartz]. Sites like Slashdot and Reddit have peer moderation systems in place. Sites xkcd and 4chan have even experimented with a bot called ROBOT9000 that mutes users when they say something that's been said before, only allowing original content [source: Munroe].

Google has made changes to YouTube to try to get a handle on its well-known trolling problem, including allowing users more leeway and tools to moderate their videos' comment sections, moving more relevant comments and comments from users' Google+ friends to the top and adding public or private commenting options [sources: Dredge, YouTube].

We, as online users, can also do things to diminish trolling, like making use of the voting and reporting features on online communities. The Stanford and Cornell researchers also found that other users engaging trolls with harsh feedback exacerbated the behavior. This reinforces a commonly repeated Internet maxim: "Don't feed the trolls!"

Author's Note: How could an algorithm spot trolls on the Internet?

Trolling is a major problem, at least for people who like to keep their spirits up and their blood pressure down when they're surfing the web. The very first time I watched a friend interact with other people in a virtual world online, a vile, unprovoked threat was hurled at her within minutes. This is probably what led me to eschew online conversations and stick to lurking for years. As with anything involving human interaction, there's no easy fix, and there will always be jerks making their own fun by trying to spoil ours. But I was happy to learn that research is being done into potential solutions. Any decrease in the mass of negativity online will be a plus. Until then, though, I may give up on article comment sections. I enjoy my sanity too much.

Related Articles


  • Academic Earth. "The Psychology of the Internet Troll." (June 2, 2015)
  • Amazon. "Amazon Mechanical Turk." (May 31, 2015)
  • Applebaum, Anne. "The Trolls Among Us." Slate. November 28, 2014. (June 2, 2015)
  • BBC. "What is an algorithm?" BBC Bitesize.
  • Breeze, Mez. "The Problems With Anonymous Trolls and Accountability in the Digital Age." Next Web. October 27, 2012. (June 2, 2015)
  • Brossard, Dominique and Dietram A. Scheufele. "This Story Stinks." New York Times. March 2, 2013. (June 2, 2015)
  • Buckels, Erin E. et al. "Trolls Just Want to Have Fun." Personality and Individual Differences. September 2014, Volume 67, Pages 97-102. (June 1, 2015)
  • Cheng, Justin et al. "Antisocial Behavior in Online Discussion Communities." Cornell University Library. April 2, 2015. (May 25, 2015)
  • Collins, Katie. "'Troll Hunting' Algorithm Could Make Web a Better Place." Wired. April 14, 2015. (May 25, 2015)
  • Dredge, Stuart. "YouTube Aims to Tame the Trolls With Changes to Its Comments Section." Guardian. Nov. 7, 2013. (June 2, 2015)
  • Felder, Adam. "How Comments Shape Perceptions of Sites' Quality — and Affect Traffic." Atlantic. June 5, 2014. (June 2, 2015)
  • Fobar, Rachel. "Researchers Develop a Troll-hunting Algorithm." Popular Science. April 13, 2015. (June 2, 2015)
  • Golbeck Ph.D., Jennifer. "Internet Trolls Are Narcissists, Psychopaths, and Sadists." Psychology Today. Sept. 18, 2014. (June 1, 2015)
  • Hern, Alex. "Algorithm 'Identifies Future Trolls From Just Five Posts.'" Guardian. April 17, 2015. (June 2, 2015)
  • Ingram, Mathew. "Research Shows That if You Remove Anonymity, You Won't Hear From Most of Your Readers." GigaOm. August 27, 2014. (June 2, 2015)
  • Khan Academy. "Intro to Algorithms — What is an algorithm and why should I care?" (June 1, 2015)
  • LaBarre, Suzanne. "Why We're Shutting Off Our Comments." Popular Science. Sept. 24, 2013. (June 2, 2015)
  • Mooney, Chris. "Internet Trolls Really Are Horrible People." Slate. Feb. 14, 2014. (June 2, 2015)
  • Mooney, Chris. "The Science of Why Comment Trolls Suck." Mother Jones. Jan. 10, 2013. (June 2, 2015)
  • Munroe, Randall P. "ROBOT9000 and #xkcd-signal: Attacking Noise in Chat." xkcd. Jan. 14, 2008. (June 2, 2015)
  • Pullen, John Patrick. "Science Says You Should Ignore Internet Trolls." April 20, 2015. (May 31, 2015)
  • Schwartz, Mattathias. "The Trolls Among Us." New York Times. Aug. 3, 2008. (June 1, 2015)
  • Vaas, Lisa. "New Algorithm Could Auto-squash Trolls." Naked Security. April 15, 2015. (May 25, 2015)
  • YouTube Official Blog. "We Hear You: Better Commenting Coming to YouTube." Sept. 24, 2013. (June 2, 2015)

More to Explore