How Facebook Photo Tags Work

By: Nathan Chandler

Facebook Face Recognition

Changes in facial expression are just one variable that can totally confuse automatic recognition technologies – and your mother, too.
Changes in facial expression are just one variable that can totally confuse automatic recognition technologies – and your mother, too.

Facial recognition technology is anything but new. Law enforcement agencies have been using these kinds of systems for years to build searchable image databases of known and suspected criminals and terrorists.

Facebook hasn't made public any details regarding the developer behind its recognition tool. But it's safe to say that this capability is similar to those in other consumer programs, such as Windows Live Photo Gallery, iPhoto, and Picasa, all of which have handy facial recognition features.


If you've used a point-and-shoot digital camera made in the past few years, you may have seen facial recognition technology at work. Many current cameras highlight faces using boxes, which indicate that the auto-focus is keeping those faces sharp and crisp. That's facial recognition at work.

Similarly, when you upload a picture, the Tag Suggestion feature finds faces in your pictures. Then it highlights those pictures and immediately prompts you to type a name. Suggested tags work only within your list of closer friends and not friends of friends.

To work their magic, facial recognition software relies on complicated algorithms driven by hardcore mathematics and clever programming. The application analyzes each image for specific characteristics, such as pixel values or gradients (the way images transition from dark to light areas), and other numerically precise data.

Then it makes (often very accurate) guesses as to which parts of the image corresponds to a human face. That part of the technology is pretty dependable. Matching a name consistently to the correct face is trickier.

That's because variables such as lighting, facial hair, and facial expressions can fool the software. This is evident in Facebook when Tag Suggestions confuses your friends and suggests the wrong names. But the technology is continually improving, and soon it will work no matter how bushy and wild you let that mustache grow.

For privacy-minded people, the implications of this technology are scary. It's possible that someday soon, a total stranger might take a photo of you from afar, enter that image into a search engine, and within minutes, know who you are, where you live, and a whole lot of other information you'd rather not disclose.

So use Facebook's Tag Suggestions feature to ease your tagging workload, but be sure to keep your guard up. Stay up to date on the seemingly endless changes Facebook administrators make to both tagging and privacy settings, and always keep questionable images off the Internet entirely. Doing so will keep you ahead in the game of Facebook fun and keep you safer, too.

Related Articles


  • Brown, Mark. "Facebook Silently Rolls Out Face Recognition Tagging to the World." June 8, 2011. (July 22, 2011)
  • Geuess, Megan. "Facebook Facial Recognition: Its Quiet Rise and Dangerous Future." April 26, 2011. (July 22, 2011)
  • Geuess, Megan and Sarah Jacobsson Purewal. "Facebook's Facial Recognition Flops." June 15, 2011. (July 22, 2011)
  • Kumar, Neeraj et al. "Describable Visual Attributes for Face Verification and Image Search." (July 22, 2011)
  • Livingston, Connor. "How Facebook Facial Recognition Works." July 17, 2011. (July 22, 2011)
  • McLaughlin, Laurianne. "Facebook Facial Recognition Flub: Not Enterprise Grade." June 9, 2011. (July 22, 2011)
  • Mitchell, Justin. "Making Photo Tagging Easier." The Facebook Blog. June 30, 2011. (July 22, 2011)
  • O'Reilly, Tim. "Facebook's Face Recognition Strategy May be Just the Ticket." June 9, 2011. (July 22, 2011)
  • Olivarez-Giles, Nathan. "Facebook Facial-Recognition Feature Draws Criticism." Los Angeles Times. June 9, 2011. (July 22, 2011)
  • Paul, Ian. "Facebook Photo Tagging: A Privacy Guide." June 9, 2011. (July 22, 2011)
  • Purewal, Sarah Jacobsson. "Google Won't Dabble in Facial Recognition System." May 19, 2011.. (July 22, 2011)
  • Purewal, Sarah Jacobsson. "Why Facebook's Facial Recognition is Creepy." June 8, 2011. (July 22, 2011)
  • Whittaker, Zack. "Facebook's Photo Facial Recognition Enabled: Time to Check Your Settings." June 7, 2011. (July 22, 2011)