How Search Engine Optimization Works

A search for "skydiving" on Yahoo yields many results. Where does your Web page rank, and how can you help it rise to the top?
A search for "skydiving" on Yahoo yields many results. Where does your Web page rank, and how can you help it rise to the top?
Screenshot by HowStuffWorks.com

It's tough getting noticed on the Web. A Web page can provide useful information about a popular subject in an interactive and engrossing way, yet still attract few visitors. One of the most reliable ways to improve traffic is to achieve a high ranking on search engine return pages (SERPs).

Imagine that you've created the definitive Web site on a subject -- we'll use skydiving as an example. Your site is so new that it's not even listed on any SERPs yet, so your first step is to submit your site to search engines like Google and Yahoo. The Web pages on your skydiving site include useful information, exciting photographs and helpful links guiding visitors to other resources. Even with the best information about skydiving on the Web, your site may not crack the top page of results on major search engines. When people search for the term "skydiving," they could end up going to inferior Web sites because yours isn't in the top results.

While most search engine companies try to keep their processes a secret, their criteria for high spots on SERPs isn't a complete mystery. Search engines are successful only if they provide a user links to the best Web sites related to the user's search terms. If your site is the best skydiving resource on the Web, it benefits search engines to list the site high up on their SERPs. You just have to find a way to show search engines that your site belongs at the top of the heap. That's where search engine optimization (SEO) comes in -- it's a collection of techniques a webmaster can use to improve his or her site's SERP position.

In this article, we'll look at two SEO philosophies: the white hat approach and the black hat approach. We'll also learn about some of the problems webmasters can encounter when trying to satisfy both the visitors to the site and search engines.

We'll take a general overview of what SEO really means on the next page.

SEO Overview

screenshot by HowStuffWorks 2008

Experts in search engine optimization can tell you the steps you need to take in order to be one of the top entries on a SERP. In our example, you've already provided the most important component of SEO: excellent content. Without strong content, SEO tips and tricks will provide a temporary boost in your site's ranking at best.

In an ideal World Wide Web, your site would rise to the top of every search engine's skydiving SERP based on content alone. While it's possible for your site to take the No. 1 SERP spot on its own, it could take months or even longer. Even worse, there's no guarantee your skydiving site will ever make it as high as the first page of search results.

For some webmasters, site traffic isn't that big a deal -- their sites might be a personal project. But for anyone who uses the Web as a way to make money, it's crucial. Whether the webmaster makes money by selling products on the site or through hosting Web advertisements, more visitors translates into more money. That's why some large companies are willing to spend money on SEO consultants -- they can be a worthy investment if the company's site is ranked higher than competitor sites.

SEO techniques rely on how search engines work. Some are legitimate methods that are a great way to let search engines know your Web page exists. Other techniques aren't good ways to get noticed and might involve exploiting a search engine so that it gives the page a higher ranking. Sometimes it's tough to tell if an approach is legitimate. If it seems a little questionable, it's probably a bad idea.

Let's take a look at the ways you can get a better spot on SERPs without getting into hot water with search engines. In the SEO business, these approaches are called white hat techniques. Read all about them on the next page.

White Hat SEO Techniques

The Webcrawler search engine spider analyzes Web pages and indexes them according to relevance.
The Webcrawler search engine spider analyzes Web pages and indexes them according to relevance.
screenshot by HowStuffWorks 2008

To improve a Web page's position in a SERP, you have to know how search engines work. Search engines categorize Web pages based on keywords -- important terms that are relevant to the content of the page. In our example, the term "skydiving" should be a keyword, but a term like "bungee jumping" wouldn't be relevant.

Most search engines use computer programs called spiders or crawlers to search the Web and analyze individual pages. These programs read Web pages and index them according to the terms that show up often and in important sections of the page. There's no way for a search engine spider to know your page is about skydiving unless you use the right keywords in the right places.

Here are some general tips about keyword placement:

  • One place you should definitely include keywords is in the title of your Web page. You might want to choose something like "Skydiving 101" or "The Art of Skydiving."
  • Another good place to use keywords is in headers. If your page has several sections, consider using header tags and include important keywords in them. In our example, headers might include "Skydiving Equipment" or "Skydiving Classes."
  • Most SEO experts recommend that you use important keywords throughout the Web page, particularly at the top, but it's possible to overuse keywords. Your skydiving site would obviously use the word "skydiving" as a keyword, but it might also include other keywords like "base jumping" or "parachute." If you use a keyword too many times, some search engine spiders will flag your page as spam. That's because of a black hat technique called keyword stuffing, but more on that later.

Keywords aren't the only important factor search engines take into account when generating SERPs. Just because a site uses keywords well doesn't mean it's one of the best resources on the Web. To determine the quality of a Web page, most automated search engines use link analysis. Link analysis means the search engine looks to see how many other Web pages link to the page in question.

Going back to our skydiving example, if a search engine sees that hundreds of other Web pages related to skydiving are linking to your Web page, the engine will give your page a higher rank. Search engines like Google weigh the importance of links based on the rank of the linking pages. In other words, if the pages linking to your site are themselves ranked high in Google's system, they boost your page's rank more than lesser-ranked pages.

So, how do you get sites to link to your page? That's a tricky task, but make sure your page is a destination people want to link to, and you're halfway there. Another way is to offer link exchanges with other sites that cover material related to your content. You don't want to trade links with just anyone because many search engines look to see how relevant the links to and from your page are to the information within your page. Too many irrelevant links and the search engine will think you're trying to cheat the system.

In the next section, we'll look more closely at ways people try to fool search engines into ranking their pages higher on a SERP.

Black Hat SEO Techniques

Jay Florian Mitchell/Hulton Archive/ Getty Images

Some people seem to believe that on the Web, the ends justify the means. There are lots of ways webmasters can try to trick search engines into listing their Web pages high in SERPs, though such a victory doesn't usually last very long.

One of these methods is called keyword stuffing, which skews search engine results by overusing keywords on the page. Usually webmasters will put repeated keywords toward the bottom of the page where most visitors won't see them. They can also use invisible text, text with a color matching the page's background. Since search engine spiders read content through the page's HTML code, they detect text even if people can't see it. Some search engine spiders can identify and ignore text that matches the page's background color.

Webmasters might include irrelevant keywords to trick search engines. The webmasters look to see which search terms are the most popular and then use those words on their Web pages. While search engines might index the page under more keywords, people who follow the SERP links often leave the site once they realize it has little or nothing to do with their search terms.

A webmaster might create Web pages that redirect visitors to another page. The webmaster creates a simple page that includes certain keywords to get listed on a SERP. The page also includes a program that redirects visitors to a different page that often has nothing to do with the original search term. With several pages that each focus on a current hot topic, the webmaster can get a lot of traffic to a particular Web site.

­Page stuffing also cheats people out of a fair search engine experience. Webmasters first create a Web page that appears high up on a SERP. Then, the webmaster duplicates the page in the hopes that both pages will make the top results. The webmaster does this repeatedly with the intent to push other results off the top of the SERP and eliminate the competition. Most search engine spiders are able to compare pages against each other and determine if two different pages have the same content.

Selling and farming links are popular black hat SEO techniques. Because many search engines look at links to determine a Web page's relevancy, some webmasters buy links from other sites to boost a page's rank. A link farm is a collection of Web pages that all interlink with one another in order to increase each page's rank. Small link farms seem pretty harmless, but some link farms include hundreds of Web sites, each with a Web page dedicated just to listing links to every other site in the farm. When search engines detect a link selling scheme or link farm, they flag every site involved. Sometimes the search engine will simply demote every page's rank. In other cases, it might ban all the sites from its indexes.

Cheating the system might result in a temporary increase in visitors, but since people normally don't like to be fooled, the benefits are questionable at best. Who wants to return to a site that isn't what it claims to be? Plus, most search engines penalize Web pages that use black hat techniques, which means the webmaster trades a short success for a long-term failure.

In the next section, we'll look at some factors that make SEO more difficult.

SEO Obstacles

Google's Webmaster Help Center gives tips on credible SEO techniques to boost your site's page rank.
Google's Webmaster Help Center gives tips on credible SEO techniques to boost your site's page rank.
screenshot by HowStuffWorks 2008

The biggest challenge in SEO approaches is finding a content balance that satisfies both the visitors to the Web page and search engine spiders. A site that's entertaining to users might not merit a blip on a search engine's radar. A site that's optimized for search engines may come across as dry and uninteresting to users. It's usually a good idea to first create an engaging experience for visitors, then tweak the page's design so that search engines can find it easily.

One potential problem with the way search engine spiders crawl through sites deals with media files. Most people browsing Web pages don't want to look at page after page of text. They want pages that include photos, video or other forms of media to enhance the browsing experience. Unfortunately, most search engines skip over image and video content when indexing a site. For sites that use a lot of media files to convey information, this is a big problem. Some interactive Web pages don't have a lot of text, which gives search engine spiders very little to go on when building an index.

Webmasters with sites that rely on media files might be tempted to use some of the black hat techniques to help even the playing field, but it's usually a bad idea to do that. For one thing, the major search engines are constantly upgrading spider programs to detect and ignore (or worse, penalize) sites that use black hat approaches. The best approach for these webmasters is to use keywords in important places like the title of the page and to get links from other pages that focus on relevant content.

Optimizing a site isn't always straightforward or easy, which is why some webmasters use an SEO consultant. When relying on an SEO consultant, it's important to check the consultant's credentials, track record and client list. It's also a good idea to stay as informed as possible about SEO issues -- if the consultant recommends a black hat approach and the webmaster takes the advice, search engines might hold both parties accountable.

Many SEO firms are completely legitimate businesses that only follow the white hat optimization philosophy. They help webmasters tweak Web page layout, choose the right words to increase traffic, and help facilitate link exchanges between sites with complementary content. If you have a Web page that needs a little help, it's a good idea to find someone who really knows how to leverage legitimate techniques to increase your page's SERP ranking.

To learn more about search engine optimization and related topics, follow the links on the following page.

Related HowStuffWorks Articles

More Great Links

Sources

  • Austin, David. "How Google Finds Your Needle in the Web's Haystack." Grand Valley State University. http://www.ams.org/featurecolumn/archive/pagerank.html
  • Callan, David. "Google Ranking Tips." AKAMarketing.com. http://www.akamarketing.com/google-ranking-tips.html
  • Cutts, Matt. "Confirming a penalty." Matt Cutts: Gadgets, Google, and SEO. February 11, 2006.
  • http://www.mattcutts.com/blog/confirming-a-penalty/
  • Greenberg, Andy. "The Saboteurs of Search." Forbes. June 28, 2007. http://www.forbes.com/technology/2007/06/28/negative-search-google-tech-ebiz-cx_ag_0628seo.html
  • Hunt, Bill. "What, Exactly, is Search Engine Spam?" Search Engine Watch. February 16, 2005. http://searchenginewatch.com/showPage.html?page=3483601
  • "Keyword stuffing." Google.com. http://www.google.com/support/webmasters/bin/answer.py?answer=66358
  • "Optimization, Over-Optimization or SEO Overkill?" webconfs.com. http://www.webconfs.com/over-optimization-article-8.php
  • "Ranking high at Google." A Promotion Guide. http://www.apromotionguide.com/google.html
  • "Search Engine Optimization (SEO)." Bruce Clay, Inc. http://www.bruceclay.com/web_rank.htm
  • Sullivan, Danny. "Intro to Search Engine Optimization." Search Engine Watch. March 12, 2007. http://searchenginewatch.com/showPage.html?page=2167921
  • Wallace, David. "Spamming Techniques That You Will Want To Avoid!" SearchRank. http://www.searchrank.com/articles/003.html
  • "Webmaster Guidelines." Google.com. http://www.google.com/support/webmasters/bin/answer.py?answer=35769
  • Wells, Terri. "Link Farming: No Good Harvest." SEOChat. May 16, 2007. http://www.seochat.com/c/a/Link-Trading-Help/Link-Farming-No-Good-Harvest/