How Email Works

Every day, the citizens of the Internet send each other billions of email messages. If you're online a lot, you yourself may send a dozen or more emails each day without even thinking about it. Obviously, email has become an extremely popular communication tool.

Have you ever wondered how email gets from your computer to a friend halfway around the world? What is a POP3 server, and how does it hold your mail? The answers may surprise you, because it turns out that email is an incredibly simple system at its core. In this article, we'll take an in-depth look at email and how it works.

An Email Message

According to Darwin Magazine: Prime Movers, the first email message was sent in 1971 by an engineer named Ray Tomlinson. Before this, you could only send messages to users on a single machine. Tomlinson's breakthrough was the ability to send messages to other machines on the Internet, using the @ sign to designate the receiving machine.

An email message has always been nothing more than a simple text message — a piece of text sent to a recipient. In the beginning and even today, email messages tend to be short pieces of text, although the ability to add attachments now makes many messages quite long. Even with attachments, however, email messages continue to be text messages — we'll see why when we get to the section on attachments.

Email Clients

You've probably already received several email messages today. To look at them, you use some sort of email client. Many people use well-known, stand-alone clients like Microsoft Outlook, Outlook Express, Eudora or Pegasus. People who subscribe to free email services like Hotmail or Yahoo use an email client that appears in a webpage. If you're an AOL customer, you use AOL's email reader. No matter which type of client you're using, it generally does four things:

Shows you a list of all of the messages in your mailbox by displaying the message headers. The header shows you who sent the mail, the subject of the mail and may also show the time and date of the message and the message size.
Lets you select a message header and read the body of the email message.
Let's you create new messages and send them. You type in the email address of the recipient and the subject for the message, and then type the body of the message.
Lets you add attachments to messages you send and save the attachments from messages you receive.

Sophisticated email clients may have all sorts of bells and whistles, but at the core, this is all that an email client does.

A Simple Email Server

Given that you have an email client on your machine, you are ready to send and receive email. All that you need is an email server for the client to connect to. Let's imagine what the simplest possible email server would look like in order to get a basic understanding of the process. Then we will look at the real thing.

If you've read How Web Servers Work, then you know that machines on the Internet can run software applications that act as servers. There are web servers, FTP servers, telnet servers and email servers running on millions of machines on the Internet right now. These applications run all the time on the server machine and they listen to specific ports, waiting for people or programs to attach to the port. The simplest possible email server would work something like this:

It would have a list of email accounts, with one account for each person who can receive email on the server. My account name might be mbrain, John Smith's might be jsmith, and so on.
It would have a text file for each account in the list. So, the server would have a text file in its directory named MBRAIN.TXT, another named JSMITH.TXT, and so on.
If someone wanted to send me a message, the person would compose a text message ("Marshall, Can we have lunch Monday? John") in an email client, and indicate that the message should go to mbrain. When the person presses the Send button, the email client would connect to the email server and pass to the server the name of the recipient (mbrain), the name of the sender (jsmith) and the body of the message.
The server would format those pieces of information and append them to the bottom of the MBRAIN.TXT file. The entry in the file might look like this:
The server would format those pieces of information and append them to the bottom of the MBRAIN.TXT file. The entry in the file might look like this: From: jsmith To: mbrain Marshall, Can we have lunch Monday? John

There are several other pieces of information that the server might save into the file, like the time and date of receipt and a subject line; but overall, you can see that this is an extremely simple process.

We'll look at the SMTP server in the next section.

More Complex Servers

As other people sent mail to mbrain, the server would simply append those messages to the bottom of the file in the order that they arrived. The text file would accumulate a series of five or 10 messages, and eventually I would log in to read them. When I wanted to look at my email, my email client would connect to the server machine. In the simplest possible system, it would:

Ask the server to send a copy of the MBRAIN.TXT file
Ask the server to erase and reset the MBRAIN.TXT file
Save the MBRAIN.TXT file on my local machine
Parse the file into the separate messages (using the word "From:" as the separator)
Show me all of the message headers in a list

When I double-clicked on a message header, it would find that message in the text file and show me its body.

As you can see, this is a very simple system. Surprisingly, the real email system that you use every day isn't much more complicated than this.

The Real Email System

For the vast majority of people right now, the real email system consists of two different servers running on a server machine. One is called the SMTP server, where SMTP stands for Simple Mail Transfer Protocol. The SMTP server handles outgoing mail. The other is either a POP3 server or an IMAP server, both of which handle incoming mail. POP stands for Post Office Protocol, and IMAP stands for Internet Mail Access Protocol. A typical email server looks like this:

The SMTP server listens on well-known port number 25, POP3 listens on port 110 and IMAP uses port 143 (see How Web Servers Work for details on ports).

The SMTP Server

Whenever you send a piece of email, your email client interacts with the SMTP server to handle the sending. The SMTP server on your host may have conversations with other SMTP servers to deliver the email.

Let's assume that I want to send a piece of email. My email ID is brain, and I have my account on howstuffworks.com. I want to send email to jsmith@mindspring.com. I am using a stand-alone email client like Outlook Express.

When I set up my account at howstuffworks, I told Outlook Express the name of the mail server — mail.howstuffworks.com. When I compose a message and press the Send button, here's what happens:

Outlook Express connects to the SMTP server at mail.howstuffworks.com using port 25.
Outlook Express has a conversation with the SMTP server, telling the SMTP server the address of the sender and the address of the recipient, as well as the body of the message.
The SMTP server takes the "to" address (jsmith@mindspring.com) and breaks it into two parts: the recipient name (jsmith) and the domain name (mindspring.com). If the "to" address had been another user at howstuffworks.com, the SMTP server would simply hand the message to the POP3 server for howstuffworks.com (using a little program called the delivery agent). Since the recipient is at another domain, SMTP needs to communicate with that domain.
The SMTP server has a conversation with a Domain Name Server, or DNS (see How Web Servers Work for details). It says, "Can you give me the IP address of the SMTP server for mindspring.com?" The DNS replies with the one or more IP addresses for the SMTP server(s) that Mindspring operates.
The SMTP server at howstuffworks.com connects with the SMTP server at Mindspring using port 25. It has the same simple text conversation that my email client had with the SMTP server for HowStuffWorks, and gives the message to the Mindspring server. The Mindspring server recognizes that the domain name for jsmith is at Mindspring, so it hands the message to Mindspring's POP3 server, which puts the message in jsmith's mailbox.

If, for some reason, the SMTP server at HowStuffWorks cannot connect with the SMTP server at Mindspring, then the message goes into a queue. The SMTP server on most machines uses a program called sendmail to do the actual sending, so this queue is called the sendmail queue. Sendmail will periodically try to resend the messages in its queue. For example, it might retry every 15 minutes. After four hours, it will usually send you a piece of mail that tells you there is some sort of problem. After five days, most sendmail configurations give up and return the mail to you undelivered.

The SMTP server understands very simple text commands like HELO, MAIL, RCPT and DATA. The most common commands are:

HELO - introduce yourself
EHLO - introduce yourself and request extended mode
MAIL FROM: - specify the sender
RCPT TO: - specify the recipient
DATA - specify the body of the message (To, From and Subject should be the first three lines.)
RSET - reset
QUIT - quit the session
HELP - get help on commands
VRFY - verify an address
EXPN - expand an address
VERB - verbose

The POP3 and IMAP Servers

In the simplest implementations of POP3, the server really does maintain a collection of text files — one for each email account. When a message arrives, the POP3 server simply appends it to the bottom of the recipient's file.

When you check your email, your email client connects to the POP3 server using port 110. The POP3 server requires an account name and a password. Once you've logged in, the POP3 server opens your text file and allows you to access it. Like the SMTP server, the POP3 server understands a very simple set of text commands. Here are the most common commands:

USER - enter your user ID
PASS - enter your password
QUIT - quit the POP3 server
LIST - list the messages and their size
RETR - retrieve a message, pass it a message number
DELE - delete a message, pass it a message number
TOP - show the top x lines of a message, pass it a message number and the number of lines

Your email client connects to the POP3 server and issues a series of commands to bring copies of your email messages to your local machine. Generally, it will then delete the messages from the server (unless you've told the email client not to).

You can see that the POP3 server simply acts as an interface between the email client and the text file containing your messages. And again, you can see that the POP3 server is extremely simple. You can connect to it through telnet at port 110 and issue the commands yourself if you would like to (see How Web Servers Work for details on telnetting to servers).

The IMAP Server

As you can see, the POP3 protocol is very simple. It allows you to have a collection of messages stored in a text file on the server. Your email client (e.g. Outlook Express) can connect to your POP3 email server and download the messages from the POP3 text file onto your PC. That is about all that you can do with POP3.

Many users want to do far more than that with their email, and they want their email to remain on the server. The main reason for keeping your email on the server is to allow users to connect from a variety of machines. With POP3, once you download your email it's stuck on the machine to which you downloaded it. If you want to read your email both on your desktop machine and your laptop (depending on whether you're working in the office or on the road), POP3 makes life difficult.

IMAP (Internet Mail Access Protocol) is a more advanced protocol that solves these problems. With IMAP, your mail stays on the email server. You can organize your mail into folders, and all the folders live on the server as well. When you search your email, the search occurs on the server machine, rather than on your machine. This approach makes it extremely easy for you to access your email from any machine, and regardless of which machine you use, you have access to all of your mail in all of your folders.

We'll look at IMAP problems and attachments in the next section.

IMAP Problems and Attachments

Your email client connects to the IMAP server using port 143. The email client then issues a set of text commands that allow it to do things like list all the folders on the server, list all the message headers in a folder, get a specific email message from the server, delete messages on the server or search through all of the emails on the server.

One problem that can arise with IMAP involves this simple question: "If all of my email is stored on the server, then how can I read my mail if I'm not connected to the Internet?" To solve this problem, most email clients have some way to cache email on their local machine. For example, the client will download all the messages and store their complete contents on the local machine (just like it would if it were talking to a POP3 server). The messages still exist on the IMAP server, but you now have copies on your machine. This allows you to read and reply to email even if you have no connection to the Internet. The next time you establish a connection, you download all the new messages you received while disconnected and send all the mail that you wrote while disconnected.

Attachments

Your email client allows you to add attachments to email messages you send, and also lets you save attachments from messages that you receive. Attachments might include word processing documents, spreadsheets, sound files, snapshots and pieces of software. Usually, an attachment is not text (if it were, you would simply include it in the body of the message). Since email messages can contain only text information, and attachments aren't text, there's a problem that needs to be solved.

In the early days of email, you solved this problem by hand, using a program called uuencode. The uuencode program assumes that the file contains binary information. It extracts 3 bytes from the binary file and converts them to four text characters (that is, it takes 6 bits at a time, adds 32 to the value of the 6 bits and creates a text character — see How Bits and Bytes Work to learn more about ASCII characters). What uuencode produces, therefore, is an encoded version of the original binary file that contains only text characters. In the early days of email, you would run uuencode yourself and paste the uuencoded file into your email message.

Considering its tremendous impact on society, having forever changed the way we communicate, today's email system is one of the simplest things ever devised! There are parts of the system, like the routing rules in sendmail, that get complicated, but the basic system is incredibly straightforward. The next time you send an email, you'll know exactly how it's getting to its destination.

Free and Paid Email Services

Whether it's for work or keeping in touch with family and friends, people rely on the Internet to send and receive email messages. According to a March 2007 study by the Pew Internet and American Life Project, 91 percent of U.S. Internet users have gone online to send or read email. The same source reports that 56 percent of email users send or read email as part of a typical day [source: Email Marketing Reports].

An October 2007 study by the market research firm, The Radicati Group, reported that 183 billion emails were sent daily in 2006. With numbers like that, it's no surprise that there's a large variety of email service providers to choose from. Choices include providers like Gmail or Yahoo! who provide free email service or providers who charge a fee like America Online, Apple or NetZero.

Gmail and Yahoo! make money from advertisers who pay them to expose email account holders to their messages. It's also easy to find reviews of various free email services. Here are few descriptions of free email services:

Gmail — Gmail is Google's free email service. It offers online storage that's practically unlimited. It also provides sorting methods that allows users to quickly organize and sift their inbox efficiently for important messages. As a free service, Gmail exposes users to contextual advertisements, which are based on keywords found in the user's email messages. It's versatile in handling many different attachment types and scans for spam, worms and virus.
Yahoo! mail — Perhaps one of the most popular free services, Yahoo! offers unlimited online storage as well as text messaging and RSS news feeds. Using more than a dozen filters, Yahoo! files incoming emails and automatically sends junk emails to a spam folder and allows you to add emails to that folder. It features easy-to-use drag and drop organization tools.
MSN Windows Hotmail — Another well-used free service, Hotmail is supported by Microsoft technology and features 5 GB of online storage space. A versatile desktop design allows users to customize the color and layout of their mail manager. It also offers users the choice of working a classic, familiar format and switching to an updated look with added features when they choose. Microsoft security features coupled with familiar drag-and-drop organization tools add further comfort-of-use for many email users.

Email providers who charge a fee may offer additional services such as increased storage space. Sample providers include Juno, EarthLink and Webmail.us.

For example, it's easier to get personalized email addresses — perhaps incorporating your actual name, for instance - using a paid service. Paid email services often allow users to keep their same email address even if they change their ISP and screen users from advertisers. They often are easier to reach on support issues and special features such as custom spam filters, additional email accounts and mobile access.

Most internet service providers can also serve as an email provider for no additional charge. Other paid email services specialize in small business accounts and can assist businesses in purchasing and maintaining a domain name.

Next we'll offer some etiquette tips for when you write and send email messages.

Email Etiquette

Most people wouldn't think of being rude or obnoxious when they speak to colleagues, clients or their supervisor. But those rules can fall by the wayside when we use email.

The opportunities for rudeness when using email are plentiful and not always easy to recognize. Most people wouldn't use foul language or derogatory terms in business communication. But what about raising one's voice? Is that possible when using email?

Here, we'll discuss some business writing rules and tips for practicing email etiquette.

People are busy, so:

Use the subject line. You might be tempted to bypass this part of the email, but just remember: everyone is in a hurry, including the recipient of your email. That person will appreciate the clue the subject line provides as to the message content. It helps them prioritize and organize.
Be brief and concise. No matter how clever and entertaining a writer you might be, your message recipients will appreciate brevity and clarity. Save the puns and witty turns of phrase for after birthday cards and toasts. State the message quickly followed by clear requests or instructions for any action needed by the recipient.
Keep it personal. Routinely copying others on emails clogs mailboxes and can lead to the main recipient wondering why you're doing so. Unless you have a specific reason for copying someone, keep the conversation between sender and recipient.
Reply quickly. Don't let emails sit around in your queue unanswered. Get back to the sender, even if it's to say you need more time to respond.

People are sensitive, so:

Don't over-punctuate. Adding multiple punctuation marks, such as ???? or !!!! after a sentence makes it seem as though you are shouting or frustrated with the recipient. Use normal punctuation rules.
Don't use all capitals. Using all caps MAKES IT SEEM LIKE YOU'RE SHOUTING. Shouting is rude. Use normal capitalization rules. If you need to emphasize something, write "I'd like to emphasize..."
Read it, out loud, before you send it. While you may think you're writing exactly what you mean, it pays to read some emails out loud to yourself, putting yourself in the recipient's shoes, before hitting that "send" button. Once it's in writing, it's hard to take back. Also, never forget that the recipient has both a printer and "forward" button. Never write something you wouldn't want to have circulated throughout the company or even beyond.
Don't write when you're angry. Perhaps one of the most common etiquette rules to break is firing off an email to someone when you're angry, either at them or a situation. See "Read it out loud" above. You can't take it back once you "flame" someone, and it can come back to haunt you.

Not everyone is as hip as you are, so:

Keep the symbols to a minimum. Using "smiles" is a trendy way to communicate mood and meaning. But do you know the difference between a sarcastic smile and a mischievous one? And even if you're sure you do, can you be sure your recipient does? It's easy to see the potential to unintentionally offend someone using these symbols. Avoid using them.
Minimize abbreviated phrases. Using abbreviations such as IMHO (in my humble opinion). FWIW (for what it's worth) and ROTFL (rolling on the floor laughing) can frustrate and confuse the recipient.

Email is like as a business letter. Ignoring basic rules can show disrespect for the recipient. Don't let informality spoil the recipient's opinion of you.

For lots more information about email and related topics, check out the links that follow.