NIST SP 800-177 Rev. 1 日本語訳

Revision 1

Scott Rose

J. Stephen Nightingale

Simson Garfinkel

Ramaswamy Chandramouli

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-177r1

Revision 1

Scott Rose

J. Stephen Nightingale*

Advanced Network Technology Division Information Technology Laboratory

Simson L. Garfinkel US Census Bureau

Ramaswamy Chandramouli Computer Security Division Information Technology Laboratory

*Former employee; all work for this publication was done while at NIST

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-177r1

February 2019

U.S. Department of Commerce Wilbur L. Ross, Jr., Secretary

National Institute of Standards and Technology

Walter Copan, NIST Director and Under Secretary of Commerce for Standards and Technology

This publication has been developed by NIST in accordance with its statutory responsibilities under the Federal Information Security Modernization Act (FISMA) of 2014, 44 U.S.C. § 3551 et seq., Public Law (P.L.) 113-283. NIST is responsible for developing information security standards and guidelines, including minimum requirements for federal information systems, but such standards and guidelines shall not apply to national security systems without the express approval of appropriate federal officials exercising policy authority over such systems. This guideline is consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130, Section 8b(3), Securing Agency Information Systems, as analyzed in Circular A-130, Appendix IV: Analysis of Key Sections. Supplemental information is provided in Circular A-130, Appendix III, Security of Federal Automated Information Resources.

Nothing in this publication should be taken to contradict the standards and guidelines made mandatory and binding on federal agencies by the Secretary of Commerce under statutory authority. Nor should these guidelines be interpreted as altering or superseding the existing authorities of the Secretary of Commerce, Director of the OMB, or any other federal official. This publication may be used by nongovernmental organizations on a voluntary basis and is not subject to copyright in the United States. Attribution would, however, be appreciated by NIST.

National Institute of Standards and Technology Special Publication 800-177 Revision 1 Natl. Inst. Stand. Technol. Spec. Publ. 800-177 Revision 1, 128 pages (February 2019) CODEN: NSPUE2

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-177r1

National Institute of Standards and Technology Attn: Advanced Network Technologies Division, Information Technology Laboratory 100 Bureau Drive (Mail Stop 8920) Gaithersburg, MD 20899-8920 Email: SP800-177@nist.gov

All comments are subject to release under the Freedom of Information Act (FOIA).

The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the Nation’s measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of concept implementations, and technical analyses to advance the development and productive use of information technology. ITL’s responsibilities include the development of management, administrative, technical, and physical standards and guidelines for the cost-effective security and privacy of other than national security-related information in federal information systems. The Special Publication 800-series reports on ITL’s research, guidelines, and outreach efforts in information system security, and its collaborative activities with industry, government, and academic organizations.

This document gives recommendations and guidelines for enhancing trust in email. The primary audience includes enterprise email administrators, information security specialists and network managers. This guideline applies to federal IT systems and will also be useful for small or medium sized organizations. Technologies recommended in support of core Simple Mail Transfer Protocol (SMTP) and the Domain Name System (DNS) include mechanisms for authenticating a sending domain: Sender Policy Framework (SPF), Domain Keys Identified Mail (DKIM) and Domain based Message Authentication, Reporting and Conformance (DMARC). Recommendations for email transmission security include Transport Layer Security (TLS) and associated certificate authentication protocols. Recommendations for email content security include the encryption and authentication of message content using S/MIME (Secure/Multipurpose Internet Mail Extensions) and associated certificate and key distribution protocols.

Email; Simple Mail Transfer Protocol (SMTP); Transport Layer Security (TLS); Sender Policy Framework (SPF); Domain Keys Identified Mail (DKIM); Domain based Message Authentication, Reporting and Conformance (DMARC); Authentication of Named Entities (DANE); S/MIME; OpenPGP.

This document gives recommendations and guidelines for enhancing trust in email. The primary audience for these recommendations is federal enterprise email administrators, information security specialists and network managers. While some of the guidelines in this document pertain to federal IT systems and network policy, most of the document will be more general in nature and could apply to any organization.

For most of this document, it will be assumed that the organization has some or all responsibility for email and can configure or manage its own email and Domain Name System (DNS) systems. Even if this is not the case, the guidelines and recommendations in this document may help in education about email security and can be used to produce a set of requirements for a contracted service.

Trademark Information

All registered trademarks belong to their respective organizations.

This document gives recommendations and guidelines for enhancing trust in email. The primary audience includes enterprise email administrators, information security specialists and network managers. This guideline applies to federal IT systems and will also be useful for small or medium sized organizations.

Email is a core application of computer networking and has been such since the early days of Internet development. In those early days, networking was a collegial, research-oriented enterprise. Security was not a consideration. The past forty years have seen diversity in applications deployed on the Internet, and worldwide adoption of email by research organizations, governments, militaries, businesses and individuals. At the same time there has been an associated increase in (Internet-based) criminal and nuisance threats.

The Internet’s underlying core email protocol, Simple Mail Transport Protocol (SMTP), was first adopted in 1982 and is still deployed and operated today. However, this protocol is susceptible to a wide range of attacks including man-in-the-middle content modification and content surveillance. The basic standards have been modified and augmented over the years with adaptations that mitigate some of these threats. With spoofing protection, integrity protection, encryption and authentication, properly implemented email systems can be regarded as sufficiently secure for government, financial and medical communications.

NIST has been active in the development of email security guidelines for many years. The most recent NIST guideline on secure email is NIST SP 800-45, Version 2 of February 2007, Guidelines on Electronic Mail Security. The purpose of that document is:

“To recommend security practices for designing, implementing and operating email systems on public and private networks,”

Those recommendations include practices for securing the environments around enterprise mail servers and mail clients, and efforts to eliminate server and workstation compromise. This guide complements SP 800-45 by providing more up-to-date recommendations and guidance for email digital signatures and encryption (via S/MIME), recommendations for protecting against unwanted email (spam), and recommendations concerning other aspects of email system deployment and configuration.

Following a description of the general email infrastructure and a threat analysis, these guidelines cluster into techniques for authenticating a sending domain, techniques for assuring email transmission security and those for assuring email content security. The bulk of the security enhancements to email rely on records and keys stored in the Domain Name System (DNS) by one party and extracted from there by the other party. Increased reliance on the DNS is permissible because of the recent security enhancements there, in particular the development and widespread deployment of the DNS Security Extensions (DNSSEC) to provide source authentication and integrity protection of DNS data.

The purpose of authenticating the sending domain is to guard against senders (both random and malicious actors) from spoofing another’s domain and initiating messages with bogus content, and against malicious actors from modifying message contents in transit. Sender Policy Framework (SPF) is the standardized way for a sending domain to identify and assert the authorized mail senders for a given domain. Domain Keys Identified Mail (DKIM) is the mechanism for asserting sending servers and eliminating the vulnerability of man-in-the-middle content modification by using digital signatures generated from the sending mail server.

Domain based Message Authentication, Reporting and Conformance (DMARC) was conceived to allow email senders to specify policy on how their mail should be handled, the types of security reports that receivers can send back, and the frequency those reports should be sent. Standardized handling of SPF and DKIM removes guesswork about whether a given message is authentic, benefitting receivers by allowing more certainty in quarantining and rejecting unauthorized mail. In particular, receivers compare the “From” address in the message to the SPF and DKIM results, if present, and the DMARC policy in the DNS. The results are used to determine how the mail should be handled. The receiver sends reports to the domain owner about mail claiming to originate from their domain. These reports should illuminate the extent to which unauthorized users are using the domain, and the proportion of mail received that is “good.”

Man-in-the-middle attacks can intercept cleartext email messages as they are transmitted hop-by-hop between mail relays. Any bad actor that can passively monitor network traffic can read such mail as it travels from submission to delivery systems. Email message confidentiality can be assured by encrypting traffic along the path. The Transport Layer Security Protocol (TLS) uses an encrypted channel to protect message transfers from man-in-the-middle attacks. TLS relies on the Public Key Infrastructure (PKI) system of X.509 certificates to carry exchange material and provide information about the entity holding the certificate. These are usually generated by a Certificate Authority (CA). The global CA ecosystem has in recent years become the subject to attack and has been successfully compromised more than once. One way to protect against CA compromises is to use the DNS to allow domains to specify their intended certificates or vendor CAs. Such uses of DNS require that the DNS itself be secured with DNSSEC. Correctly configured deployment of TLS may not stop a passive eavesdropper from viewing encrypted traffic but does practically eliminate the chance of deciphering it.

Server to server transport layer encryption also assures the integrity of email in transit, but senders and receivers who desire end-to-end assurance, (i.e., mailbox to mailbox) may wish to implement end-to-end, message-based authentication and confidentiality protections. The sender may wish to digitally sign and/or encrypt the message content, and the receiver can authenticate and/or decrypt the received message. Secure Multipurpose Internet Mail Extensions (S/MIME) is the recommended protocol for email end-to-end authentication and confidentiality. This usage of S/MIME is not common at the present time but is recommended. Certificate distribution remains a significant challenge when using S/MIME, especially the distribution of certificates between organizations. Research is underway on protocols that will allow the DNS to be used as a lightweight publication infrastructure for S/MIME certificates.

S/MIME is useful for authenticating email, since the protocol usually includes PKI certificates in messages in order to authenticate them, avoiding the necessity of distributing the sender’s public key certificate in advance. Encrypted mass mailings are more problematic, as S/MIME senders need to possess the certificate of each recipient if the sender wishes to send encrypted mail.

Email communications cannot be made trustworthy with a single package or application. It involves incremental additions to basic subsystems, with each technology adapted to a particular task. Some of the techniques use other protocols such as DNS to facilitate specific security functions like domain authentication, content encryption and message originator authentication. These can be implemented discretely or in aggregate, according to organizational needs.

3.1.3 Unauthorized Email Messages from a Valid DNS Domain (Address Spoofing)

— 17

3.1.4 Tampering/Modification of Email Content — 17

3.1.5 DNS Cache Poisoning — 18

3.1.6 Phishing and Spear Phishing — 18

3.2 Confidentiality-related Threats — 20

3.3 Availability-related Threats — 20

3.3.1 Email Bombing — 21

3.3.2 Unsolicited Bulk Email (Spam) — 21

3.3.3 Availability of Email Servers — 22

3.4 Summary of Threats and Mitigations — 22

3.5 Security Recommendations Summary — 24

4 Authenticating a Sending Domain and Individual Mail Messages — 25

4.1 Introduction — 25

4.2 Visibility to End Users — 27

4.3 Requirements for Using Domain-based Authentication Techniques for Federal Systems — 28

4.4 Sender Policy Framework (SPF) — 28

4.4.1 Background — 28

4.4.2 SPF on the Sender Side — 30

4.4.3 SPF and DNS — 33

4.4.4 Considerations for SPF when Using Cloud Services or Contracted Services — 33

4.4.5 SPF on the Receiver Side — 34

4.5 DomainKeys Identified Mail (DKIM) — 35

4.5.1 Background — 35

4.5.2 DKIM on the Sender Side — 36

4.5.3 Generation and Distribution of the DKIM Key Pair — 36

4.5.4 Example of a DKIM Signature — 37

4.5.5 Generation and Provisioning of the DKIM Resource Record — 38

4.5.6 Example of a DKIM RR — 39

4.5.7 DKIM and DNS — 39

4.5.8 DKIM Operational Considerations — 39

4.5.9 DKIM on the Receiver Side — 41

4.5.10 Issues with Mailing Lists — 41

4.5.11 Considerations for Enterprises When Using Cloud or Contracted Email

Services — 42

4.6 Domain-based Message Authentication, Reporting and Conformance

(DMARC) — 42

4.6.1 DMARC on the Sender Side — 43

4.6.2 The DMARC DNS Record — 44

4.6.3 Example of DMARC RR’s — 46

4.6.4 DMARC on the Receiver Side — 47

4.6.5 Policy and Reporting — 48

4.6.6 Considerations for Agencies When Using Cloud or Contracted Email Services — 50

4.6.7 Mail Forwarding and Indirect Email Flows — 50

4.7 Authenticating Mail Messages with Digital Signatures — 53

4.7.1 End-to-End Authentication Using S/MIME Digital Signatures — 54

4.8 Recommendation Summary — 55

5 Protecting Email Confidentiality — 57

5.1 Introduction — 57

5.2 Email Transmission Security — 57

5.2.1 TLS Configuration and Use — 58

5.2.2 X.509 Certificates — 59

5.2.3 STARTTLS — 63

5.2.4 SMTP Security via Opportunistic DNS-based Authentication of Named Entities (DANE) Transport Layer Security (TLS) — 64

5.2.5 SMTP MTA Strict Transport Security (MTA-STS) — 66

5.2.6 Comparing DANE and MTA-STS — 69

5.2.7 Reporting TLS Errors to Senders — 69

5.3 Email Content Security — 70

5.3.1 S/MIME and SMIMEA — 71

5.3.2 OpenPGP and OPENPGPKEY — 73

5.4 Security Recommendation Summary — 74

6 Reducing Unsolicited Bulk Email — 75

6.1 Introduction — 75

6.2 Why an Organization May Want to Reduce Unsolicited Bulk Email — 75

6.3 Techniques to Reduce Unsolicited Bulk Email — 75

6.3.1 Approved/Non-approved Sender Lists — 76

6.3.2 Domain-based Authentication Techniques — 77

6.3.3 Content Filtering — 78

6.4 User Education — 78

7 End User Email Security — 80

7.1 Introduction — 80

7.2 Webmail Clients — 80

7.3 Standalone Clients — 80

7.3.1 Sending via SMTP — 80

7.3.2 Receiving via IMAP — 81

7.3.3 Receiving via POP3 — 81

7.4 Mailbox Security — 82

7.4.1 Confidentiality of Data in Transit — 82

7.4.2 Confidentiality of Data at Rest — 82

7.5 Security Recommendation Summary — 83

B.5 Domain-based Message Authentication, Reporting and Conformance

(DMARC) — 88

B.6 Cryptography and Public Key Infrastructure (PKI) — 88

B.7 Other — 90

Appendix C— Overlay of NIST SP 800-53 Controls to Email Messaging Systems 93

C.1 Introduction — 93

C.2 Applicability — 93

C.3 Trustworthy Email Overlay — 93

C.4 Control Baselines — 94

C.5 Additional/Expanded Controls — 108

This guide provides recommendations for deploying protocols and technologies that improve the trustworthiness of email. These recommendations reduce the risk of spoofed email being used as an attack vector and reduce the risk of email contents being disclosed to unauthorized parties. These recommendations cover both the email sender and receiver.

Several of the protocols discussed in this guide use technologies beyond the core email protocols and systems. These include the Domain Name System (DNS), Public Key Infrastructure (PKI) and other core Internet protocols. This guide discusses how these systems can be used to provide security services for email.

This guide views email as a service, and thus it does not discuss topics such as individual server hardening, configuration and network planning. These topics are covered in NIST Special Publication 800-45, Version 2 of February 2007, Guidelines on Electronic Mail Security [SP800-45]. This guide should be viewed as a companion document to SP 800-45 that provides more updated guidance and recommendations that covers multiple components. This guide attempts to provide a holistic view of email and will only discuss individual system recommendations as examples warrant.

Likewise, this guide does not give specific configuration details for email components. There are a variety of hardware and software components that perform one or multiple email related tasks and it would be impossible to list them all in one guide. This guide will discuss protocols and configuration in an implementation neutral manner and administrators will need to consult their system documentation on how to execute the guidance for their specific implementations.

The rest of the document is presented in the following manner:

Throughout this guide, the following format conventions are used to denote special use text:

keyword - The text relates to a protocol keyword or text used as an example.

Security Recommendation: - Denotes a recommendation that administrators should note and account for when deploying the given protocol or security feature.

URLs are also included in the text and references to guide readers to a given website or online tool designed to aid administrators. This is not meant to be an endorsement of the website or any product/service offered by the website publisher. All URLs were considered valid at the time of writing.

There are a number of software components used to produce, send and transfer email. These components can be classified as clients or servers, although some components act as both. Some components are used interactively, and some are completely automated. In addition to the core components, some organizations use special purpose components that provide a specific set of security features. There are also other components used by mail servers when performing operations. These include the Domain Name System (DNS) and other network infrastructure pieces.

Fig 2-1 shows the relationship between the email system components on a network, which are described below in greater detail.

Figure 2-1: Main Components Used for Email

Most end users interact with their email system via a Mail User Agent (MUA). A MUA is a software component (or web interface) that allows an end user to compose and send messages and to one or more recipients. A MUA transmits new messages to a server for further processing (either final delivery or transfer to another server). The MUA is also the component used by end users to access a mailbox where in-bound emails have been delivered. MUAs are available for a variety of systems including mobile hosts. The proper secure configuration for a MUA depends on the MUA in question and the system it is running on. Some basic recommendations can be found in Section 7.

MUAs may utilize several protocols to connect to and communicate with email servers, (see Section 2.3.2 below). There may also be other features as well such as a cryptographic interface for producing encrypted and/or digitally signed email.

Email is transmitted, in a “store and forward” fashion, across networks via Mail Transfer Agents (MTAs). MTAs communicate using the Simple Mail Transfer Protocol (SMTP) described below and act as both client and server, depending on the situation. For example, an MTA can act as a server when accepting an email message from an end user's MUA, then act as a client in connecting to and transferring the message to the recipient domain's MTA for final delivery.

MTAs can be described with more specialized language that denotes specific functions:

Mail servers may also perform various security functions to prevent malicious email from being delivered or include authentication credentials such as digital signatures (see Sender Policy Framework Section 4.3 and DomainKeys Identified Mail (DKIM) Section 4.5). These security functions may be provided by other components that act as lightweight MTAs or these functions may be added to MTAs via filters or patches.

An email message may pass through multiple MTAs before reaching the final recipient. Each MTA in the chain may have its own security policy (which may be uniform within an organization but may not be uniform) and there is currently no way for a sender to request a particular level of security for the email message. However, there is work in progress¹ for specifying how a client can request the use of TLS for message transmission.

In addition to MUAs and MTAs, an organization may use one or more special purpose components for a particular task. These components may provide a security function such as malware filtering or may provide some business process functionality such as email archiving or content filtering. These components may exchange messages with other parts of the email infrastructure using all or part of SMTP (see Section 2.3.1) or use another protocol altogether.

Given the variety of components, there is no one single set of configurations for an administrator to deploy, and different organizations have deployed very different email architectures. An administrator should consult the documentation for their given component and their existing site-specific architecture.

Organizations that outsource their email service (whole or in part) may not have direct access to MTAs, Authoritative DNS servers, or any possible special use components. In cases of Email as a Service (EaaS), the service provider is responsible for the email infrastructure. Customers of Infrastructure as a Service (IaaS) may have sufficient access privileges to configure their email servers themselves. In either architecture, the enterprise may have complete configuration control over MUAs in use.

How an organization architects its email infrastructure is beyond the scope of this document. It is up to the organization and administrators to identify key requirements (availability, security, etc.) and available product or service offerings to meet those requirements. Federal IT administrators also need to take relevant federal IT policies into account when acquiring and deploying email systems.

Guidance for deploying and configuring an MTA for federal agency use exists as NIST SP 800-45 Guidelines on Electronic Mail Security [SP800-45]. In addition, the Dept. of Homeland Security (DHS) has produced the Email Gateway Reference Architecture [REFARCH] for agencies to use as a guide when setting up or modifying the email infrastructure for an agency.

In addition to MUAs and MTAs, there are other network components used to support the email service for an organization. Most obviously is the physical infrastructure: the cables, wireless access points, routers and switches that make up the network. In addition, there are network components used by email components in the process of completing their tasks. This includes the DNS, Public Key Infrastructure (PKI), and network security components that are used by the organization.

The DNS is a global, distributed database and associated lookup protocol. DNS is used to map a piece of information (most commonly a domain name) to an IP address or some other network information used by a computer system. The DNS is used by MUAs to find MSAs and by MTAs to find the IP address of the next-hop server for mail delivery. Sending MTAs query DNS for the Mail Exchange Resource Record (MX RR) of the recipient's domain (the part of an email address to the right of the “@” symbol) in order to find the receiving MTA to contact.

In addition to the “forward” DNS (translate domain names to IP addresses or other data), there is also the “reverse” DNS tree that is used to map IP addresses to their corresponding DNS name, or other data. Traditionally, the reverse tree is used to obtain the domain name for a given client based on the source IP address of the connection, but it is also used as a crude, highly imperfect authentication check. A host compares the forward and reverse DNS trees to check that the remote connection is likely valid and not a potential attacker abusing a valid IP address block. This can be more problematic in IPv6, where even small networks can be assigned very large address blocks. Email anti-abuse consortiums such as the Messaging, Malware Mobile Anti-Abuse Working Group (M3AAWG) recommend that enterprises should make sure that DNS reverse trees identify the authoritative mail servers for a domain [M3AAWG].

The DNS is also used as the publication method for protocols designed to protect email and combat malicious, spoofed email. Technologies such as SPF, DKIM and other use the DNS to publish policy artifacts or public keys that can be used by receiving MTAs to validate that a given message originated from the purported sending domain's mail servers. These protocols are discussed in Section 4. In addition, there are new proposals to encode end-user certificates or public keys (for S/MIME or OpenPGP) in the DNS using a mailbox as the hostname. These protocols are discussed in Section 5.3.

A third use of the DNS with email is with reputation services. These services provide information about the authenticity of an email based on the purported sending domain or originating IP address. These services do not rely on the anti-spoofing techniques described above but through historical monitoring, domain registration history, and other information sources. These services are often used to combat unsolicited bulk email (i.e., spam) and malicious email that could contain malware or links to subverted websites.

The Domain Name System Security Extensions (DNSSEC) [RFC4033] provides cryptographic security for DNS queries. Without security, DNS can be subjected to a variety of spoofing and man-in-the-middle attacks. Recommendations for deploying DNS in a secure manner are beyond the scope of this document. Readers are directed to NIST SP 800-81 [SP800-81] for recommendations on deploying DNSSEC.

Organizations may utilize security components that do not directly handle email but may perform operations that affect email transactions. These include network components like firewalls, Intrusion Detection Systems (IDS) and similar malware scanners. These systems may not play any direct role in the sending and delivering of email but may have a significant impact if misconfigured. This could result in legitimate SMTP connections being denied and the failure of valid email to be delivered. Network administrators should take the presence of these systems into consideration when making changes to an organization's email infrastructure. This document makes no specific recommendations regarding these peripheral components.

Organizations that send and receive S/MIME or OpenPGP protected messages, as well as those that use TLS, will also need to rely on the certificate infrastructure (S/MIME) or key server ecosystem (OpenPGP) used with these protocols. These infrastructures do not always require the deployment of a dedicated system, but does require administrator time to obtain, configure and distribute security credentials to end-users.

X.509 certificates can be used to authenticate one (or both) ends of a TLS connection when SMTP runs over TLS (usually MUA to MTA). S/MIME also uses X.509 certificates [RFC5280] to certify and store public keys used to validate digital signatures and encrypt email. The Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile is commonly called PKIX and is specified by [RFC5280]. Certificate Authorities (CA) (or the organization itself) issues X.509 certificates for an individual end-user or enterprise/business role (performed by a person or not) that sends email (for S/MIME). Recommendations for S/MIME protected email are given in Section 5. Recommendations for SMTP over TLS are given in Section 5. Federal agency network administrators should also consult NIST SP 800-57 Part 3 [SP800-57P3] and the Federal PKI Key Recovery Policy [FPKIKRP] for further guidance on cryptographic parameters and deployment of any PKI components and credentials within an organization.

There are two types of protocols used in the transmission of email. The first are the protocols used to transfer messages between MTAs and their end users (using MUAs). The second is the protocol used to transfer messages between mail servers.

This guide is not meant to be an in-depth discussion of the protocols used in email. These protocols are discussed here simply for background information.

Email messages are transferred from one mail server to another (or from an MUA to MSA/MTA) using SMTP. SMTP was originally specified in 1982 in [RFC821] and has undergone several revisions, the most current being [RFC5321]. SMTP is a text-based client-server protocol where the client (email sender) contacts the server (next-hop MTA) and issues a set of commands to tell the server about the message to be sent, and then transmits the message itself. The majority of these commands are ASCII text messages sent by the client and a resulting return code (also ASCII text) returned by the server. The basic SMTP connection procedure is shown below in Figure 2-2:

Figure 2-2: Basic SMTP Connection Set-up

In the above, the client initiates the connection using TCP over port 252. After the initial connection, the client and server perform a series of SMTP transactions to send the message. These transactions take the form of first stating the return address of the message (known as the return path) using the MAIL command, then the recipient(s) using the RCPT command and ending with the DATA command which contains the header and body of the email message. After each command the server responds with either a positive or negative (i.e., error) code.

SMTP servers can advertise the availability of options during the initial connection. These extensions are currently defined in [RFC5321]. These options usually deal with the transfer of the actual message and will not be covered in this guide except for the STARTTLS option. This option advertised by the server is used to indicate to the client that Transport Layer Security (TLS) is available. SMTP over TLS allows the email message to be sent over an encrypted channel to protect against monitoring a message in transit. Recommendations for configuring SMTP over TLS are given in Section 5.2.

MUAs typically do not use SMTP when retrieving mail from an end-user's mailbox, only for submission. MUAs use another client-server protocol to retrieve the mail from a server for display on an end-user's host system. These protocols are commonly called Mail Access Protocols and are either Post Office Protocol (POP) or Internet Message Access Protocol (IMAP). Most modern MUAs support both protocols but an enterprise service may restrict the use of one in favor of a single protocol for ease of administration or other reasons. Recommendations for the secure configuration of these protocols are given in Section 7.

POP version 3 (POP3) [STD35] is the simpler of the two protocols and typically downloads all mail for a user from the server, then deletes the copy on the server, although there is an option to maintain it on the server. POP3 is similar to SMTP, in that the client connects to a port (normally port 110 or port 995 when using TLS) and sends ASCII commands, to which the server responds, only instead of sending messages, it retrieves messages from the MTA. When the session is complete, the client terminates the connection. POP3 transactions are normally done in the clear, but an extension is available to do POP3 over TLS using the STLS command, which is very similar to the STARTTLS option in SMTP. Clients may connect initially over port 110 and invoke the STLS command, or alternatively, most servers allow TLS by default connections on port 995.

IMAP [RFC3501] is an alternative to POP3 but includes more built-in features that make it more appealing for enterprise use. IMAP clients can download email messages, but the messages remain on the server. This and the fact that multiple clients can access the same mailbox simultaneously mean that end-users with multiple devices (laptop and smartphone for example), can keep their email synchronized across multiple devices. Like POP3, IMAP also has the ability to secure the connection between a client and a server. Traditionally, IMAP uses port 143 with no encryption. Encrypted IMAP runs over port 993, although modern IMAP servers also support the STARTTLS option on port 143.

In addition to POP3 and IMAP, there are other proprietary protocols in use with certain enterprise email implementations. Microsoft Exchange clients³ can use the Messaging Application Programming Interface (MAPI/RPC) to access a mailbox on a Microsoft Exchange server (and some other compatible implementations). Some cloud providers require clients to access their cloud-based mailbox using a web portal as the MUA instead of a dedicated email client. With the exception of Microsoft’s Outlook Web Access, most web portals use IMAP to access the user’s mailbox.

Two distinct email addresses are used when sending an email via SMTP: the SMTP MAIL FROM address and the email header FROM address. The SMTP envelope MAIL FROM (also sometimes referred to as the RFC5321.From, or the return-path address, or envelope From:) is from address used in the client SMTP mail from: command as shown in Fig. 2-2 above. This email address may be altered by a sending MTA and may not always match the email address of the original sender. In the rest of this document, the term envelope-From: will be used. The second is the sender email address (sometimes referred to as the RFC5322.From). This is the address end-users see in the message header. In the rest of this document, the term message-From: will be used to denote this email address. The full details of the syntax and semantics of email addresses are defined in [RFC3696], [RFC5321] and [RFC5322].

Both types of contemporary email addresses consist of a local-part separated from a domain-part (a fully-qualified domain name) by an at-sign ("@") (e.g., local-part@domain-part). Typically, the local-part identifies a user of the mail system or server identified by the domain-part. The semantics of the local-part are not standardized, which occasionally causes confusion among both users and developers.⁴ The domain-part is typically a fully qualified domain name of the system or service that hosts the user account that is identified by the local-part (e.g., user@example.com).

While the user@example.com is by far the most widely used form of email address, other forms of addresses are sometimes used. For example, the local-part may include “sub-addressing” that typically specifies a specific mailbox/folder within a user account (e.g., user+folder@example.com). Exactly how such local-parts are interpreted can vary across specific mail system implementations. The domain-part can refer to a specific MTA server, the domain of a specific enterprise or email service provider (ESP).

The remainder of this document will use the terms email-address, local-part and domain-part to refer the Internet email addresses and their component parts.

Email messages may be formatted as plain text or as compound documents containing one or more components and attachments. Modern email systems layer security mechanisms on top of these underlying systems.

Internet email was originally sent as plain text ASCII messages [RFC2822]. The Multi-purpose Internet Mail Extensions (MIME) [RFC2045] [RFC2046] [RFC2047] allows email to contain non-ASCII character sets as well as other non-text message components and attachments. Essentially MIME allows for an email message to be broken into parts, with each part identified by a content type. Typical content types include text/plain (for ASCII text), image/jpeg, text/html, etc. A mail message may contain multiple parts, which themselves may contain multiple parts, allowing MIME-formatted messages to be included as attachments in other MIME-formatted messages. The available types are listed in an IANA registry⁵ for developers, but not all may be understood by all MUAs.

The Secure Multi-purpose Internet Mail Extensions (S/MIME) is a set of widely implemented proposed Internet standards for cryptographically securing email [RFC5750] [RFC5751]. S/MIME provides authentication, integrity and non-repudiation (via digital signatures) and confidentiality (via encryption). S/MIME utilizes asymmetric keys for cryptography (i.e., public key cryptography) where the public portion is normally encoded and presented as X.509 digital certificates.

With S/MIME, signing digital signatures and message encryption are two distinct operations: messages can be digitally signed, encrypted, or both digitally signed and encrypted (Figure 2-5). Because the process is first to sign and then encrypt, S/MIME is vulnerable to re-encryption attacks⁶; a protection is to include the name of the intended recipient in the encrypted message.

Figure 2-3: S/MIME Messages can be signed, encrypted, or both signed and encrypted

OpenPGP [RFC3156] [RFC4880] is an alternative proposed Internet standard for digitally signing and encrypting email. OpenPGP is an adaptation of the message format implemented by the Pretty Good Privacy (PGP) email encryption system that was first released in 1991. Whereas the PGP formats were never formally specified, OpenPGP specifies open, royalty-free formats for encryption keys, signatures, and messages. Today the most widely used implementation of OpenPGP is Gnu Privacy Guard (gpg)⁷, an open source command-line program that runs on many platforms, with APIs in popular languages such as C, Python and Perl. Most desktop and web-based applications that allow users to send and receive OpenPGP-encrypted mail rely on gpg as the actual cryptographic engine.

OpenPGP provides similar functionality as S/MIME, with three significant differences:

OpenPGP keys may also be registered with the OpenPGP “public key servers” (described below). OpenPGP “public key servers” are internet connected systems that maintain a database of PGP public keys organized by email address. Anyone may post a public key to the OpenPGP key servers, and that public key may contain any email address. Some OpenPGP clients can search the key servers for all of the keys that belong to a given email address and download the keys that match. Because there are no access controls on the servers, attackers are free to submit a fraudulent public key, and it is the responsibility of the person or program that downloads the public key to validate it.

The Web-of-Trust is designed to minimize the problems of the key server. After an OpenPGP user downloads all of the keys associated with a particular email address, the correct OpenPGP certificate is selected by the signatures that it carries. Because Web-of-Trust supports arbitrary validation geometries, it allows both the top-down certification geometry of X.509 as well as peer-to-peer approaches. However, studies have demonstrated that users find this process confusing [WHITTEN1999], and the Web-of-Trust has not seen widespread adoption.

An alternative way to publish OpenPGP keys using the DNS is described in Section 5.3.2, OpenPGP, although the technique has not yet been widely adopted.

Like S/MIME, among the biggest hurdles of deploying OpenPGP are the need for users to create certificates in advance, the difficulty of obtaining the certificate of another user in order to send an encrypted message and incorporating this seamlessly into mail clients. However, in OpenPGP this difficulty impacts both digital signatures and encryption, since OpenPGP messages may not include the sender’s public key (or generated certificate).

These differences are summarized in Table 2-1.

Table 2-1: Comparison of S/MIME and OpenPGP operations

Whereas S/MIME and OpenPGP provide a security overlay for traditional Internet email, some organizations have adopted secure web-mail systems as an alternative approach for sending encrypted e-mail messages between users. Secure web-mail systems can protect email messages solely with host-based security, or they can implement a cryptographic layer using S/MIME, OpenPGP, or other algorithms, such as the Boneh-Franklin (BF) and Boneh-Boyen (BB1) Identity-Based Encryption (IBE) algorithms [RFC5091] [RFC5408] [RFC5409].

Secure webmail systems can perform message decryption at the web server or on the end-user’s client. In general, these systems are less secure than end-to-end systems because the private key is under the control of the web server, which also has access to the encrypted message. These systems cannot ensure non-repudiation since the server has direct access to the signing key.

An exception is webmail-based systems that employ client-side software to make use of a private key stored at the client—for example, a webmail plug-in that allows the web browser to make use of a private key stored in a FIPS-201 compliant smartcard. In these cases, the message is decrypted and displayed at the client, and the server does not access the decrypted text of the message.

The security threats to email service discussed in this section are related to canonical functions of the service such as: message submission (at the sender end), message transmission (transfer) and message delivery (at the recipient end).

Threats to the core email infrastructure functions can be classified as follows:

The security threats due to insufficiency of core security functions are not covered. These include threats to support infrastructure such as network components and firewalls, host OS and system threats, and potential attacks due to lax security policy at the end user or administrator level (e.g., poor password choices). Threats directed to these components and recommendations for enterprise security policies are found in other documents.

Integrity in the context of an email service assumes multiple dimensions. Each dimension can be the source of one or more integrity-related threats:

An unauthorized email sender is some MSA or MTA that sends email messages that appear to be from a user in a specific domain (e.g., user@example.com) but is not identified as a legitimate mail sender by the organization that runs the domain.

The main risk that an unauthorized email sender may pose to an enterprise is that a sender may be sending malicious email and using the enterprise’s IP address block and reputation to avoid anti-spam filters. A related risk is that the sender may be sending emails that present themselves as legitimate communications from the enterprise itself.

There are many scenarios that might result in an unauthorized email sender:

One way to mitigate the risk of unauthorized senders is for the enterprise to block outbound port 25 (used by SMTP) for all hosts except those authorized to send mail. In addition, domains can deploy the sender authentication mechanism described in Section 4.3 (Sender Policy Framework (SPF)), using which senders can assert the IP addresses of the authorized MTAs for their domain using a DNS Resource Record.

Security Recommendation 3-1: To mitigate the risk of unauthorized sender, an enterprise administrator should block outbound port 25 (except for authorized mail senders) and look to deploy firewall or intrusion detection systems (IDS) that can alert the administrator when an unauthorized host is sending mail via SMTP to the Internet.

The proliferation of virtualization greatly increases the risk that an unauthorized virtual server running on a virtual machine (VM) within a particular enterprise might send email. This is because VMs may be configured (by default or otherwise) to run email servers (MTAs), and many VM hypervisors use network address translation (NAT) to share a single IP address between multiple VMs. Thus, a VM that is unauthorized to send email may share an IP address with a legitimate email sender. To prevent such a situation, ensure that VMs that are authorized mail senders and those VMs that are not authorized, do not share the same set of outbound IP addresses. An easy way to do this is assigning these VMs to different NAT instances. Alternatively, internal firewall rules can be used to block outbound port 25 for VMs that are not authorized to send outbound email.

Security Recommendation 3-2: Systems that are not involved in the organization’s email infrastructure should be configured to not run Mail Transfer Agents (MTAs). Internal systems that need to send mail should be configured to use a trusted internal MSA.

Unauthorized mail receivers are a risk to the enterprise IT security posture because they may be an entry point for malicious email. If the enterprise email administrator does not know of the unauthorized email receiver, they cannot guarantee the server is secure and provide the appropriate mail handling rules for the enterprise such as scanning for malicious links/code, filtering spam, etc. This could allow malware to bypass the enterprise perimeter defenses and enter the local network undetected.

Security Recommendation 3-3: To mitigate the risk of unauthorized receivers, an enterprise administrator should block inbound port 25 and look to deploy firewall or intrusion detection systems (IDS) that can alert the administrator when an unauthorized host is accepting mail via SMTP from the Internet.

Just as organizations face the risk of unauthorized email senders, they also face the risk that they might receive email from an unauthorized sender. This is sometimes called “spoofing,” especially when one group or individual sends mail that appears to come from another. In a spoofing attack, the adversary spoofs messages using another (sometimes even non-existent) user’s email address.

For example, an attacker sends emails that purport to come from user@example.com, when in fact the email messages are being sent from a compromised home router. Spoofing the message-From: address is trivial, as the SMTP protocol [RFC2821] allows clients to set any message-From: address. Alternatively, the adversary can simply configure a MUA with the name and email address of the spoofed user and send emails to an open SMTP relay (see [RFC2505] for a discussion of open relays).

The same malicious configuration activity can be used to configure and use wrong, misleading or malicious display names. When a display name that creates a degree of trust such as “Administrator” shows up on the email received at the recipient’s end, it might make the recipient reveal some sensitive information which the recipient would not normally do. Thus, the spoofing threat/attack also has a social engineering dimension as well.

Section 4 discusses a variety of countermeasures for this type of threat. The first line of defense is to deploy domain-based authentication mechanisms (see Section 4). These mechanisms can be used to alert on or block email that was sent using a spoofed domain. Another end-to-end authentication technique is to use digital signatures to provide integrity protection for email headers. The digital signature used should cover the header portion of the email message that contains the address of the sender.

The content of an email message, just like any other message content traveling over the Internet, is liable to be altered in transit. Hence the content of the received email may not be the same as what the sender originally composed. The countermeasure for this threat is for the sender to digitally sign the message, attach the signature to the plaintext message and for the receiver to verify the signature.

There are several solutions available to mitigate this risk by either encrypting the transmission of email messages between servers using Transport Layer Security (TLS) for SMTP or using an end-to-end solution to digitally sign email between initial sender and final receiver. Recommendations for using TLS with SMTP are discussed in Section 5.2.1 and end-to-end email encryption protocols are discussed in Section 4.6. The use of digital signatures within the S/MIME and OpenPGP protocols is described in section 5.3.

Email systems rely on DNS for many functions. Some of them are:

There are risks to using the DNS as a publication mechanism for authenticating email. First, those highly motivated to conduct phishing/spam campaigns, may attempt to spoof a given domain’s DNS-based email authentication mechanisms in order to continue to deliver spoofed email masquerading as the domain in question. The second risk is that an attacker would spoof a domain’s DNS-based authentication mechanisms in order to disrupt legitimate email from the source domain. For example, maliciously spoofing the SPF record of authorized mail relays, to exclude the domains legitimate MTAs, could result in all legitimate email from the target domain being dropped by other MTAs. Lastly, a resolver whose cache has been poisoned can potentially return the IP address desired by an attacker, rather than the legitimate IP address of a queried domain name. In theory, this allows email messages to be redirected or intercepted.

Another impact of a DNS server with a poisoned cache as well as a compromised web server is that the users are redirected to a malicious server/address when attempting to visit a legitimate web site. Although the visit to an illegitimate web site can occur by clicking on a link in a received email, this use case has no direct relevance to integrity of an email service and hence is outside the scope of this document.

As far as DNS cache poisoning is concerned, DNSSEC security extension [RFC4033] [RFC4034] [RFC4035] can provide protection from these kinds of attacks since it ensures the integrity of DNS resolution through an authentication chain from the root to the target domain of the original DNS query. However, even the presence of a single non-DNSSEC aware server in the chain can lead to a risk of compromise of the integrity of the DNS resolution.

Phishing is the process of using a spoofed email to collect private information, distribute malware, or commit fraud. This is done with the intention of committing identity theft, gaining access to credit cards and bank accounts of the victim etc. Adversaries use a variety of tactics to make the recipient of the email believe that they have received the phishing email from a legitimate user or a legitimate domain, including:

As part of the email message, the recipient may usually be asked to click on a link to what appears like a legitimate website, but in fact is a URL that will take the recipient into a spoofed website set up by the adversary. If the recipient clicks on the embedded URL, the victim often finds that the sign-in page, logos and graphics are identical to the legitimate website in the adversary-controlled website, thereby creating the trust necessary to make the recipient submit the required information such as user ID and the password. Some attackers use web pages to deliver malware directly to the victim’s web browser.

In many instances, the phishing emails are generated in thousands without focus on profile of the victims. Hence, they will have a generic greeting such as “Dear Member”, “Dear Customer” etc. A variant of phishing is spear phishing where the adversary is aware of, and specific about, the victim’s profile. More than a generic phishing email, a spear phishing email makes use of more contextual information to make users believe that they are interacting with a legitimate source. For example, a spear phishing email may appear to relate to some specific item of personal importance or a relevant matter at the organization—for instance, discussing payroll discrepancies or a legal matter. As in phishing, the ultimate motive is the same—to lure the recipient to an adversary-controlled website masquerading as a legitimate website to collect sensitive information about the victim or attack the victim’s computer.

There are two minor variations of phishing: clone phishing and whaling. Clone phishing is the process of cloning an email from a legitimate user carrying an attachment or link and then replacing the link or attachment alone with a malicious version and then sending altered email from an email address spoofed to appear to come from the original sender (carrying the pretext of re-sending or sending an updated version). Whaling is a type of phishing specifically targeted against high profile targets so that the resulting damage carries more publicity and/or financial rewards for the perpetrator is greater.

The most common countermeasures used against phishing are domain-based checks such as SPF, DKIM and DMARC (see Section 4). More elaborate is to design anti-phishing filters that can detect text commonly used in phishing emails, recovering hidden text in images, intelligent word recognition – detecting cursive, hand-written, rotated or distorted texts as well as the ability to detect texts on colored backgrounds. While these techniques will not prevent malicious email sent using compromised legitimate accounts, they can be used to reduce malicious email sent from spoofed domains or spoofed “From:” addresses.

A confidentiality-related threat occurs when the data stream containing email messages with sensitive information are accessible to an adversary. The type of attack that underlies this threat can be passive since the adversary only requires read access but not write access to the email data being transmitted. The two variations of this type of attack include:

Encryption is the best defense against eavesdropping attacks. Encrypting the email messages either between MTAs (using TLS as described in Section 5) can thwart attacks involving packet interception. End-to-end encryption (described in Section 5.3) can protect against both eavesdropping attacks as well as MTA software compromise.

A second form of passive attack is a traffic analysis attack. In this scenario, the adversary is not able to directly interpret the contents of an email message, mostly due to the fact that the message is encrypted. However, since inference of information is still possible in certain circumstances (depending upon interaction or transaction context) from the observation of external traffic characteristics (volume and frequency of traffic between any two entities) and hence the occurrence of this type of attack constitutes a confidentiality threat.

Although the impact of traffic analysis is limited in scope, it is much easier to perform this attack in practice—especially if part of the email transmission media uses a wireless network, if packets are sent over a shared network, or if the adversary has the ability to run network management or monitoring tools against the victim’s network. TLS encryption provides some protection against traffic analysis attacks, as the attacker is prevented from seeing any message headers. End-to-end email encryption protocols do not protect message headers, as the headers are needed for delivery to the destination mailbox. Thus, organizations may wish to employ both kinds of encryption to secure email from confidentiality threats.

An availability threat exists in the email infrastructure (or for that matter any IT infrastructure), when potential events occur that prevent the resources of the infrastructure from functioning according to their intended purpose. The following availability-related threats exist in an email infrastructure:

Email bombing is a type of attack that involves sending several thousands of messages to a particular mailbox in order to cause overflow. These can be many large messages or a very large number of small messages. Such a mailbox will become unusable for the legitimate email account holder to access. No new messages can be delivered, and the sender receives an error asking to resend the message. In some instances, the mail server may also crash.

The motive for email bombing is denial of service (DoS). A DoS attack by definition either prevents authorized access to resources or causes delay (e.g., long response times) of time-critical operations. Hence email bombing is a major availability threat to an email system since it can potentially consume substantial Internet bandwidth as well as storage space in the message stores of recipients. An email bombing attack can be launched in several ways.

There are many ways to perpetrate an email bombing attack, including:

Possible countermeasures for protection against email bombing are: (a) Use filters that are based on the logic of filtering identical messages that are received within a chosen short span of time and (b) configuring email receivers to block messages beyond a certain size and/or containing attachments that exceed a certain size.

Spam is the internet slang for unsolicited bulk email (UBE). Spam refers to indiscriminately sent messages that are unsolicited, unwanted, irrelevant and/or inappropriate, such as commercial advertising in mass quantities. Thus spam, generally, is not targeted towards a particular email receiver or domain. However, when the volume of spam coming into a particular email domain exceeds a certain threshold, it has availability implications since it results in increased network traffic and storage space for message stores. Spam that looks for random gullible victims or targets particular users or groups of users with malicious intent (gathering sensitive information for physical harm or for committing financial fraud) is called phishing. From the above discussion of email bombing attacks, it should be clear that spam can sometimes be a type of email bombing.

Protecting the email infrastructure against spam is a challenging problem. This is due to the fact that the two types of techniques currently used to combat spam have limitations. See Section 6 for a more detailed discussion of unsolicited bulk email.

The email infrastructure, just like any other IT infrastructure, should provide for fault tolerance and avoid single points of failure. A domain with only a single email server or a domain with multiple email servers, but all located in a single IP subnet is likely to encounter availability problems either due to software glitches in MTA, hardware maintenance issues or local data center network problems. The typical measures for ensuring high availability of email as a service are: (a) Multiple MTAs with placement based on the email traffic load encountered by the enterprise; and, (b) Distribution of email servers in different network segments or even physical locations.

A summary of the email related threats to an enterprise is given in Table 3-1. This includes threats to both the email the receiver and the purported sender - often spoofed, and who may not be aware an email was sent using their domain. Mitigations are listed in the final column to reduce the risk of the attack being successful, or to prevent them.

Table 3-1 Email-based Threats and Mitigations

Security Recommendation 3-1: To mitigate the risk of unauthorized sender, an enterprise administrator should block outbound port 25 (except for authorized mail senders) and look to deploy firewall or intrusion detection systems (IDS) that can alert the administrator when an unauthorized host is sending mail via SMTP to the Internet.

Security Recommendation 3-2: Systems that are not involved in the organization’s email infrastructure should not be configured to run Mail Transfer Agents (MTAs). Internal systems that need to send mail should be configured to use a trusted internal MSA.

Security Recommendation 3-3: To mitigate the risk of unauthorized receivers, an enterprise administrator should block inbound port 25 and look to deploy firewall or intrusion detection systems (IDS) that can alert the administrator when an unauthorized host is accepting mail via SMTP from the Internet.

RFC 5322 defines the Internet Message Format (IMF) for delivery over the Simple Mail Transfer Protocol (SMTP) [RFC5321], but in its original state any sender can write any envelope-From: address in the header (see Section 2.3.3). This envelope-From: address can however be overridden by malicious senders or enterprise mail administrators, who may have organizational reasons to rewrite the header, and so both [RFC 5321] and [RFC 5322] defined From: addresses can be aligned to some arbitrary form not intrinsically associated with the originating IP address. In addition, any man in the middle attack can modify a header or data content. New protocols were developed to detect these envelope-From: and message-From: address spoofing or modifications.

Sender Policy Framework (SPF) [RFC4408] uses the Domain Name System (DNS) to allow domain owners to create records that associate the envelope-From: address domain name with one or more IP address blocks used by authorized MTAs. It is then a simple matter for a receiving MTA to check a SPF TXT record in the DNS. This check would confirm the purported sender of a message is in the listed approved sending MTA is indeed authorized to transmit email messages for the domain listed in the envelope-From: address. Mail messages that do not pass this check may be marked, quarantined or rejected. SPF is described in subsection 4.4 below.

The DomainKeys Identified Mail (DKIM) [RFC6376] protocol allows a sending MTA to digitally sign selected headers and the body of the message with a RSA signature and include the signature in a DKIM header that is attached to the message prior to transmission. The DKIM signature header field includes a selector, which the receiver can use to retrieve the public key from a record in the DNS to validate the DKIM signature over the message. So, validating the signature assures the receiver that the message has not been modified in transit – other than additional headers added by MTAs en route which are ignored during the validation. Use of DKIM also ties the email message to the domain publishing the public key, regardless of the From: address (which could be different). DKIM is detailed in subsection 4.5.

Deploying SPF and DKIM may curb illicit activity against a sending domain, but the sender gets no indication of the extent of the beneficial (or otherwise) effects of these policies. Sending domain owners may choose to construct pairwise agreements with selected recipients to manually gather feedback, but this is not a scalable solution. The Domain-based Message Authentication, Reporting and Conformance protocol (DMARC) [RFC7489] institutes such a feedback mechanism, to let sending domain owners know the proportionate effectiveness of their SPF and DKIM policies, and to signal to receivers what action should be taken in various individual and bulk attack scenarios. After setting a policy to advise receivers to deliver, quarantine or reject messages that fail both SPF and DKIM, email receivers then return DMARC aggregate and/or failure reports of email dispositions to the domain owner, who can review the results and potentially refine the policy. DMARC is described in subsection 4.6.

While DMARC can do a lot to curb spoofing and phishing (Section 3.1.6 above), it does need careful configuration. Intermediaries that forward mail have many legitimate reasons to rewrite headers, usually related to legitimate activities such as operating mailing lists, mail groups, and end-user mail forwarding. It should be noted that mail server forwarding changes the source IP address, and without rewriting the envelope-From: address, this can make SPF checks fail. On the other hand, header rewriting, or adding a footer to mail content, may cause the DKIM signature to fail. Both of these interventions can cause problems for DKIM validation and for message delivery. Section 4.6 expands on the problems of mail forwarding, and its mitigations.

SPF, DKIM and DMARC authenticate that the sending MTA is an authorized, legitimate sender of email messages for the domain-part of the envelope-From: (and message-From: for DMARC) address, but these technologies do not verify that the email message is from a specific individual or logical account. That kind of assurance is provided by end-to-end security mechanisms such as S/MIME (or OpenPGP). The DKIM and S/MIME/OpenPGP signature standards are not-interfering: DKIM signatures go in the email header, while S/MIME/OpenPGP signatures are carried as MIME body parts. The signatures are also complementary: a message is typically signed by S/MIME or OpenPGP immediately after it is composed, typically by the sender’s MUA, and the DKIM signature is added after the message passes through the sender’s MSA or MTA.

The interrelation of SPF, DKIM, DMARC, and S/MIME signatures are shown in the Figure 4-1 below:

Figure 4-1: The interrelationship of DNSSEC, SPF, DKIM, DMARC and S/MIME for assuring message authenticity and integrity.

As mentioned above, the domain-based authentication protocols discussed in this section were designed with MTAs in mind. There was thought to be no need for information passed to the end recipient of the email. The results of SPF and DKIM checks are not normally visible in MUA components unless the end user views the message headers directly (and knows how to interpret them). This information may be useful to some end users who wish to filter messages based on these authentication results. [RFC7601] specifics how an MTA/MDA can add a new header to a message upon receipt that provides status information about any authentication checks done by the receiving MTA. Some MUAs make use of this information to provide visual cues (an icon, text color, etc.) to end users that this message passed the MTAs checks and was deemed valid. This does not explicitly mean that the email contents are authentic or valid, just that the email passed the various domain-based checks performed by the receiving MTA.

Email administrators should be aware if the MUAs used in their enterprise can interpret and show results of the authentication headers to end users. Email administrators should educate end users about what the results mean when evaluating potential phishing/spam email as well as not assuming positive results means they have a completely secure channel.

As of the time of writing of this guidance document, the DHS has called out the use of domain-based authentication techniques for email as Binding Operational Directive (BOD) 18-01 [BOD18-01]. This includes the techniques discussed below. This section gives best-common-practice guidance of the domain-based authentication techniques listed (but not described) in BOD 18-01 and [FISMAMET]. This document does not extend those requirements in anyway but gives guidance on how to meet existing requirements.

Sender Policy Framework (SPF) is a standardized way for the domain of the envelope-From: address to identify and assert the mail originators (i.e., mail senders) for a given domain. The sending domain does this by placing a specially formatted Text Resource Record (TXT RR) in the DNS database for the domain. The idea is that a receiving MTA can check the IP address of the connecting MTA against the purported sending domain (the domain-part of the envelope-From: address) and see if the domain vouches for the sending MTA. The receiving MTA does this by sending a DNS query to the purported sending domain for the list of valid senders.

SPF was designed to address phishing and spam being sent by unauthorized senders (i.e., botnets). SPF does not stop all spam, in that spam email being sent from a domain that asserts its sending MTAs via an SPF record will pass all SPF checks. That is, a spammer can send email using an envelope-From: address using a domain that the spammer controls, and that email will not result in a failed SPF check. SPF checks fail when mail is received from a sending MTA other than those listed as approved senders for the envelope-From: domain. For example, an infected botnet of hosts in an enterprise may be sending spam on its own (i.e., not through the enterprises outgoing SMTP server), but those spam messages would be detected as the infected hosts would not be listed as valid senders for the enterprise domain and would fail SPF checks. See [HERZBERG2009] for a detailed review of SPF and its effectiveness.

SPF works by comparing the sender's IP address (IPv4 or IPv6, depending on the transport used to deliver the message) with the policy encoded in any SPF record found at the sending domain. That is, the domain-part of the envelope-From: address. This means that SPF checks can actually be applied before the bulk of the message is received from the sender. For example, in Fig 4-1, the sender with IP address 192.168.0.1 uses the envelope MAIL FROM: tag as alice@example.org even though the message header is alice.sender@example.net. The receiver queries for the SPF RR for example.org and checks if the IP address is listed as a valid sender. If it is listed, or no valid SPF record is found, the message is processed as usual. If not, the receiver may mark the message as a potential spoofed email, quarantine it for further (possibly administrator) analysis or reject the message, depending on the SPF policy and/or the policy discovered in any associated DMARC record (see subsection 4.5, below) for example.org.

Figure 4-2: SMTP envelope header vs. message header

Because of the nature of DNS (which SPF uses for publication) an SPF policy is tied to one domain. That is, @example.org and @sub.example.org are considered separate domains just like @example.net and all three need their own SPF records. This complicates things for organizations that have several domains and subdomains that may (or may not) send mail. There is a way to publish a centralized SPF policy for a collection of domains using the include: tag (see Sec 4.2.2.2 below)

SPF was first specified in [RFC4408] as an experimental protocol, since at the same time other, similar proposals were also being considered. Over time however, SPF became widely deployed and was finalized in [RFC7208] (and its updates). The changes between the final version and the original version are mostly minor, and those that base their deployments on the experimental version are still understood by clients that implement the final version. The most significant difference is that the final specification no longer calls for the use of a specialized RRType (simply called a SPF RR) and instead calls for the sender policy to be encoded in a TXT Resource Record, in part because it proved too difficult to universally upgrade legacy DNS systems to accept a new RRType. Older clients may still look for the SPF RR, but the majority will fall back and ask for a TXT RR if it fails to find the special SPF RR. Resolution of the Sender Policy Framework (SPF) and Sender ID Experiments [RFC6686] presents the evidence that was used to justify the abandonment of the SPF RR.

SPF is seen as a way to reduce the risk of phishing email being delivered and used as to install malware inside an agency's network. Since it is relatively easy to check using the DNS, SPF is seen as a useful layer of email checks.

Deploying SPF for a sending domain is fairly straightforward. It does not even require SPF aware code in mail servers, as receivers, not senders, perform the SPF processing. The only necessary actions are identifying IP addresses or ranges of permitted sending hosts for a given domain and adding that information in the DNS as a new resource record.

The first step in deploying SPF for a sending domain is to identify all the hosts that send email out of the domain (i.e., SMTP servers that are tasked with being email gateways to the Internet). This can be hard to do because:

If the senders cannot be listed with certainty, the SPF policy can indicate that receivers should not necessarily reject messages that fail SPF checks by using the “~” or “?” mechanisms, rather than the “-“ mechanism (see 4.3.2.2 below) in the SPF TXT record.

Note: Deployment of DMARC [RFC7489] (discussed below) allows for reporting SPF check results back to sending domain owners, which allows senders to modify and improve their policy to minimize improper rejections.

Once all the outgoing senders are identified, the appropriate policy can be encoded and put into the domain database. The SPF syntax is fairly rich and can express complex relationships between senders. Not only can entities be identified and called out, but the SPF statement can also request what emphasis should be placed on each test.

SPF statements are encoded in ASCII text (as they are stored in DNS TXT resource records) and checks are processed in left to right order. Every statement begins with v=spf1 to indicate that this is an SPF (version 1) statement⁸.

Other mechanisms are listed in Table 4-1:

Table 4-1: SPF Mechanisms

Each mechanism in the string is separated by whitespace. In addition, there are qualifiers that can be used for each mechanism (Table 4-2):

Table 4-2: SPF Mechanism Qualifiers

There are other mechanisms available as well that are not listed here. Administrators interested in seeing the full depth of the SPF syntax are encouraged to read the full specification in [RFC7208]. To aid administrators, there are some online tools that can be used assist in the generation and testing of an SPF record. These tools take administrator input and generate the text that the administrator then places in a TXT RR in the given domain's zone file.

Some examples of the mechanisms for SPF are given below. In each example, the purported sender in the SMTP envelope is example.com

The given domain has one mail server that both sends and receives mail. No other system is authorized to send mail. The resulting SPF RR would be:

example.com IN TXT "v=spf1 mx -all"

The given enterprise has a DMZ that allows hosts to send mail but is not sure if other senders exist. As a temporary measure, they list the SPF as:

example.com IN TXT "v=spf1 ip4:192.168.1.0/16 ~all"

The enterprise has several domains for projects, but only one set of sending MTAs. For each domain, there is an SPF RR with the include: declaration pointing to a central TXT RR with the SPF policy that covers all the domains. For example, each domain could have:

example.com IN TXT "v=spf1 include:spf.example.net."

The follow up query for the spf.example.net then has:

spf.example.net IN TXT "v=spf1 ip4:192.168.0.1 …"

This makes SPF easier to manage for an enterprise with several domains and/or public subdomains. Administrators only need to edit spf.example.net to make changes to the SPF RR while the other SPF RR's in the other domains simply use the include: tag to reference it. No email should originate from the domain:

example.com IN TXT "v=spf1 -all"

The above should be added to all domains that do not send mail to prevent them being used by phishers looking for sending domains to spoof that they believe may not be monitored as closely as those that accept and send enterprise email. This is an important principle for domains that think they are immune from email related threats. Domain names that are only used to host web or services are advised to publish a “-all” record, to protect their reputation.

Notice that semicolons are not permitted in the SPF TXT record.

Security Recommendation 4-1: Organizations are recommended to deploy SPF to specify which IP addresses are authorized to transmit email on behalf of the domain. Domains controlled by an organization that are not used to send email, for example Web only domains, should include an SPF RR with the policy indicating that there are no valid email senders for the given domain.

Since SPF policies are now only encoded in DNS TXT resource records, no specialized software is needed to host SPF RRs. Organizations can opt to include the old (no longer mandated) unique SPF RRType as well, but it is usually not needed, as clients that still query for the type automatically query for a TXT RR if the SPF RR is not found.

Organizations that deploy SPF should also deploy DNS security (DNSSEC) [RFC4033], [RFC4034], [RFC4035]. DNSSEC provides source authentication and integrity protection for DNS data. SPF RRs in DNSSEC signed zones cannot be altered or stripped from responses without DNSSEC aware receivers detecting the attack. Its use is more fully described in Section 5.

Changing the policy statement in an SPF RR is straightforward but requires timing considerations due to the caching nature of DNS. It may take some time for the new SPF RR to propagate to all authoritative servers. Likewise, the old, outgoing SPF RR may be cached in client DNS servers for the length of the SPF's TXT RR Time-to-Live (TTL). An enterprise should be aware that some clients might still have the old version of the SPF policy for some time before learning the new version. To minimize the effect of DNS caching, it is useful to decrease the DNS timeout to a small period of time (e.g., 300 seconds) before making changes, and then restoring DNS to a longer time period (e.g., 3600 seconds) after the changes have been made, tested, and confirmed to be correct.

When an organization outsources its email service (whole or part) to a third party such as a cloud provider or contracted email service, that organization needs to make sure any email sent by those third parties will pass SPF checks. To do this, the enterprise administrator should include the IP addresses of third party senders in the enterprise SPF policy statement RR. Failure to include all the possible senders could result in valid email being rejected due to a failure when doing the SPF check.

Including third-parties to an SPF RR is done by adding the IP addresses/hostnames individually or using the include: tag to reference a third party's own SPF record (if one exists). In general, it is preferable to use the include: mechanism, as the mechanism avoids hard-coding IP addresses in multiple locations. The include: tag does have a hard limit on the number of “chained” include: tag that a client will look up to prevent an endless series of queries. This value is ten unique DNS lookups by default.

For instance, if example.com has its own sending MTA at 192.0.0.1 but also uses a third party (third-example.net) to send non-transactional email as well, the SPF RR for example.com would look like:

example.com IN TXT "v=spf1 ip4:192.0.0.1 include:third-example.net -all"

As mentioned above, the include: mechanism does not simply concatenate the policy tests of the included domain (here: third-example.net) but performs all the checks in the SPF policy referenced and returns the final result. An administrator should not include the modifier "+" (requiring the mechanism to pass in order for the whole check to pass) to the include: unless they are also in control of the included domain, as any change to the SPF policy in the included domain will affect the SPF validation check for the sending domain.

Unlike senders, receivers need to have SPF-aware mail servers to check SPF policies. SPF has been around in some form (either experimental or finalized) and available in just about all major mail server implementations. There are also patches and libraries available for other implementations to make them SPF-aware and perform SPF queries and processing⁹. There may be plug-ins available for a particular Mail User Agent so end users can perform SPF checks even if their incoming mail server does not.

As mentioned above, SPF uses the envelope-From: address domain-part and the IP address of the sender. This means that SPF checks can be started before the actual text of the email message is received. Alternatively, messages can be quickly received and held in quarantine until all the checks are finished. In either event, checks must be completed before the mail message is sent to an end user's inbox (unless the only SPF checks are performed by the end user using their own MUA).

The resulting action based on the SPF checks depends on local receiver policy and the statements in the purported sending domain's SPF statement. The action should be based on the modifiers (listed above) on each mechanism. If no SPF TXT RR is returned in the query, or the SPF has formatting errors that prevent parsing, the default behavior is to accept the message. This is the same behavior for mail servers that are not SPF-aware.

Just as an organization that deploys SPF should also deploy DNSSEC [SP800-81], receivers that perform SPF processing should also perform DNSSEC validation (if possible) on responses to SPF queries. A mail server should be able to send queries to a validating DNS recursive server if it cannot perform its own DNSSEC validation.

Security Recommendation 4-2: Organizations should deploy DNSSEC for all DNS name servers and validate DNSSEC responses on all systems that receive email.

DomainKeys Identified Mail (DKIM) permits a person, role, or organization that owns the signing domain to claim some responsibility for a message by associating the domain with the message. This can be an author's organization, an operational relay, or one of their agents. DKIM separates the question of the identity of the signer of the message from the purported author of the message. Assertion of responsibility is validated through a cryptographic signature and by querying the signer's domain directly to retrieve the appropriate public key. Message transit from author to recipient is through relays that typically make no substantive change to the message content and thus preserve the DKIM signature. Because the DKIM signature covers the message body, it also protects the integrity of the email communication. Changes to a message body will result in a DKIM signature validation failure, which is why some mailing lists (that add footers to email messages or change Subject header fields) will cause DKIM signature validation failures (discussed below).

A DKIM signature is generated by the signing MTA using the email message body and headers and places it in the header of the message along with information for the client to use in validation of the signature (i.e., key selector, algorithm, etc.). When the receiving MTA gets the message, it attempts to validate the signature by looking for the public key indicated in the DKIM signature. The MTA issues a DNS query for a text resource record (TXT RR) that contains the encoded key.

Like SPF (see Section 4.4), DKIM allows an enterprise to vouch for an email message sent from a domain it does not control (as would be listed in the SMTP envelope). The sender only needs the private key to generate signatures. This allows an enterprise to have email sent on its behalf by an approved third party. The presence of the public key in the enterprises' DNS implies that there is a relationship between the enterprise and the sender.

Since DKIM requires the use cryptographic keys, enterprises must have a key management plan in place to generate, store and retire key pairs. Administrative boundaries complicate this plan if one organization sends mail on another organization's behalf.

DKIM was originally developed as part of a private sector consortium and only later transitioned to an IETF standard. The threat model that the DKIM protocol is designed to protect against was published as [RFC4686] and assumes bad actors with an extensive corpus of mail messages from the domains being impersonated, knowledge of the businesses being impersonated, access to business public keys, and the ability to submit messages to MTAs and MSAs at many locations across the Internet. The original DKIM protocol specification was developed as [RFC4871], which is now considered obsolete. The specification underwent several revisions and updates and the current version of the DKIM specification is published as [RFC6376].

Unlike SPF, DKIM requires specialized functionality on the sender MTA to generate the signatures. Therefore, the first step in deploying DKIM is to ensure that the organization has an MTA that can support the generation of DKIM signatures. DKIM support is currently available in some implementations or can be added using open source filters¹⁰. Administrators should remember that since DKIM involves digital signatures, signing MTAs should also have appropriate cryptographic tools to create and store keys and perform cryptographic operations.

The next step in deploying DKIM, after ensuring that the sending MTA is DKIM-aware, is to generate a signing key pair.

Cryptographic keys should be generated in accordance with NIST SP 800-57, “Recommendations for Key Management” [SP800-57pt1] and NIST SP 800-133, “Recommendations for Cryptographic Key Generation.” [SP800-133] Although there exist web-based systems for generating DKIM public/private key pairs and automatically producing the corresponding DNS entries, such systems should not be used for federal information systems because they may compromise the organization’s private key.

Currently the DKIM standard specifies that messages must be signed with one of two digital signature algorithms: RSA/SHA-256 and the Edwards-curve Digital Signature Algorithm Curve25519 (ed25519) [RFC8463]. Of these, only RSA/SHA-256 is approved for use by government agencies with DKIM, as the hash algorithm SHA-1 is no longer approved for use in conjunction with digital signatures (see Table 4-3).

Table 4-3: Recommended Cryptographic Key Parameters

Once the key pair is generated, the administrator should determine a selector name to use with the key. A DKIM selector name is a unique identifier for the key that is used to distinguish one DKIM key from any other potential keys used by the same sending domain, allowing different MTAs to be configured with different signing keys. This selector name is used by receiving MTAs to query the validating key.

The public part of the key pair is stored in a the DKIM TXT Resource Record (RR). This record should be added to the organization’s DNS server and tested to make sure that it is accessible both within and outside the organization.

The private part of the key pair is used by the MTA to sign outgoing mail. Administrators must configure their mail systems to protect the private part of the key pair from exposure to prevent an attacker from learning the key and using it to spoof email with the victim domain's DKIM key. For example, if the private part of the key pair is kept in a file, file permissions must be set so that only the user under which the MTA is running can read it.

As with any cryptographic keying material, enterprises should use a Cryptographic Key Management System (CKMS) to manage the generation, distribution, and lifecycle of DKIM keys. Federal agencies are encouraged to consult NIST SP 800-130 [SP800-130] and NIST SP 800-152 [SP800-152] for guidance on how to design and implement a CKMS within an agency.

Security Recommendation 4-3: Federal agency administrators shall only use keys with approved algorithms and lengths for use with DKIM.

Security Recommendation 4-4: Administrators should ensure that the private portion of the key pair is adequately protected on the sending MTA and that only the MTA software has read privileges for the key. Federal agency administrators should follow FISMA control SC-12 [SP800-53] guidance with regards to distributing and protecting DKIM key pairs.

Security Recommendation 4-5: Each sending MTA should be configured with its own private key and its own selector value, to minimize the damage that may occur if a private key is compromised. This private key must have protection against both accidental disclosure or attacker’s attempt to obtain or modify.

Below is an example of a DKIM signature as would be seen in an email header. A signature is made up of a collection of tag=value pairs that contain parameters needed to successfully validate the signature as well as the signature itself. An administrator usually cannot configure the tags individually as these are done by the MTA functionality that does DKIM, though some require configuration (such as the selector, discussed above). Some common tags are described in Table 4-4.

Table 4-4: DKIM Signature Tag and Value Descriptions

Thus, a DKIM signature from a service provider sending mail on behalf of example.gov might appear as an email header:

DKIM-Signature: v=1; a=rsa-sha256; d=example.gov; c=simple; i=@gov-sender.example.gov; t=1425066098; s=adkimkey; bh=base64 string; b=base64 string

Note that DKIM requires the use of semicolons between statements.

The public portion of the DKIM key is encoded into a DNS TXT Resource Record (RR) and published in the zone indicated in the FROM: field of the email header. The DNS name for the RR uses the selector the administrator chose for the key pair and a special tag to indicate it is for DKIM ("_domainkey"). For example, if the selector value for the DKIM key used with example.gov is "dkimkey", then the resulting DNS RR has the name dkimkey._domainkey.example.gov.

Like SPF, there are other tag=value pairs that need to be included in a DKIM RR. The full list of tags is listed in the specification [RFC6376], but relevant ones are listed below:

Table 4-5: DKIM RR Tag and Value Descriptions

Below is an example for the DKIM key that would be used to validate the DKIM signature above. Here, not all the flags are given:

adkimkey._domainkey.example.gov. IN TXT "v=DKIM1; k=rsa;

p=<base64 string>"

Note that unlike SPF, DKIM requires the use of semicolons between statements in a DNS RR.

Since DKIM public keys are encoded in DNS TXT resource records, no specialized software is needed to host DKIM public keys. Organizations that deploy DKIM should also deploy DNS security (DNSSEC) [RFC4033] [RFC4034] [RFC4035]. DNSSEC provides source authentication and integrity protection for DNS data. This prevents attackers from spoofing or intercepting and deleting responses for receivers’ DKIM key TXT queries.

Security Recommendation 4-6: Organizations should deploy DNSSEC to provide authentication and integrity protection to the DKIM DNS resource records.

There are several operations an email administrator will need to perform to maintain DKIM for an email service. New email services are acquired; DKIM keys are introduced, rolled (i.e., changed), and eventually retired, etc. Since DKIM requires the use of DNS, administrators need to take the nature of DNS into account when performing maintenance operations. [RFC5863] describes the complete set of maintenance operations for DKIM in detail, but the three most common operations are summarized below.

When initially deploying DKIM for enterprise email, or a new email service to support an organization, an administrator should insure that the corresponding public key is available for validation. Thus, the DNS entry with the DKIM public portion should be published in the sender's domain before the sending MTA begins using the private portion to generate signatures. The order should be:

DKIM keys may change for various purposes: suspected weakness or compromise, scheduled policy, change in operator, or because the DKIM key has reached the end of its lifetime.

Changing, or rolling, a DKIM key pair consists of introducing a new DKIM key before its use and keeping the old, outgoing key in the DNS long enough for clients to obtain it to validate signatures. This requires multiple DNS changes with a wait time between them. The relevant steps are:

The necessary period of time to wait before deleting the outgoing DKIM key’s TXT RR cannot be a universal constant value due to the nature of DNS and SMTP (i.e., mail queuing). An enterprise cannot be certain when all of its email has passed DKIM checks using its old key. An old DKIM key could still be queried for by a receiving MTA for hours (or potentially days) after the email had been sent. Therefore, the outgoing DKIM key should be kept in the DNS for a period of time (potentially a week) before final deletion.

If it is necessary to revoke or delete a DKIM key, it can be immediately retired by either be removing the key’s corresponding DKIM TXT RR or by altering the RR to have a blank p=. Either achieves the same effect (the client can no longer validate the signature) but keeping the DKIM RR with a blank p= value explicitly signals that the key has been removed.

Revoking a key is similar to deleting it but the enterprise may pre-emptively delete (or change) the DKIM RR before the sender has stopped using it. This scenario is possible when an enterprise wishes to break DKIM authentication and does not control the sender (i.e., a third party or rogue sender). In these scenarios, the enterprise can delete or change the DKIM RR in order to break validation of DKIM signatures. Additional deployment of DMARC (see Section 4.5) can be used to indicate that this DKIM validation failure should result in the email being rejected or deleted.

On the receiver side, email administrators should first make sure their MTA implementation has the functionality to verify DKIM signatures. Most major implementations have the functionality built-in or can be included using open source patches or a mail filter (i.e., milter). In some cases, the administrator may need to install additional cryptographic libraries to perform the actual validation.

Just as an organization that deploys DKIM should deploy DNSSEC, receivers that perform DKIM processing should also perform DNSSEC validation (if possible) on responses to DKIM TXT queries. A mail server should be able to send queries to a validating DNS recursive server if it cannot perform its own DNSSEC validation.

Security Recommendation 4-7: Organizations should enable DNSSEC validation on DNS servers used by MTAs that verify DKIM signatures.

DKIM assumes that the email came from the MTA domain that generated the signature. This presents some problems when dealing with certain mailing lists. Often, MTAs that process mailing lists change the bodies of mailing list messages—for example, adding a footer with mailing list information or similar. Such actions are likely to invalidate DKIM signatures, unless for example, a message length is specified in the signature headers, and the additions come beyond that length.

Fundamentally, mailing lists act as active mail parties. They receive messages from senders and resend them to recipients. Sometimes they send messages as they are received, sometimes the messages are bundled and sent as a single combined message, and sometimes recipients are able to choose their delivery means. As such, mailing lists should verify the DKIM signatures of incoming messages, and then re-sign outgoing messages with their own DKIM signature, made with the MTA’s public/private key pair. See [RFC6377], “DomainKeys Identified Mail (DKIM) and Mailing Lists,” also identified as IETF BCP 167, for additional discussion of DKIM and mailing lists.

Additional assurance can be obtained by providing mailing lists with a role-based (i.e., not a named individual) S/MIME certificate and digitally signing outgoing. Such signatures will allow verification of the mailing list signature using S/MIME aware clients such as Microsoft Outlook, Mozilla Thunderbird, and Apple Mail. See Sections 2.4.2 and 4.7 for a discussion of S/MIME. Signatures are especially important for broadcast mailing lists that are sent with message-From: addresses that are not monitored, such as “do-not-reply” email addresses.

Security Recommendation 4-8: Mailing list software should verify DKIM signatures on incoming mail and re-sign outgoing mail with new DKIM signatures.

Security Recommendation 4-9: Mail sent to broadcast mailing lists from do-not-reply or unmonitored mailboxes should be digitally signed with S/MIME signatures so that recipients can verify the authenticity of the messages.

As with SPF (subsection 4.2 above), DKIM does not prevent a spammer/advertiser from using a legitimately obtained domain to send unsolicited, DKIM-signed email. DKIM is used to provide assurance that the purported sender is the originator of the message, and that the message has not been modified in transit by an unauthorized intermediary.

An enterprise that uses third party senders for email services needs to have a policy in place for DKIM key management. The nature of DKIM requires that the sending MTA have the private key in order to generate signatures while the domain owner may only have the public portion. This makes key management controls difficult to audit and or impossible to enforce. Compartmentalizing DKIM keys is one approach to minimize risk when sharing keying material between organizations.

When using DKIM with cloud or contracted services, an enterprise should generate a unique key pair for each service. No private key should be shared between contracted services or cloud instances. This includes the enterprise itself, if email is sent by MTAs operated within the enterprise.

Security Recommendation 4-10: A unique DKIM key pair should be used for each third party that sends email on the organization's behalf.

Likewise, at the end of contract lifecycle, all DKIM keys published by the enterprise must be deleted or modified to have a blank p= field to indicate that the DKIM key has been revoked. This prevents the third party from continuing to send DKIM validated email.

SPF and DKIM were created so that email sending domain owners could give guidance to receivers about whether mail purporting to originate from them was valid, and thus whether it should be delivered, flagged, or discarded. Both SPF and DKIM offer implementation flexibility and different settings can have different effects at the receiver. However, neither SPF nor DKIM include a mechanism to tell receivers if SPF or DKIM are in active use, nor do they have feedback mechanism to inform sending domain owners of the effectiveness of their authentication techniques. For example, if a message arrives at a receiver without a DKIM signature, DKIM provides no mechanism to allow the receiver to learn if the message is authentic but was sent from a sender that did not implement DKIM, or if the message is a spoof.

DMARC [RFC7489] allows email sending domain owners to specify policy on how receivers can verify the authenticity of their email, how the receiver can handle email that fails to verify, and the frequency and types of report that receivers should send back. DMARC benefits receivers by removing the guesswork about which security protocols are in use, allowing more certainty in quarantining and rejecting inauthentic mail.

To further improve authentication, DMARC adds a link between the domain of the sender with the authentication results for SPF and DKIM. In particular, receivers compare the domain in the message-From: address in the message to the SPF and DKIM results (if deployed) and the DMARC policy in the DNS. The results of this data gathering are used to determine how the mail should be handled. Thus, when an email fails SPF and DKIM verification, or the message-From: domain-part doesn’t match the authentication results, the email can be treated as inauthentic according to the sending domain owners DMARC policy.

DMARC also provides a mechanism that allows receivers to send reports to the domain owner about mail claiming to originate from their domain. These reports can be used to illuminate the extent to which unauthorized users are using the domain, and the proportion of mail received that is from the purported sender.

DMARC policies work in conjunction with SPF and/or DKIM, so a mail domain owner intending to deploy DMARC must deploy SPF or DKIM or (preferably) both. To deploy DMARC, the sending domain owner will publish SPF and/or DKIM RRs in the DNS and calculate a signature for the DKIM header of every outgoing message. The domain owner also publishes a DMARC policy in the DNS advising receivers on how to treat messages purporting to originate from the sender’s domain. The domain owner does this by publishing its DMARC policy as a TXT record in the DNS¹²; identified by creating a _dmarc DNS record and publishing it in the sending domain name. For example, the DMARC policy for “example.gov” would reside at the fully qualified domain name _dmarc.example.gov.

When implementing email authentication for a domain for the first time, a sending domain owner is advised to first publish a DMARC RR with a “none” policy before deploying SPF or DKIM. This allows the sending domain owner to immediately receive reports indicating the volume of email being sent that purports to be from their domain. These reports can be used in crafting an email authentication policy that reduces the risk of errors.

Since the sending domain owner will be soliciting feedback reports by email from receivers, the administrator should establish email addresses to receive aggregate and failure reports. As the DMARC RR is easily discovered, the reporting inboxes could be subject to voluminous unsolicited bulk email (i.e., spam). Therefore, some kind of abuse counter-measures for these email in-boxes should be deployed.

Even if a sending domain owner does not deploy SPF or DKIM records it may be useful to deploy a DMARC record with policy p=none and a rua tag, to encourage receivers to send aggregate reports about the use to which the sender’s domain is being put. This can help with preliminary evaluation to determine whether a mail sender should mount SPF and DKIM defenses.

The DMARC policy is encoded in a TXT record placed in the DNS by the sending domain owner. Similar to SPF and DKIM, the DMARC policy is encoded in a series of tag=value pairs separated by semicolons. Common keys are:

Table 4-6: DMARC RR Tag and Value Descriptions

Like SPF and DKIM, the DMARC record is actually a DNS TXT RR. Like all DNS information, it should be signed using DNSSEC [RFC4033], [RFC4034], and [RFC4035] to prevent an attacker from spoofing the DNS response and altering the DMARC check by a client.

Below are several examples of DMARC policy records using the above tags. The most basic example is a DMARC policy that effectively does not assert anything and does not request the receiver send any feedback reports, so it is, in effect, useless.

_dmarc.example.gov 3600 IN TXT “v=DMARC1; p=none;“

An agency that is preparing to deploy SPF and/or DKIM, or has deployed these technologies, but may not be confident in their current policies may request aggregate reports from receivers, but otherwise advises no specific action. The agency can do so by publishing a p=none policy as in the example below.

_dmarc.example.gov 3600 IN TXT “v=DMARC1; p=none; rua=reports@example.gov;“

An agency that has deployed SPF and DKIM and advises receivers to reject any messages that fail these checks would publish a p=reject policy as in the example below. Here, the agency also wishes to receive aggregate reports on a daily basis (the default).

_dmarc.example.gov 3600 IN TXT “v=DMARC1; p=reject; rua=reports@example.gov;“

The agency in the process of deploying DKIM (but has confidence in their SPF policy) may wish to receive feedback solely on DKIM failures, but does not wish to be inundated with feedback, so requests that the policy be applied to a subset of messages received. In this case, the DMARC policy would include the fo= option to indicate only DKIM failures are to be reported and a pct= value of 10 to indicate that only 1 in 10 email messages should be subjected to this policy (and subsequent reporting on a failure). Note that this is not a wise strategy in that it reduces the enforcement policy and the completeness of reporting. The use of the pct value in values other than 0 or 100 (i.e., none or full) limits DMARC effectiveness and usefulness of reporting. It is also burdensome for receivers to choose that intermediate percentage of mail for testing.

_dmarc.example.gov 3600 IN TXT “v=DMARC1; p=none; pct=10; fo=d; ruf=reports@example.gov;“

An agency with several subdomains may wish to have a single unified policy, in which case a DMARC RR with the sp= tag is used. In this example, the domain has a policy to reject any mail from a subdomain of example.gov that fails checks, while only quarantining email that failed checks from the parent domain.

_dmarc.example.gov 3600 IN TXT “v=DMARC1; p=quarantine; sp=reject; rua=reports@example.gov;”

Security Recommendation 4-11: Sending domain owners who deploy SPF and/or DKIM are recommended to publish a DMARC record signaling to mail receivers the disposition expected for messages purporting to originate from the sender’s domain.

Receivers of email purporting to originate from a given domain will look up the SPF, DKIM and DMARC records in the DNS and act on the policies encoded therein. The recommended processing order per [RFC7489] is given below. Note that it is possible that some steps could be done in parallel and local policy may alter the order of some steps (i.e., steps 2, 3 and 4).

Note that local email processing policy may override a sending domain owner’s stated DMARC policy. The receiver should also store the results of evaluating each received message in some persistent form for the purpose of compiling aggregate reports.

Even if steps 2-5 in the above procedure yield no SPF or DKIM records to evaluate the message, it is still useful to send aggregate reports based on the sending domain owner’s DMARC preferences, as it helps shape sending domain responses to spam in the system.

Security Recommendation 4-12: Mail receivers who evaluate SPF and DKIM results of received messages are recommended to dispose them in accordance with the sending domain’s published DMARC policy, if any. They are also recommended to initiate failure reports and aggregate reports according to the sending domain’s DMARC policies.

DMARC can be seen as consisting of two components: a policy on linking SPF and DKIM checks to the message-From: address, and a reporting mechanism. The reason for DMARC reporting is so that domain owners can get feedback on their SPF, DKIM, Identifier Alignment and message disposition policies so these can be made more effective. The DMARC protocol specifies a system of aggregate reports sent by receivers on a periodic basis, and failure reports sent on a message-by-message basis for email that fail some component part of the DMARC checks. The specified form in which receivers send aggregate reports is as a compressed (zipped) XML file based on the AFRF format [RFC6591], [RFC7489]¹³. Each aggregate report from a mail receiver back to a particular domain owner includes aggregate figures for successful and unsuccessful message authentications including:

Based on the return flow of aggregate reports from the aggregation of all receivers, a domain owner can build up a picture of the email being sent and how it appears to outside receivers. This allows the domain owner to identify gaps in email infrastructure and policy and how (and when) it can be improved. In the early stages of building up this picture, the sending domain should set a DMARC policy of p=none, so the ultimate disposition of a message that fails some checks rests wholly on the receiver's local policy. As DMARC aggregate reports are collected, the domain owner will have a quantitatively better assessment of the extent to which the sender’s email is authenticated by outside receivers and will be able to set a policy of p=reject, indicating that any message that fails the SPF, DKIM and alignment checks really should be rejected via a SMTP reply code signaling rejection, or silently discarding the message. Note, however, mailing lists and some other email uses (e.g., “share this post” type links) may fail DMARC validation and may need to be considered when forming a policy. From their own traffic analysis, receivers can develop a determination of whether a sending domain owner’s p=reject policy is sufficiently trustworthy to act on.

Failure reports from receivers to domain owners help debug and tune the component SPF and DKIM mechanisms as well as alerting the domain owner that their domain is being used as part of a phishing/spam campaign. Typical initial rollout of DMARC in an enterprise will include the ruf tag with the values of the fo tag progressively modified to capture SPF debugging, DKIM debugging or alignment debugging. Failure reports are expensive to produce and bear a real danger of providing a DDoS source back to domain owners, so when sufficient confidence is gained in the integrity of the component mechanisms, the ruf tag may be dropped from DMARC policy statements if the sending domain no longer wants to receive failure reports. Note however that failure reports can also be used to alert domain owners about phishing attacks being launched using their domain as the purported sender and therefore dropping the ruf tag is not recommended.

The same AFRF report format as for aggregate reports [RFC6591], [RFC7489] is also specified for failure reports, but the DMARC standard updates it for the specificity of a single failure report:

The rua and ruf tags typically specify mailto: addresses in the sender’s domain. These reporting addresses are normally assumed to be in the same domain as the purported sender, but not always. Cloud providers and contracted services may provide DMARC report collection as part of their service offerings. In these instances, the mailto: domain will differ from the sending domain. To prevent DMARC reporting being used as a DoS vector, the owner of the mailto: domain must signal its legitimacy by posting a DMARC TXT DNS record with the Fully Qualified Domain Name (FQDN):

original-sender-domain._report._dmarc.mailto-domain

For example, an original message sent from example.gov is authenticated with a DMARC record:

_dmarc.example.gov. IN TXT "v=DMARC1; p=reject; rua=mailto:reports.example.net"

The recipient then queries for a DMARC TXT RR at example.gov._report._dmarc.example.net and checks the rua tag includes the value rua=mailto:reports.example.net to ensure that the address specified in the sending domain owner's DMARC record is the legitimate receiver for DMARC reports.

Note that, as with DKIM, DMARC records require the use of semicolons between tags.

The message authentication devices of SPF, DKIM and DMARC are designed to work directly between a sender domain and a receiver domain. The message envelope and RFC5322.From address pass through a series of MTAs and are authenticated by the receiver. The DKIM signature, message headers and message body arrive at the receiver unchanged. The email system has additional complexities as there are a variety of message forwarding activity that will very often either modify the message or change the apparent message-From: domain. For example, user@example.gov sends a message to ourgroup@example.net, which is subsequently forwarded to all members of the mail group. If the mail group software simply relays the message, the envelope-From: address denoting the forwarder differs from the message-From: address, denoting the original sender. In this case DMARC processing will rely on DKIM for authentication. If the forwarder modifies the message-From: field to match the HELO of the sending MTA (see Section 2.3.1), SPF may authenticate, but the modified header will make the DKIM signature invalid.

Another example is the use of some third-party email scanning services. In cases where email is received by a domain, and then “shunted” to a third party for scanning. The email is then sent back (via SMTP) to the original receiving domain, which may consider the incoming email as a newly seen message. This is sometimes seen in federal agencies that use one provider for email and another provider for Trusted Internet Connection (TIC) services (see Figure 4-2). Incoming mail is received by the cloud provider, then forwarded to the TIC provider for scanning. After scanning, the email message is sent back to the cloud provider. Upon the second delivery, the message appears to be spoofed, as SPF and DMARC validations will fail.

Figure 4-3: Use of Cloud Email with Third Party Email Scanning

Table 4-7 below summarizes the various forwarding techniques and their effect on domain-based authentication mechanisms:

Table 4-7: Common relay techniques and their impact on domain-based authentication

One solution that can reduce the impact due to domain-based validation failures is the Authenticated Receiver Chain (ARC)¹⁴ extension. ARC is an extension of DKIM that generates a chain of custody (called an “ARC Set”) as an email message moves from one MTA to another. The ARC Set is actually three new headers that contain a receiver’s authentication results, an additional signature over the message a second signature over selected headers (including the other ARC headers). Downstream receivers include their authentication results as well as also signing the previous handler’s ARC headers. ARC can be used to give information about authentication results throughout the chain of possession. ARC is not perfect because a malicious actor can alter the ARC Set, so ARC should only be seen as a purported chain of custody and a way to use indirect email flows without invalidating previous authentication results.

ARC may be especially useful when using third party email scanners (or TIC providers). A federal agency’s cloud email provider can forward all email to the TIC provider for scanning. Upon resubmission of the email, the cloud email provider can examine the ARC chain to see that the incoming email was validated previously and only returning from (known) third party scanning and override the sending domain’s DMARC policy in favor of a local policy about messages received from the TIC provider that has a valid ARC chain. As of the time of writing, ARC is still being finalized in the IETF. However, email service providers and administrators are encouraged to look at ARC as a means to minimize the impact of domain-based validation failures when forwarding email, or when using email mailing lists.

Forwarding in general creates problems for DMARC results processing, and as of this writing, universal solutions are still in development. There is a currently existing set of mitigations that could be used by the mail relay and by the receiver, but would require modified MTA processing from traditional SPF and DKIM processing:

It should also be noted that if one or the other (SPF or DKIM) authentication and domain alignment checks pass, then the DMARC policy could be satisfied.

At the receiver side, if a message fails DMARC and is bounced (most likely in the case where the sender publishes a p=reject policy), then a mailing list may respond by unsubscribing the recipient. Mailing list managers should be sensitive to the reasons for rejection and avoid unsubscribing recipients if the bounce is due to message authentication issues. If the mailing list is in a domain where the recommendations in this document can be applied, then such mailing list managers should be sensitive to and accommodate DMARC authentication issues. In the case where the mailing list is outside the domain of influence, the onus is on senders and receivers to mitigate the effects of forwarding as best they can. ARC may also be useful in mitigating these types of errors, as a receiver can examine the ARC chain for additional information about how the message traveled through different MTAs.

In addition to authenticating the sender of a message, the message contents can be authenticating with digital signatures. Signed email messages protect against phishing attacks, especially targeted phishing attacks, as users who have been conditioned to expect signed messages from co-workers and organizations are likely to be suspicious if they receive unsigned messages instructing them to perform an unexpected action [GAR2005]. For this reason, the Department of Defense requires that all e-mails containing a hyperlink or an attachment be digitally signed [DOD2009].

Because it interoperates with existing PKI and most deployed software, S/MIME is the recommended format for digitally signing messages. Users of most email clients who receive S/MIME signed messages from organizations that use well-known CAs will observe that the message signatures are automatically validated, without the need to manually add or trust certificates for each sender. If users receive mail that originates from a sender that uses a non-public CA, then either the non-public CA must be added or else each S/MIME sender must be individually approved. Today, the US Government PIV [FIPS 201] cards are signed by well-known CAs, whereas the US Department of Defense uses CAs that are generally not trusted outside the Department of Defense. Thus, email signed by PIV cards will generally be validated with no further action, while email signed by DoD Common Access Cards will result in a warning that the sender’s certificate is not trusted.

Figure 4-4: Two models for sending digitally signed mail.

Organizations (and individuals in organizations) can use S/MIME digital signatures to certify email that is sent within or external to the organization. Because support for S/MIME is present in many modern mail clients¹⁵, S/MIME messages that are signed with a valid digital signature will automatically validate when they are displayed. This is particularly useful for messages that are designed to be read but not replied to—for example, status reports and alerts that are sent programmatically, as well as messages that are sent to announcement-only distribution lists.

To send S/MIME digitally signed messages, organizations must first obtain a S/MIME certificate where the sender matches the message-From: address that will be used to sign the messages. Typically, this will be done with a S/MIME certificate and matching private key that corresponds to the role, rather than to an individual.¹⁶ Once a certificate is obtained, the message is first composed. Next, software uses both the S/MIME certificate and the private portion of their S/MIME key pair to generate the digital signature. S/MIME signatures contain both the signature and the signing certificate, allowing recipients to verify the signed message without having to fetch the certificate from a remote server; the certificate itself is validated using PKI. Sending S/MIME signed messages thus requires either a MUA that supports S/MIME and the necessary cryptographic libraries to access the private key and generate the signature, or else an intermediate program that will sign the message after it is created but before it is delivered (Fig 4-3).

The receiver of the signed S/MIME message then uses the sender's public key (from the sender's attached X.509 certificate) and validates the digital signature. The receiver should also check to see if the senders certificate has a valid PKIX chain back to a root certificate the receiver trusts to further authenticate the sender. Some organizations may wish to configure MUAs to perform real-time checks for certificate revocation and an additional authentication check (See Section 5.2.2.3).

The principal barrier to using S/MIME for end-user digital signatures has been the difficulty of arranging for end-users to obtain S/MIME certificates. One approach is to issue S/MIME credentials in physical identity tokens, as is done with the US Government’s PIV (Personal Identity Verification) cards [FIPS 201]. Individuals can obtain free S/MIME certificates from a number of online providers, who verify the individual’s address with an email challenge.

The principal barrier to using S/MIME for signing organizational email has been the lack of attention to the issue, since only a single certificate is required for signing mail and software for verifying S/MIME signatures is already distributed.

Security Recommendation 4-11: Use S/MIME signatures for assuring message authenticity and integrity.

Security Recommendation 4-1: Organizations are recommended to deploy SPF to specify which IP addresses are authorized to transmit email on behalf of the domain. Domains controlled by an organization that are not used to send email, for example Web only domains, should include an SPF RR with the policy indicating that there are no valid email senders for the given domain.

Security Recommendation 4-2: Organizations should deploy DNSSEC for all DNS name servers and validate DNSSEC responses from all systems that receive email.

Security Recommendation 4-3: Federal agency administrators shall only use keys with approved algorithms and lengths for use with DKIM.

Security Recommendation 4-4: Administrators should ensure that the private portion of the key pair is adequately protected on the sending MTA and that only the MTA software has read privileges for the key. Federal agency administrators should follow FISMA control SC-12 [SP800-53] guidance with regards to distributing and protecting DKIM key pairs.

Security Recommendation 4-5: Each sending MTA should be configured with its own private key and its own selector value, to minimize the damage that may occur if a private key is compromised.

Security Recommendation 4-6: Organizations should deploy DNSSEC to provide authentication and integrity protection to the DKIM DNS resource records.

Security Recommendation 4-7: Organizations should enable DNSSEC validation on DNS servers used by MTAs that verify DKIM signatures.

Security Recommendation 4-8: Mailing list software should verify DKIM signatures on incoming mail and re-sign outgoing mail with new DKIM signatures.

Security Recommendation 4-9: Mail sent to broadcast mailing lists from do-not-reply or unmonitored mailboxes should be digitally signed with S/MIME signatures so that recipients can verify the authenticity of the messages.

Security Recommendation 4-10: A unique DKIM key pair should be used for each third party that sends email on the organization's behalf.

Security Recommendation 4-11: Use S/MIME signatures for assuring message authenticity and integrity.

Cleartext mail messages are submitted by a sender, transmitted hop-by-hop over a series of relays and delivered to a receiver. Any successful man-in-the-middle can intercept such traffic and read it directly. Any bad actor, or organizationally privileged actor, can read such mail on the submission or delivery systems. Email transmission security can be assured by encrypting the traffic along the path. The Transport Layer Security protocol (TLS) [RFC5246] protects confidentiality by encrypting bidirectional traffic and prevents passive monitoring. TLS relies on public key cryptography and uses X.509 certificates [RFC5280] to encapsulate the public key, and the Certificate Authority (CA) system to issue certificates and authenticate the origin of the key.

In recent years the CA system has become the subject of attack and has been successfully compromised on several occasions.¹⁷¹⁸ The DANE protocol [RFC6698] is designed to overcome problems in the CA system by providing an alternative channel for authenticating public keys using DNSSEC. The result is that the same trust relationships used to certify IP addresses can be used to certify servers operating on those addresses the mechanisms that combine to improve the assurance of email transmission security are described in section 5.2.

Encryption at the transport layer gives assurance of the integrity of data in transit, but senders and receivers who want end-to-end assurance, (i.e., mailbox to mailbox) of confidentiality have two alternative mechanisms for achieving this: S/MIME [RFC5750] and OpenPGP [RFC4880]. Both protocols are capable of signing (for authentication) and encryption (for confidentiality). The S/MIME protocol is deployed to sign and/or encrypt message contents, using keys stored as X.509 certificates and a PKI (See Section 2.4.2) while OpenPGP uses a different certificate and a Web-of-Trust model for authentication of identities (See Section 2.4.3). Both of these protocols have the issue of trustworthy certificate publication and discovery. Traditionally, this was done using dedicated key/certificate servers, LDAP servers, etc., but these methods are not always easy to deploy and standardize for every enterprise. These certificates can be published through the DNS by a different implementation of the DANE mechanism for S/MIME [RFC8162] and OpenPGP [RFC7929]. S/MIME and OpenPGP, with their strengthening by DANE authentication are discussed below.

Email proceeds towards its destination from a Message Submission Agent (MSA), through a sequence of Message Transfer Agents (MTAs), to a Message Delivery Agent (MDA), as described in Section 2. This translates to the use of SMTP [RFC5321] for submission and hop-by-hop transmission and IMAP [RFC3501] or POP3 [RFC1939] for final delivery into a recipient’s mailbox. TLS [RFC5246] can be used to protect email in transit for one or more hops, but intervening hops may be under autonomous control, so a securely encrypted end-to-end path cannot be guaranteed. This is discussed further in section 5.2.1. Opportunistic encryption over some portions of the path can provide “better-than-nothing” security. The use of STARTTLS [RFC3207] is a standard method for establishing a TLS connection. TLS has a secure handshake that relies on asymmetric encryption, to establish a secure session (using symmetric encryption). As part of the handshake, the server sends the client an X.509 certificate containing its public key, and the cipher suite and symmetric key are negotiated with a preference for the optimally strongest cipher that both parties support. SMTP clients have traditionally not verified the server’s certificate due to the lack of an appropriate mechanism to specify allowable certificates and certificate authorities. The newly adopted RFC 7672 [RFC 7672] rectifies this, by providing rules for applying the DANE protocol to SMTP servers. The use of DANE in conjunction with SMTP is discussed Section 5.2.4.

Ultimately the entire path from sender to receiver will be protected by TLS. But this may consist of many hops between MTAs, each the subject of a separate transport connection. These are not compelled to upgrade to TLS at the same time, however in the patchwork evolutionary development of the global mail system, this cannot be completely guaranteed. There may be some MTAs along the route uncontrolled by the sender or receiver domains that have not upgraded to TLS. In the interim until all mail nodes are certifiably secure, the principle is that some incrementally improving security is better than no security, so opportunistic TLS (using DANE or other methods to validate certificates) should be employed at every possible hop.

Traditionally, sending email begins by opening an SMTP connection over TCP and entering a series of cleartext commands, possibly even including usernames and passwords. This leaves the connection exposed to potential monitoring, spoofing, and various man-in-the-middle interventions. A clear improvement would be to open a secure connection that is encrypted so that the message contents cannot be passively monitored, and third parties cannot spoof message headers or contents. Transport Layer Security (TLS) offers the solution to these problems.

TCP provides a reliable, flow-controlled connection for transmitting data between two peers. Unfortunately, TCP provides no built-in security. Transport connections carry all manner of sensitive traffic, including web pages with financial and sign-in information, as well as email messages. This traffic can only be secured through physical isolation, which is not possible on the Internet, or by encrypting the traffic.

The Secure Sockets Layer (SSL) was developed to provide a standard protocol for encrypting TCP connections. SSL evolved into Transport Layer Security (TLS), the most recent version at the time of writing being Version 1.3 [RFC8446]. TLS negotiates a secure connection between initiator and responder (typically client and server) parties. The negotiation entails the exchange of the server’s certificate, and possibly the client’s certificate, and agreement on a cipher to use for encrypting the data. In essence, the protocol uses the public-private key pair: the public key in the server’s certificate, and the server’s closely held private key, to negotiate a symmetric algorithm and establish a key known to both parties, and with which both can encrypt, transmit and decrypt the application data. RFC 5246 Appendix A describes a range of permissible ciphers, and the parties agree on one from this set. This range of ciphers may be restricted on some hosts by local policy (such as only ciphers Approved for federal use). Data transmitted over the connection is encrypted using the negotiated session key. At the end, the connection is closed, and the session key can be deleted (but not always, see below).

Negotiating a TLS connection involves a significant time and processor load, so when the two parties have the need to establish frequent secure connections between them, a session resumption mechanism allows them to continue with the previously negotiated cipher, for a subsequent connection.

TLS gains its security from the fact that the server holds the private key securely and the public key can be authenticated due to it being wrapped in an X.509 certificate that is guaranteed by some Certificate Authority. If the Certificate Authority is somehow compromised, there is no guarantee that the key in the certificate is truly the one belonging to the server, and a client may inadvertently negotiate with a man-in-the-middle. An investigation of what X.509 certificates are, how they work, and how they can be better secured, follows.

Security Recommendation 5-1: Organizations are recommended only use approved TLS versions listed in NIST SP 800-52 [SP800-52] with FIPS approved cryptographic modules.

The idea of certificates as a secure and traceable vehicle for locating a public key, its ownership and use was first proposed by the Consultative Committee for International Telephony and Telegraphy (CCITT), now the International Telecommunications Union (ITU). The X.509 specification was developed and brought into worldwide use as a result. In order to vest a certificate with some authority, a set of Certificate Authorities is licensed around the world as identifiable authentic sources. Each certificate hierarchy has a traceable root for authentication and has specific traceable requirements for revocation if that is necessary. As a certificate has a complex set of fields, the idea of a certificate profile has more recently come into play. X.509 certificate formats are described in Section 5.2.2.1, their authentication in Section 5.2.2.2, and possible revocation in Section 5.2.2.3. The profile concept and a specific example are described in Section 5.2.2.4

A trusted Certificate Authority (CA) is licensed to validate applicants’ credentials, store each applicant’s public key in a X.509 [RFC5280] structure, and digitally sign it with the CA’s private key. Each applicant must first generate their own public and private key pair, save the private key securely, and wrap the public key into an X.509 request. The openssl req command is an example of how to do this on Unix/Linux systems with OpenSSL¹⁹ installed. Many CAs will generate a certificate without receiving a request (in effect, generating the request themselves on the customer’s behalf). The resulting digitally encoded structure is transmitted to the CA, vetted according to the CA’s policy, and a certificate is issued. An example certificate is given below in Figure 5-1, with salient fields described.

Certificate: Data: Version: 3 (0x2) Serial Number: 760462 (0xb9a8e) Signature Algorithm: sha1WithRSAEncryption Issuer: C=IL, O=ExampleCA LLC, OU=Secure Digital Certificate Signing, CN=ExampleCA Primary Intermediate Server CA Validity

Not Before: Aug 20 15:32:55 2013 GMT Not After : Aug 21 10:17:18 2014 GMT Subject: description=I0Yrz4bhzFN7q1lb, C=US, CN=www.example.com/emailAddress=admin@example.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:b7:14:03:3b:87:aa:ea:36:3b:b2:1c:19:e3:a7: 7d:84:5b:1e:77:a2:44:c8:28:b7:c2:27:14:ef:b5: 04:67 Exponent: 65537 (0x10001)

X509v3 extensions: X509v3 Basic Constraints: CA:FALSE X509v3 Key Usage: Digital Signature, Key Encipherment, Key Agreement X509v3 Extended Key Usage: TLS Web Server Authentication X509v3 Subject Key Identifier: C2:64:A8:A0:3B:E6:6A:D5:99:36:C2:70:9B:24:32:CF:77:46:28:BD X509v3 Authority Key Identifier: keyid:EB:42:34:D0:98:B0:AB:9F:F4:1B:6B:08:F7:CC:64:2E:EF:0E: 2C:45 X509v3 Subject Alternative Name: DNS:www.example.com, DNS:example.com X509v3 Certificate Policies: Policy: 2.23.140.1.2.1 Policy: 1.3.6.1.4.1.23223.1.2.3

CPS: http://www.exampleCA.com/policy.txt User Notice: Organization: ExampleCA Certification Authority Number: 1 Explicit Text: This certificate was issued according to the Class 1 Validation requirements of

the ExampleCA CA policy, reliance only for the intended purpose in compliance of the relying party obligations.

X509v3 CRL Distribution Points: Full Name: URI:http://crl.exampleCA.com/crl.crl

Authority Information Access: OCSP - URI:http://ocsp.exampleCA.com/class1/server/ocsp CA Issuers - URI:http://aia.exampleCA.com/certs/ca.crt

X509v3 Issuer Alternative Name: URI:http://www.exampleCA.com/ Signature Algorithm: sha1WithRSAEncryption 93:29:d1:ed:3a:2a:91:50:b4:64:1d:0f:06:8a:79:cf:d5:35: ba:25:39:b0:dd:c0:34:d2:7f:b3:04:5c:46:50:2b:97:72:15: ea:3a:4f:b6

Figure 5-1: Example of X.509 Certificate

The certificate given above is an example of an end certificate. Although it claims to be signed by a well-known CA, anyone receiving this certificate in communication has the problem of authenticating that signature. For this, full PKIX authentication back to the root certificate is required. The CA issues a well-known self-signed certificate containing its public key. This is the root certificate. A set of current root certificates, often numbering in the hundreds of certificates, are held by individual browser developers and operating system suppliers as their set of trusted root certificates. The process of authentication is the process of tracing the end certificate back to a root certificate, through a chain of zero or more intermediate certificates.

Every certificate has a period of validity typically ranging from 30 days up to a number of years. There may, however, be reasons to revoke a certificate prior to its expiration, such as the compromise or loss of the private key [RFC5280]. The act of revocation is associated with the CA publishing a certificate revocation list. Part of authenticating a certificate chain is perusing the certificate revocation list (CRL) to determine if any certificate in the chain is no longer valid. The presence of a revoked certificate in the chain should result in failure of authentication. Among the problems of CRL management, the lack of real-time revocation checks leads to non-determinism in the authentication mechanism. Problems with revocation led the IETF to develop a real-time revocation management protocol, the Online Certificate Status Protocol (OCSP) [RFC6960]. Mozilla has now taken the step to deprecate CRLs in favor of OCSP.

The Federal Public Key Infrastructure (FPKI) Policy Authority has specified profiles (called the FPIX profile) for two types of X.509 version 3 certificates that can be used for confidentiality and integrity protection of federal email systems [FPKI-CERT]. The applicable certificate profile is identified by the KeyPurposeId with value id-kp-emailProtection (1.3.6.1.5.5.7.3.4) and includes the following:

The overall FPIX profile is an instantiation of IETF’s PKI profile developed by the PKIX working group (and hence called the PKIX profile) [PKIX] with unique parameter settings for Federal PKI systems. Thus, a FPIX certificate profile complements the corresponding PKIX certificate profile. The following is a brief overview of the two applicable FPIX profiles referenced above.

The public key of a Key Management certificate is used by a device (e.g., a Mail Transfer Agent (MTA) in this context) to set up a session key (a symmetric key) with its transacting entity (e.g., the next-hop MTA in this context). The parameter values specified in the profile for this certificate type, for some of the important fields are:

Unlike the World Wide Web, where the URL indicates that the secure variant (i.e., HTTPS) is in use, an email sender has only the email address, “user@domain”, to signal the destination and no way to direct that the channel must be secured. This is an issue not just on a sender-to-receiver basis, but also on a transitive basis, as SMTP is not an end-to-end protocol but instead a protocol that sends mail messages as a series of hops (i.e., MUA, MSA, multiple MTAs, etc.). Not only is there no way to signal that message submission must be secure, there is also no way to signal that any hop in the transmission should be secure. STARTTLS was developed to address some of the shortcomings of this system.

RFC 3207 [RFC3207] describes an extension to SMTP that allows an SMTP client and server to use TLS to provide private, authenticated communication across the Internet. This gives SMTP agents the ability to protect some or all of their communications from eavesdroppers and attackers. If the client initiates the connection over a TLS-enabled port (e.g., port 465 was previously used for SMTP over SSL), the server advertises that the STARTTLS option is available to connecting clients. The client can then issue the STARTTLS command in the SMTP command stream, and the two parties proceed to establish a secure TLS connection. An advantage of using STARTTLS is that the server can offer SMTP service on a single port, rather than requiring separate port numbers for secure and cleartext operations. Similar mechanisms are available for running TLS over IMAP and POP protocols.

When STARTTLS is initiated as a request by the server side, it may be susceptible to a downgrade attack, where a man-in-the-middle (MITM) is in place. In this case the MITM receives the STARTTLS request from the server reply to a connection request and scrubs it out. The initiating client sees no TLS upgrade request and proceeds with an unsecured connection (as originally anticipated). Likewise, most MTAs default to sending messages over unencrypted TCP if certificate validation fails during the TLS handshake.

Domains can signal their desire to receive email over TLS by publishing a public key in their DNS records using DANE (Section 5.2.4). Domains can also configure their email servers to reject mail that is delivered without being preceded by a TLS upgrade. Unfortunately, doing so at the present time may result in email not being delivered from clients that are not capable of TLS. Furthermore, mail that is sent over TLS will still be susceptible to MITM attacks unless the client verifies the that the server’s certificate matches the certificate that is advertised using DANE.

If the client wants to ensure an encrypted channel, there is a new proposal to allow for signaling the required use of TLS via the REQUIRETLS SMTP option²⁰. This option is set by senders to indicate that they wish TLS to be used for each hop and not deliver to the next hop if TLS is not available. If the server wishes to indicate that an encrypted channel is available and should be used by senders, this can be indicated through an advertisement using DANE. If the end user wants security over the message content, then the message should be encrypted using S/MIME or OpenPGP, as discussed in Section 5.3.

In this long transition period towards “TLS everywhere,” there will be security gaps where some MTA to MTA hop offers TCP only. In these cases, the receiving MTA suggestion of STARTTLS can be downgraded by the above MITM attack. In such cases, a channel thought secure by the end user can be compromised. A mitigating consolation is that opportunistic security (i.e., use encryption when available) is better than no security. The more mail administrators who actively deploy TLS, the fewer opportunities for effective MITM attacks. In this way global email security improves incrementally.

Security Recommendation 5-2: TLS-capable servers should prompt clients to invoke the STARTTLS command. TLS clients should attempt to use STARTTLS for SMTP, either initially, or issuing the command when offered.

For years, TLS has solved the problem of distributing public keys by using a certificate, signed by some well-known Certification Authority (CA). Every browser developer and operating system supplier maintain a list of CA root certificates as trust-anchors. These are called the software’s root certificates and are stored in the root certificate store. The PKIX procedure allows the certificate recipient to trace a certificate back to the root. So long as the root certificate remains trustworthy, and the authentication concludes successfully, the client can proceed with the connection.

Currently, there are hundreds of organizations acting as CAs on the Internet. If one CA infrastructure or vetting procedure is compromised, the attacker can obtain the CA’s private key, or get issued certificates under a false name. There is no limitation of scope for the global PKI, and a compromise of a single CA damages the integrity of the entire PKI system.

Aside from a CA compromise, some CAs have engaged in poor security practices. For example, some CAs have issued wildcard certificates that allow the holder to issue sub-certificates for any domain or entity, anywhere in the world.²¹

DANE introduces mechanisms for domains to specify to a client which certificates should be trusted for the domain. With DANE, a domain owner can publish DNS records that declare clients should only trust certificates from a particular CA or that they should only trust only a specific certificate or public key. Essentially, DANE replaces reliance on the security provided by the CA system with reliance on the security provided by DNSSEC.

DANE complements TLS. The TLS handshake yields an encrypted connection between a server and a client and provides a server’s X.509 certificate to the client.²² The TLS protocol does not define how the certificate should be authenticated. Some implementations may do this as part of the TLS handshake, and some may leave it to the application to perform authentication. Whichever way is used, there is still a vulnerability: a CA can issue certificates for any domain, and if that a CA is compromised (as has happened more than once all too recently), an attacker can have it can issue a replacement certificate for any domain and take control of a server’s connections. Ideally, issuance and delivery of a certificate should be tied absolutely to the given domain. DANE creates this explicit link by allowing the server domain owner to create a TLSA resource record in the DNS [RFC6698] [RFC7671], which identifies the certificate, its public key, or a hash of either. When the client receives an X.509 certificate in the TLS negotiation, it looks up the TLSA RR for that domain and matches the TLSA data against the certificate as part of the client’s certificate validation procedure.

DANE has a number of usage models (called Certificate Usages) to accommodate users who require different forms of authentication. These Certificate Usages are given mnemonic names [RFC7218]:

In DANE-EE(3) the server certificate is directly specified by the TLSA record. Thus, the certificate may be self-issued, or it may be issued by a well-known CA. The certificate may be current or expired. Indeed, operators may employ either a public or a private CA for their DANE certificates and publish a combination of “3 1 1” and “2 1 1” TLSA records, both of which should match the server chain and be monitored. This allows clients to verify the certificate using either DANE or the traditional Certificate Authority system, significantly improving reliability.

Secure SMTP communications involves additional complications because of the use of mail exchanger (MX) and canonical name (CNAME) DNS RRs, which may cause mail to be routed through intermediate hosts or to final destinations that reside at different domain names. [RFC 7671] and [RFC7672] describe a set of rules that are to be used for finding and interpreting DANE policy statements.

As originally defined, TLS did not offer a client the ability to specify a particular hostname when connecting to a server; this was a problem in the case where the server offers multiple virtual hosts from one IP address, and there was a desire to associate a single certificate with a single hostname. [RFC6066] defines a set of extensions to TLS that include the Server Name Indication (SNI), allowing a client to specifically reference the desired server by hostname, and the server can respond with the correct certificate.

[RFC7671] and [RFC7672] require the client to send SNI just in case the server needs this to select the correct certificate. There is no obligation on the server to employ virtual hosting, or to return a certificate that matches the client's SNI extension. There is no obligation on the client to match anything against the SNI extension. Rather, the requirement on the client is to support at least the TLSA base domain as a reference identifier for the peer identity when performing name checks (matching against a TLSA record other than DANE-EE(3)). With CNAME expansion either as part of MX record resolution or address resolution of the MX exchange, additional names must be supported as described in [RFC7671] and [RFC7672].

A DANE matching condition also requires that the connecting server match the SubjectAltName from the delivered end certificate to the certificate indicated in the TLSA RR. DANE-EE authentication allows for the server to deliver a self-signed certificate. In effect, DANE-EE is simply a vehicle for delivering the public key. Authentication is inherent in the trust provided by DNSSEC, and the SNI check is not required.

Some email providers regard the requirement that DANE records be secured with DNSSEC as a major barrier to deployment. As an alternative, they have proposed SMTP Strict Transport Security [RFC8461], which relies on records that are announced via DNS but authenticated using information distributed via HTTPS. The goal of MTA-STS is the same as DANE: to have a way for a receiving MTA to publish its TLS policy and mitigate Man-in-the-Middle (MITM) spoofing. SMA-STS can be used with DANE, as neither method precludes the use of the other.

MTA-STS works by publishing both a special TXT RR in the DNS and a policy document at a Well-Known URL. The client obtains both artifacts before attempting to establish a connection to the receiving domain’s mail servers.

The receiving domain administrator generates an MTA-STS policy RR (a TXT Text RR) with the following tag:value pairs (separated by “;”):

Table 5-1: MTA-STS Resource Record Tags and Descriptions

The MTA-STS RR is published as a TXT RR using the receiving domain with _mta-sts prepended. For example, if the receiving domain is example.gov, the MTA-STS RR is:

_mta-sts.example.gov IN TXT “v=STSv1; id=20170101000000Z”

The receiver then published a detailed policy document at a well-known URL consisting of the domain with mta-sts prepended and .well-known/mta-sts.txt as the path. So, in the example above, the URL containing the MTA-STS policy for example.gov would be found at:

https://mta-sts.example.gov/.well-known/mta-sts.txt

The policy must only be accessible via HTTPS and contains a plain/text resource used by the client to connect to the receiver. The document contains tag:values pairs, separated by newlines. The tags are:

Table 5-2: MTA-STS Policy Tags and Descriptions

An example MTA-STS policy for example.gov may look like the following (found at the URL above):

version: STSv1 mode: enforce mx: mail1.example.gov. mx: mail2.example.gov. max_age:86400

In the above, example.gov lists two mail servers for the domain (mail1.example.gov and mail2.example.gov). The domain also sets its policy to enforce, meaning that if a client sees a server certificate that lacks mail1.example.gov or mail2.example.gov, or encounters some other PKIX validation failure, it is to reject the connection.

An MTA-STS compliant sender first checks for the presence of an MTA-STS policy at the receiver domain. First by checking its cache to see if an earlier discovered policy was found, or by looking in the DNS for the MTA-STS DNS RR. If it is a newly discovered policy, the client first gets the policy over HTTPS, then attempts to connect to each candidate MX listed in order in the policy. For each receiving mail server, the sender attempts to connect via STARTTLS, and validates the receiver’s server certificate. If successful, the message is delivered. If not, the sender moves on to the next mail server listed in the policy. If none of the connections are successful, the sender does not deliver the message.

At the time of writing, there are no publicly available MTA-STS implementations, and only a single MTA-STS Internet draft has been posted. Therefore, it is not possible for organizations to deploy MTA-STS aware clients at the present time.

Both DANE and MTA-STS were designed to assist opportunistic encryption and combat passive monitoring of SMTP connections. Receiving domains can support both if desired, to support all clients. Senders can implement both as well, as the current MTA-STS spec states that DANE DNSSEC responses take precedence. The basic merits of both are summarized in the table below:

Table 5-3: Comparing DANE and MTA-STS

Security Recommendation 5-3: Receiving domains should implement protocols to signal TLS usage to clients. Receivers should implement DANE, MTA-STS (or both) for all mail servers listed in the domain’s MX Resource Record set.

Security Recommendation 5-4: As federal agency use requires certificate chain authentication against a known CA, Certificate Usage DANE-TA(2) is recommended when deploying DANE to specify the CA that the agency has chosen to employ. Agencies should also publish a DANE-EE(3) RR alongside the DANE-TA(2) RR for increased reliability. In both cases the TLSA record should use a selector of SPKI(1) and a Matching field type of SHA2-256(1), for parameter values of “3 1 1” and “2 1 1” respectively.

Currently, there is no way for an MTA to report TLS failures to a receiving domain. If a sending MTA cannot establish a TLS protected connection, there is no automated signaling to the receiver as to the nature of the failure, only the receiver’s own logs. Previously, most MTAs would simply continue to connect without TLS and deliver the mail. However, with options such as MTA-STS (Section 5.2.5) or REQUIRETLS (Section 5.2.3), TLS errors will cause more failures in delivery.

There is a way to have a standard way to report TLS failures back to receivers known as SMTP TLS Reporting [RFC8460]. The concept is similar to DMARC (see Section 4.6) where receivers send failure reports back to senders, only here senders send the failure report. The specification includes the report format as well as how to signal reporting over SMTP or HTTPS. HTTPS is given as an option for senders that wish to use a secure channel but believe SMTP over TLS will not work. Also, like DMARC, the location (via email or HTTPS) where reports should be sent are published in a DNS TXT resource record that the sender can query for in the receiver’s domain. Here the TXT RR has a well-known string _smtp.-tls prepended and using the following tag:value pairs each separated by a semicolon ‘;’:

Table 5-4: TLS Reporting Value Tags and Descriptions

An example TLS reporting RR is given below for example.gov:

_smtp-tlsrpt.example.gov IN TXT

“v=TLSRPTv1;rua=https://reporttls.example.gov/reports”

Indicating that TLS failure reports when connecting to example.gov mail receivers should be sent to the URI listed in the rua tag. A reporting RR may have multiple values in the rua tag, indicating several alternative means to send reports.

End users and their institutions have an interest in rendering the contents of their messages completely secure against unauthorized eyes. They can take direct control over message content security using either S/MIME [RFC5751] or OpenPGP [RFC4880]. In each of these protocols, the sender signs a message with a private key, and the receiver authenticates the signature with the public key obtained (somehow) from the sender. Signing provides a guarantee of the message source, but any man in the middle can use the public key to decode and read the signed message. For proof against unwanted readers, the sender encrypts a message with the recipient’s public key or with a generated symmetric key that is encrypted with the receiver’s public key which is obtained (somehow) from the receiver. The receiver decrypts the message with the corresponding private key, or a symmetric key encrypted with the recipient’s public key, and the message content is kept confidential from mailbox to mailbox. Both S/MIME and OpenPGP are protocols that facilitate signing and encryption, but secure open distribution of public keys is still a hurdle. Two recent DANE protocols have been proposed to address this. The SMIMEA (for S/MIME certificates) and OPENPGPKEY (for OpenPGP keys) initiatives specify new DNS RR types for storing email end user key material in the DNS. S/MIME and SMIMEA are described in subsection 5.3.1, while OpenPGP and OPENPGPKEY are described in subsection 5.3.2.

S/MIME is a protocol that allows email users to authenticate messages by digitally signing with a private key and including the public key in an attached certificate. The recipient of the message performs a PKIX validation on the certificate, authenticating the message’s originator. On the encryption side, the S/MIME sender typically encrypts the message text using a generated symmetric key, which is encrypted in turn with the public key of the recipient, which was previously distributed using some other, out of band, method. Within an organization it is common to obtain a correspondent’s S/MIME certificate from an LDAP directory server. Another way to obtain a S/MIME certificate is by exchanging digitally signed messages.

S/MIME had the advantage of being based on X.509 certificates, allowing existing software and procedures developed for the PKI to be used for email. Hence, where the domain-owning enterprise has an interest in securing the message content, S/MIME is preferred.

The Secure/Multipurpose Internet Mail Extensions (S/MIME) [RFC5751] describes a protocol that will sign, encrypt or compress some, or all, of the body contents of a message. Signing is done using the sender’s private key, while key encipherment is done with the recipient’s known public key. Message encryption using the data encryption key, signing and compression can be done in any order and any combination. The operation is applied to the body, not the RFC 5322 headings of the message. In the signing case, the certificate containing the sender’s public key is also attached to the message.

The receiver uses the associated public key to authenticate the digital signature over the message, demonstrating proof of origin and non-repudiation. The usual case is for the receiver to authenticate the supplied certificate using PKIX back to the Certificate Authority. Users who want more assurance that the key supplied is bound to the sender’s domain can deploy the SMIMEA mechanism [RFC8162] in which the certificate and key can be independently retrieved from the DNS and authenticated per the DANE mechanism, similar to that described in Sub-section 5.2.5, above. The user who wants to encrypt a message retrieves the receiver’s public key: which may have been sent on a prior signed message²³. If no prior signed message is at hand, or if the user seeks more authentication than PKIX, then the key can be retrieved from the DNS in an SMIMEA record. The receiver decrypts the data encryption key using the corresponding private key, decrypts the message using the newly decrypted key and reads or stores the message as appropriate.

Figure 5-2: Sending an Encrypted Email

To send an S/MIME encrypted message (Fig 2-4) to a user, the sender must first obtain the recipient's X.509 certificate and may (optionally) validate the certificate to a trusted root, if possible. The sender then uses the certificate’s public key to generate a data encryption key and use that generated key to encrypt the composed message. In this case the sender must possess the recipient's certificate before sending the message.

An enterprise looking to use S/MIME to provide email confidentiality will need to obtain or produce credentials for each end user in the organization. An organization can generate its own root certificate and give its members a certificate generated from that root, or purchase certificates for each member from a well-known Certificate Authority (CA).

Using S/MIME for end-user encryption is further complicated by the need to distribute each end-users’ certificate to potential senders. Traditionally this is done by having correspondents exchange email messages that are digitally signed that includes the sender’s encryption certificate, but not encrypted. Alternatively, organizations can configure LDAP servers to make S/MIME public keys available as part of a directory lookup; mail clients such as Outlook and Apple Mail can be configured to query LDAP servers for public keys necessary for message encryption.

Official use requires certificate chain authentication against a known Certificate Authority.

Current MUAs use S/MIME private keys to decrypt the data encryption key that was used to encrypt the email message each time that it is displayed, but leave the message encrypted in the email store. This mode of operation is not recommended, as it forces the recipient of the encrypted email to maintain their private key indefinitely. Instead, the email should be decrypted prior to being stored in the mail store. The mail store, in turn, should be secured using an appropriate cryptographic technique (for example, disk encryption), extending protection to both encrypted and unencrypted email. If it is necessary to store mail encrypted on the mail server (for example, if the mail server is outside the control of the end-user’s organization), then the messages should be re-encrypted with a changeable session key on a message-by-message basis.

Where the DNS performs canonicalization of email addresses, a client requesting a hash encoded OPENPGPKEY or SMIMEA RR shall perform no transformation on the left part of the address offered, other than UTF-8 and lower-casing. This is an attempt to minimize the queries needed to discover an S/MIME certificate in the DNS for newly learned email addresses and allow for the initial email to be sent encrypted (if desired).

OpenPGP [RFC4880] is a proposed Internet Standard for providing authentication and confidentiality for email messages. Although similar in purpose to S/MIME, OpenPGP is distinguished by using message and key formats that are built on the “Web of Trust” model (see Section 2.4.3).

The OpenPGP standard is implemented by PGP-branded software from Symantec²⁴ and by the open source GNU Privacy Guard.²⁵ These OpenPGP programs have been widely used by activists and security professionals for many years but have never gained a widespread following among the general population owing to usability programs associated with installing the software, generating keys, obtaining the keys of correspondents, encrypting messages, and decrypting messages. Academic studies have found that even “easy-to-use” versions of the software that received good reviews in the technical media for usability were found to be not usable when tested by ordinary computer users. [WHITTEN1999]

Key distribution was an early usability problem that OpenPGP developers attempted to address. Initial efforts for secure key distribution involved key distribution parties, where all participants are known to and can authenticate each other. This method does a good job of authenticating users to each other and building up webs of trust, but it does not scale at all well, and it is not greatly useful where communicants are geographically widely separated.

To facilitate the distribution of public keys, a number of publicly available key servers have been set up and have been in operation for many years. Among the more popular of these is the pool of SKS key servers²⁶. Users can freely upload public keys on an opportunistic basis. In theory, anyone wishing to send a PGP user encrypted content can retrieve that user’s public key from the SKS server, use it to encrypt a generated data encryption key used to encrypt the message, and send it. However, there is no authentication of the identity of the key owners; an attacker can upload their own key to the key server, then intercept the email sent to the unsuspecting user.

A renewed interest in personal control over email authentication and encryption has led to further work within the IETF on key sharing, and the DANE mechanism [RFC7929] is being adopted to place a domain and user’s public key in an OPENPGPKEY record in the DNS. Unlike DANE/TLS and SMIMEA, OPENPGPKEY does not use X.509 certificates, or require full PKIX authentication as an option. Instead, full trust is placed in the DNS records as certified by DNSSEC: The domain owner publishes a public key and minimal “certificate” information. The key is available for the receiver of a signed message to authenticate, or for the sender of a message to encrypt a data encryption key.

Security Recommendation 5-5: For Federal use, OpenPGP is not preferred for message confidentiality. The use of S/MIME with a certificate signed by a known CA is preferred.

Where an institution requires signing and encryption of end-to-end email, S/MIME is preferred over OpenPGP. Like the S/MIME discussion above, if used, the email should be decrypted prior to being stored in the mail store. The mail store, in turn, should be secured using an appropriate cryptographic technique (for example, disk encryption), extending protection to both encrypted and unencrypted email. If it is necessary to store mail encrypted on the mail server (for example, if the mail server is outside the control of the end-user’s organization), then the messages should be re-encrypted with a changeable session key on a message-by-message basis. In addition, where the DNS performs canonicalization of email addresses, a client requesting a hash encoded OPENPGPKEY or SMIMEA RR shall perform no transformation on the left part of the address offered, other than UTF-8 and lower-casing.

Security Recommendation 5-1: Organizations are recommended only use approved TLS versions listed in NIST SP 800-52 with FIPS approved cryptographic modules.

Security Recommendation 5-2: TLS-capable servers should prompt clients to invoke the STARTTLS command. TLS clients should attempt to use STARTTL for SMTP, either initially, or issuing the command when offered.

Security Recommendation 5-3: Receiving domains should implement protocols to signal TLS usage to clients. Receivers should implement DANE, MTA-STS (or both) for all mail servers listed in the domain’s MX Resource Record set.

Security Recommendation 5-4: Official use of digitally signed/encrypted email requires certificate chain authentication against a known CA and using DANE-TA Certificate Usage values when deploying DANE.

Security Recommendation 5-5: Do not use OpenPGP for message confidentiality. Instead, use S/MIME with a certificate that is signed by a known CA.

Unsolicited Bulk Email (UBE) has an analogy with “beauty”, in that it is often in the eye of the beholder. To some senders, it is a low-cost marketing campaign for a valid product or service. To many receivers and administrators, it is a scourge that fills up message inboxes and can be a vector for criminal activity or malware. Both of these views can be true, as the term Unsolicited Bulk Email (or spam, as it is often called) comprises a wide variety of email received by an enterprise.

While some unsolicited email is from legitimate marketing firms and may only rise to the level of being a nuisance, it can also lead to increased resource usage in the enterprise. UBE can fill up user inbox storage, consume bandwidth in receiving email and consume end users' time as they sort through and delete unwanted email. However, some UBE may rise to the level of being a legitimate threat to the organization in the form of fraud, illegal activity, or the distribution of malware.

Depending on the organization's jurisdiction, UBE may include advertisements for goods or services that are illegal. Enterprises or organizations may wish to limit their employees' (and users') exposure to these offers. Other illegitimate UBE are fraud attempts aimed at the users of a given domain and used to obtain money or private information. Lastly, some UBE is simply a Trojan horse aimed at trying to infiltrate the enterprise to install malware.

There are a variety of techniques that an email administrator can use to reduce the amount of UBE delivered to the end users’ inboxes. Enterprises can use one or multiple technologies to provide a layered defense against UBE since no solution is completely effective against all UBE. Administrators should consider using a combination of tools for processing incoming, and outgoing email.

Figure 6-1: Inbound email "pipeline" for UBE filtering

These techniques can be performed in serial as a "pipeline" for both incoming and outgoing email [REFARCH]. Less computationally expensive checks should be done early in the pipeline to prevent wasted effort later. For example, a UBE/SMTP connection that would be caught and refused by a blacklist filter should be done before more computationally expensive content analysis is performed on an email that will ultimately be rejected or deleted. In Figure 6-1, an example pipeline for incoming email checks is given. Figure 6-2 shows an example outbound pipeline for email checks.

Figure 6-2: Outbound email "pipeline" for UBE filtering

The most basic technique to reduce UBE is to simply accept or deny messages based on some list of known bad or known trusted senders. This is often the first line of UBE defense utilized by an enterprise because, if a message was received from a known bad sender, it could reasonably be dropped without spending resources in further processing. Or, email originating from a trusted source could be marked so as not to be subject to other anti-UBE checks and inadvertently deleted or thrown out.

A non-approved sender list can be composed of individual IP addresses, IP blocks, or sending domain bases [RFC5782]. For example, it is normal for enterprises to refuse email from senders using a source address that has not be allocated, or part of a block reserved for private use (such as 192.168/16). Or an administrator could choose to not accept email from a given domain if there is no reason to assume that they have any interaction with senders using a given domain. This could be the case where an organization does not do business with certain countries and may refuse mail from senders using those country code Top Level Domains (ccTLDs).

Given the changing nature of malicious UBE, static lists are not effective. Instead, a variety of third party services produce dynamic lists of known bad UBE senders that enterprise administrators can subscribe to and use. These lists are typically accessed by DNS queries and include the non-commercial ventures such as the Spamhaus Project²⁷ and the Spam and Open Relay Blocking System (SORBS)²⁸, as well as commercial vendors such as SpamCop.²⁹ An extensive list of DNS-based blacklists can be found at http://www.dnsbl.info. Because an individual service may be unavailable, many organizations configure their mailers to use multiple blacklists. Email administrators should use these services to maintain a dynamic reject list rather than attempting to maintain a static list for a single organization.

An approved list is the opposite of a non-approved list. Instead of refusing email from a list of known bad actors, an approved list is composed of known trusted senders. It is often a list of business partners, community members, or similar trusted senders that have an existing relationship with the organization or members of the organization. This does not mean that all email sent by members on an approved list should be accepted without further checks. Email sent by an approved sender may not be subject to other anti-UBE checks but may still be checked for possible malware or malicious links. Email administrators wishing to use approved list should be very stringent about which senders make the list. Frequent reviews of the list should also occur to remove senders when the relationship ends or add new members when new relationships are formed. Some email tools allow for end users to create their own approved list, so administrators should make sure that end users do not approve a known bad sender.

A list of approved/non-approved receivers can also be constructed for outgoing email to identify possible victims of malicious UBE messages or infected hosts sending UBE as part of a botnet. That is, a host or end user sending email to a domain or setting the message-From: address domain to one listed in a non-approved receiver list. Again, since this is a relatively easy (computational) activity, it should be done before any more intensive scanning tools are used.

Techniques that use sending policy encoded in the DNS, such as Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM), and Domain-based Message Authentication and Reporting Conformance (DMARC) can also be used to reduce some UBE. Receiving MTAs use these protocols to see if a message was sent by an authorized sending MTA for the purported domain. These protocols are discussed in Section 4 and should be utilized by email administrators for both sending and receiving email.

These protocols only authenticate that an email was sent by a mail server that is considered a valid email sender by the purported domain and does not authenticated the contents of the email message. Messages that pass these checks should not automatically be assumed to not be UBE, as a malicious bulk email sender can easily set up and use their own sending infrastructure that would pass these checks. Likewise, malicious code that uses an end user's legitimate account to send email will also pass domain-based authentication checks.

Domain-based authentication checks require more processing by the receiver MTA and thus should be performed on any mail that has passed the first set of blacklist checks. These checks do not require the MTA to have the full message and can be done before any further and more computationally expensive content checks.³⁰

The third type of UBE filtering measures involves analysis of the actual contents of an email message. These filtering techniques examine the content of a mail message for words, phrases or other elements (images, web links, etc.) that indicate that the message may be UBE.

Examining the textual content of an email message is done using word/phrase filters or Bayesian filters [UBE1] to identify possible UBE. Since these techniques are not foolproof, most tools that use these techniques allow for administrators or end users to set the threshold for UBE identification or allow messages to be marked as possible UBE to prevent false positives and the deletion of valid transactional messages.

Messages that contain URLs or other non-text elements (or attachments) can also be filtered and tested for possible malware, UBE advertisements, etc. This could be done via blacklisting (blocking email containing links to known malicious sites) or by opening the links in a sandboxed browser-like component³¹ in an automated fashion to record the results. If the activity corresponds to anomalous or known malicious activity, the message will be tagged as malicious UBE and deleted before placed into the end-user's in-box.

Content filtering and URL analysis is more computationally expensive than other UBE filtering techniques since the checks are done over the message contents. This means that the checks are often done after blacklisting and domain-based authentication checks have completed. This avoids accepting and processing email from a known bad or malicious sender. End-to-end encryption of messages (e.g., S/MIME or OpenPGP) can also prevent content filtering.

Content filtering could also be applied to outgoing email to identify possible botnet infection or malicious code attempting to use systems within the enterprise to send UBE. Some content filters may include organization-specific filters or keywords to prevent the loss of private or confidential information.

The final line of defense against malicious UBE is an educated end user. An email user that is aware of the risks inherent in using email should be less likely to fall victim to fraud attempts, social engineering or convinced into clicking links containing malware. While such training may not stop all suspicious email, often times an educated end user can sometimes detect and avoid malicious UBE that passes all automated checks.

How to setup a training regime that includes end user education on the risks of UBE to the enterprise is beyond the scope of this document. There are several federal programs to help in end user IT security training, such as the “Stop. Think. Connect.”³² program from the Department of Homeland Security (DHS). Individual organizations should tailor available IT security education programs to the needs of their organization.

User education does not fit into the pipeline model in Section 6.3 above, as it takes place at the time that the end user views the email using their MUA. At this point, all of the above techniques have failed to identify the threat that now has been placed in the end user's in-box. For outgoing UBE, the threat is being sent out (possibly using the user's email account) via malicious code installed on the end user's system. User education can help to prevent users from allowing their machines to become infected with malicious code or teach them to identify and remediate the issue when it arises.

In terms of the canonical email processing architecture as described in Section 2, the client may play the role of the MUA. This section we will discuss clients and their interactions and constraints when using POP3, IMAP, and SMTP. The range of an end user’s interactions with a mailbox is usually done using one of two classes of clients: webmail clients and standalone clients. These clients communicate with the mailbox in different ways. Webmail clients use HTTPS. These are discussed in Section 7.2. Mail client applications for desktop or mobile devices may use IMAP or POP3 for receiving and SMTP for sending, and these are examined in Section 7.3. There is also the case of command-line clients, the original email clients that are still used for certain embedded system accesses. However, these represent no significant proportion of the enterprise market and will not be discussed in this document.

Many enterprises permit email access while away from the workplace or the corporate LAN. The mechanisms for this access is a Virtual Private Network (VPN) or a web interface through a browser. In the latter case, the security posture is determined at the web server. Actual communication between a client and server is conducted over HTTP or HTTPS. Federal agencies implementing a web-based solution should refer to NIST SP 800-95 [SP800-95] and adhere to other federal policies regarding web-based services. Federal agencies are required to provide a certificate that can be authenticated through PKIX to a well-known trust-anchor. An enterprise may choose to retain control of its own trusted roots. In this case, DANE can be used to configure a TLSA record and authenticate the certificate using the DNS (see Section 5.2.5).

For the purposes of this guide, a standalone client refers to a software component used by an end user to send and/or receive email. Examples of such clients include Mozilla Thunderbird and Microsoft Outlook. These components are typically found on a host computer, laptop or mobile device. These components may have many features beyond basic email processing, but these are beyond the scope of this document.

Sending requires connecting to an MSA or an MTA using SMTP. This is discussed in Section 7.3.2. Receiving is typically done via POP3 and IMAP,³³ and mailbox management differs in each case.

Email message submission occurs between a client and a server using the Simple Mail Transfer Protocol (SMTP) [RFC5321], either using port 25, 465 or 587. The client is operated by an end-user, and the server is hosted by a public or corporate mail service. Clients should authenticate using client authentication schemes such as usernames and passwords or PKI-based authentication as provided by the protocol.

It is further recommended that the connection between the client and MSA be secured using TLS [RFC5246] [RFC8314], associated with the full range of protective measures described in Section 5.2.

Email message receiving, and management occurs between a client and a server using the Internet Message Access Protocol (IMAP) protocol [RFC3501] over port 143 and 993. A client may be located anywhere on the Internet, establish a transport connection with the server, authenticate itself, and manipulate the remote mailbox with a variety of commands. Depending on the server implementation, it is feasible to have access to the same mailbox from multiple clients. IMAP has operations for creating, deleting and renaming mailboxes; checking for new messages; permanently removing messages; parsing; searching; and selective fetching of message attributes, texts and parts thereof. It is equivalent to the local control of a mailbox and its folders.

Establishing a connection with the server over TCP and authenticating to a mailbox with a username and password sent without encryption is not recommended. IMAP clients should connect to servers using TLS [RFC5246] [RFC8314], which should be associated with the full range of applicable protective measures described in Section 5.2.

Before IMAP [RFC3501] was invented, the Post Office Protocol (POP, now POP version 3 or POP3) had been created as a mechanism for remote users to connect to mailbox, download mail, and delete it off the server. It was expected at the time that access be from a single, dedicated user, with no conflicts. Provision for encrypted transport was not made.

The protocol went through an evolutionary cycle of upgrades, and the current instance, POP3 [RFC5034] is aligned with the Simple Authentication Security Layer (SASL) [RFC4422] and optionally operated over a secure encrypted transport layer, TLS [RFC5246]. POP3 defines a simpler mailbox access alternative to IMAP, without the same fine control over mailbox file structure and manipulation mechanisms. Users who access their mailboxes from multiple hosts or devices should use IMAP clients instead of POP3, to maintain a synchronization of clients with the single, central mailbox.

Clients with POP3 access should configure them to connect over TLS [RFC8314], which should be associated with the full range of protective measures described above in Section 5.2, Email Transmission Security.

Security Recommendation 7-1: IMAP and POP3 clients should connect to servers using TLS [RFC5246] and be associated with the full range of protective measures described in Section 5.2, Email Transmission Security. Connecting with unencrypted TCP and authenticating with username and password is strongly discouraged.

The security of data in transit is only useful if the security of data at rest can be assured. This means maintaining confidentiality at the sender and receiver endpoints of:

Confidentiality and the encryption for data in transit is discussed in Section 7.4.1, while the confidentiality of data at rest is discussed in Section 7.4.2.

A common element for users of TLS for SMTP, IMAP and POP3, as well as for S/MIME and OpenPGP, is the need to maintain current and accessible private keys, as used for decryption of received mail, and signing of authenticated mail. A range of different users require access to these disparate private keys:

Special care is needed to ensure that only the relevant parties have access and control over the respective keys. For federal agencies, this means compliance with all relevant policy and best practice for the protection of key material [SP800-57pt1] and message contents (e.g., FOIA, eDiscovery, etc.).

Security Consideration 7-2: Enterprises should establish a cryptographic key management system (CKMS) for keys associated with protecting email sessions with end users. For federal agencies, this means compliance with all relevant policy and best practice for the protection of key material [SP800-57pt1].

This publication is about securing email and its associated data. This is one aspect of securing data in transit. To the extent that email comes to rest in persistent storage in mailboxes and file stores, there is some overlap with NIST SP 800-111 [SP800-111].

There is an issue in the tradeoff between accessibility and confidentiality when using mailboxes as persistent storage. End users and their organizations are expected to manage their own private keys, and historical versions of these may remain available to enable the decryption of mail encrypted by communicating partners, and to authenticate cc: mail sent to partners, which have been also stored locally. Partners who sign their mail, and decrypt received mail, make their public keys available through certificates, or through DANE records (i.e., TLSA, OPENPGPKEY, SMIMEA) in the DNS. These certificates generally have a listed expiry date and are rolled over and replaced with new certificates containing new keys. For people who use their mailboxes as persistent, large-scale storage, this can create a management problem. If a mail user’s expired keys cannot be found, historical encrypted messages cannot be read.

Email keys for S/MIME and OpenPGP should only be used for messages in transit. Messages intended for persistent local storage should be decrypted, stored in user-controllable file storage, and, if necessary, re-encrypted with user-controlled keys. For maximum security, all email should be stored encrypted—for example, with a cryptographic file system.

Security Recommendation 7-3: Cryptographic keys used for encrypting data in persistent storage (e.g., in mailboxes) should be different from keys used for the transmission of email messages.

Security Recommendation 7-1: IMAP and POP3 clients should connect to servers using TLS [RFC5246] and be associated with the full range of protective measures described in Section 5.2, Email Transmission Security. Connecting with unencrypted TCP and authenticating with username and password is strongly discouraged.

Security Consideration 7-2: Enterprises should establish a cryptographic key management system (CKMS) for keys associated with protecting email sessions with end users. For federal agencies, this means compliance with all relevant policy and best practice for the protection of key material [SP800-57pt1].

Security Recommendation 7-3: Cryptographic keys used for encrypting data in persistent storage (e.g., in mailboxes) should be different from keys used for the transmission of email messages.

Selected acronyms and abbreviations used in this paper are defined below.

DHS — Department of Homeland Security

DKIM — DomainKeys Identified Mail

DMARC — Domain-based Message Authentication, Reporting and Conformance

DNS — Domain Name System

DNSSEC — Domain Name System Security Extensions

FISMA — Federal Information Security Management Act

FRN — Federal Network Resiliency

IMAP — Internet Message Access Protocol

MDA — Mail Delivery Agent

MSA — Mail Submission Agent

MTA — Mail Transport Agent

MUA — Mail User Agent

MIME — Multipurpose Internet Message Extensions

NIST SP — NIST Special Publication

PGP/OpenPGP Pretty Good Privacy

PKI — Public Key Infrastructure

POP3 — Post Office Protocol, Version 3

RR — Resource Record

S/MIME — Secure/Multipurpose Internet Mail Extensions

SMTP — Simple Mail Transport Protocol

SPF — Sender Policy Framework

TLS — Transport Layer Security

VM — Virtual Machine

VPN — Virtual Private Network

[FIPS 201] Federal Information Processing Standards Publication 201-2: Personal Identity Verification (PIV) of Federal Employees and Contractors. National Institute of Standards and Technology, Gaithersburg, Maryland, August 2013. https://doi.org/10.6028/NIST.FIPS.201-2

[SP800-45] NIST Special Publication 800-45 version 2. Guidelines on Electronic Mail Security. National Institute of Standards and Technology, Gaithersburg, Maryland, February 2007. https://doi.org/10.6028/NIST.SP.800-45ver2

[SP800-52] NIST Special Publication 800-52 Revision 1. Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations. National Institute of Standards and Technology, Gaithersburg, Maryland, April 2014. https://doi.org/10.6028/NIST.SP.800-52r1

[SP800-53] NIST Special Publication 800-53 Revision 4. Security and Privacy Controls for Federal Information Systems and Organizations. National Institute of Standards and Technology, Gaithersburg, Maryland, April 2013. https://doi.org/10.6028/NIST.SP.800-53r4

[SP800-57pt1] NIST Special Publication 800-57 Part 1 Revision 4. Recommendation for Key Management, Part 1: General. National Institute of Standards and Technology, Gaithersburg, Maryland, January 2016. https://doi.org/10.6028/NIST.SP.800-57pt1r4

[SP800-57pt3] NIST Special Publication 800-57 Part 3 Revision 1. Recommendation for Key Management, Part 3: Application-Specific Key Management Guidance. National Institute of Standards and Technology, Gaithersburg, Maryland, January 2015. https://doi.org/10.6028/NIST.SP.800-57pt3r1

[SP800-81] NIST Special Publication 800-81 Revision 2, Secure Domain Name System (DNS) Deployment Guide, National Institute of Standards and Technology, Gaithersburg, Maryland, September 2013. https://doi.org/10.6028/NIST.SP.800-81-2

[SP800-95] NIST Special Publication 800-95. Guide to Secure Web Services. National Institute of Standards and Technology, Gaithersburg, Maryland, August 2007. https://doi.org/10.6028/NIST.SP.800-95

[SP800-111] NIST Special Publication 800-111. Guide to Storage Encryption Technologies for End User Devices. National Institute of Standards and Technology, Gaithersburg, Maryland, November 2007. https://doi.org/10.6028/NIST.SP.800-111

[SP800-130] NIST Special Publication 800-130. A Framework for Designing Cryptographic Key Management Systems. National Institute of Standards and Technology, Gaithersburg, Maryland, August 2013. https://doi.org/10.6028/NIST.SP.800-130