Securing the Internet with IPsec (Internet Security Architecture)

Part 1 (page 1): The IP Security Architecture
Part 2 (page 2): How IPsec works – IPsec, IPv4 and IPv6
Part 3 (page 3): IPsec Protocols and Operations
Part 4 (page 4): Cryptographic Algorithms and Deploying IPsec

The IP Security Architecture

The IP Security Architecture, or IPsec, offers an interoperable and open standard for building security into any Internet application. By adding security at the network layer (the IP layer, or layer 3 in the OSI reference model), IPsec enables security for individual applications as well as for virtual private networks (VPNs) capable of securely carrying enterprise data across the open Internet.

IPsec and its related protocols are already being widely implemented in virtual private network products. Despite its growing importance to existing deployed systems, not too many people truly grok IPsec, probably because it is complicated (a solid couple of dozen RFCs describe IPsec and its related protocols–please refer to the list of related RFCs at the end of the article).

Saying that IPsec specifies protocols for encrypting and authenticating data sent within IP packets is an oversimplification, and even obscures IPsec’s full potential.

IPsec offers the following security services:

Altogether, IPsec provides for the integration of algorithms, protocols, and security infrastructures into an overarching security architecture.

The stated goal of the IP Security Architecture is “to provide various security services for traffic at the IP layer, in both the IPv4 and IPv6 environments.” [RFC2401]. This means security services that are: interoperable, high-quality, and cryptographically-based.

The IP security architecture allows systems to choose the required security protocols, identify the cryptographic algorithms to use with those protocols, and exchange any keys or other material or information necessary to provide security services.

Part 2: How IPsec works – IPsec, IPv4 and IPv6

How IPsec Works

IPsec uses the Authentication Header (AH) and the Encapsulating Security Payload (ESP) to apply security to IP packets. These protocols define IP header options (for IPv4) or header extensions (for IPv6). Both AH and ESP headers include a Security Parameter Index (SPI). The SPI, along with the security protocol in use (AH or ESP) and the destination IP address, combine to form the Security Association (SA).

The sending host knows what kind of security to apply to the packet by looking in a Security Policy Database (SPD). The sending host determines what policy is appropriate for the packet, depending on various selectors (for example, destination IP address or transport layer ports), by looking in the SPD. The SPD indicates what the policy is for a particular packet: either the packet requires IPsec processing of some sort, in which case it is passed to the IPsec module for processing; or it does not, in which case it is simply passed along for normal IP processing. Outbound packets must be checked against the SPD to see what kind (if any) of IPsec processing to apply. Inbound packets are checked against the SPD to see what kind of IPsec service should be present in those packets.

A second database, called the Security Association Database (SAD), includes all security parameters associated with all active SAs. When an IPsec host wants to send a packet, it checks the appropriate selectors to determine the Security Policy Database security policy for that referenced destination/port/application. If the SPD references a particular Security Association, the host can look up the SA in the Security Association Database to identify appropriate security parameters for that packet.

Key management is another important aspect of IPsec. Two important key management specifications associated with IPsec are: the Internet Security Association and Key Management Protocol (ISAKMP) and the Internet Key Exchange (IKE).

ISAKMP, a generalized protocol for establishing Security Associations and cryptographic keys within an Internet environment, defines the procedures and packet formats needed to establish, negotiate, modify, and delete Security Associations. It also defines payloads for exchanging key generation and authentication data. These formats provide a consistent framework for transferring this data, independent of how the key is generated or what types of encryption or authentication algorithms are being used.

ISAKMP was designed to provide a framework that can be used by any security protocols that use Security Associations, not just IPsec. To be useful for a particular security protocol, a Domain of Interpretation or DOI must be defined. The DOI groups related protocols for the purpose of negotiating Security Associations–security protocols that share a DOI choose protocol and cryptographic transforms from a common namespace. They also share key exchange protocol identifiers, as well as a common interpretation of payload data content.

While ISAKMP and the IPsec DOI provide a framework for authentication and key exchange, ISAKMP does not actually define how those functions are executed. The Internet Key Exchange (or IKE) protocol, working within the framework defined by ISAKMP, does define a mechanism for hosts to perform these exchanges.

By defining a separate protocol for the generalized formats required to do key and Security Association exchanges, ISAKMP can be used as a base to build specific key exchange protocols. The foundation protocol can be used for any security protocol, and does not have to be replaced if an existing key exchange protocol is replaced for some reason, such as if a security flaw was found in the protocol.

IPsec, IPv4, and IPv6

IPsec provides security services for either IPv4 or IPv6, but the way it provides those services is slightly different in each. IPv4 uses header options: every IP packet contains 20 bytes-worth of required fields, and any packet that has any “special” requirements can use up to 40 bytes for those options. This tends to complicate packet processing, since routers must check the length of each packet it receives for forwarding–even though many of those header options are related to end-to-end functions such as security, with which routers are not otherwise concerned.

IPv6 simplifies header processing: every IPv6 packet header is the same length, 40 bytes, but any options can be accommodated in extension headers that follow the IPv6 header. IPsec services are provided through these extensions.

The ordering of IPsec headers, whether within IPv4 or IPv6, has significance. For example, it makes sense to encrypt a payload with the ESP header, and then use the Authentication Header to provide data integrity on the encrypted payload. In this case, the AH header appears first, followed by the ESP header and encrypted payload. Reversing the order, by doing data integrity first and then encrypting the whole lot, means that the recipient can be sure of who originated the data, but not necessarily certain of who did the encryption.

Part 3: IPsec Protocols and Operations

IPsec Protocols and Operations

One of the fundamental constructs of IPsec is the Security Association, or SA. According to RFC 2401, a “Security Association is a simplex ‘connection’ that affords security services to the traffic carried by it.” SAs provide security services by using either AH or ESP, but not both (if a traffic stream uses both AH and ESP, it has two–or more–SAs). For typical IP traffic, there will be two SAs: one in each direction that traffic flows (one each for source and destination host).

An SA is identified by three things:

IPsec defines two modes for exchanging secured data, tunnel mode and transport mode. IPsec transport mode protects upper-layer protocols, and is used between end-nodes. This approach allows end-to-end security, because the host originating the packet is also securing it and the destination host is able to verify the security, either by decrypting the packet or certifying the authentication.

Tunnel mode IPsec protects the entire contents of the tunneled packets. The tunneled packets are accepted by a system acting as a security gateway, encapsulated inside a set of IPsec/IP headers, and forwarded to the other end of the tunnel, where the original packets are extracted (after being certified or decrypted) and then passed along to their ultimate destination.

The packets are only secured as long as they are “inside” the tunnel, although the originating and destination hosts could be sending secured packets themselves, so that the tunnel systems are encapsulating packets that have already been secured.

Transport mode is good for any two individual hosts that want to communicate securely; tunnel mode is the foundation of the Virtual Private Network or VPN. Tunnel mode is also required any time a security gateway (a device offering IPsec services to other systems) is involved at either end of an IPsec transmission. Two security gateways must always communicate by tunneling IP packets inside IPsec packets; the same goes for an individual host communicating with a security gateway. This occurs any time a mobile laptop user logs into a corporate VPN from the road, for example.

The Authentication Header (AH) protocol offers connectionless integrity and data origin authentication for IP datagrams, and can optionally provide protection against replays.

The Encapsulating Security Payload (ESP) protocol provides a mix of security services:

ESP and AH authentication services are slightly different: ESP authentication services are ordinarily provided only on the packet payload, while AH authenticates almost the entire packet including headers.

AH is specified in RFC 2402, and the header is shown in the figure below (taken from RFC 2402).

Figure 1

The Sequence Number field is mandatory for all AH and ESP headers, and is used to provide anti-replay services. Every time a new packet is sent, the Sequence Number is increased by one (the first packet sent with a given SA will have a Sequence Number of 1). When the receiving host elects to use the anti-replay service for a particular SA, the host checks the Sequence Number: if it receives a packet with a Sequence Number value that it has already received, that packet is discarded.

The authentication data field contains whatever data is required by the authentication mechanisms specified for that particular SA to authenticate the packet. This value is called an Integrity Check Value or (ICV), it may contain a keyed Message Authentication Code (MAC) based on a symmetric encryption algorithm (such as CAST or Triple-DES) or a one-way hash function such as MD5 or SHA-1.

ESP is specified in RFC 2406, and while similar to AH in many ways it provides a wider selection of security services and can be a bit more complex.

The ESP header format is shown in the figure below (taken from RFC 2406).

Figure 2

The most obvious difference between ESP and AH is that the ESP header’s Next Header field appears at the end of the security payload. Of course, since the header may be encapsulating an encrypted payload, you don’t need to know what header to expect next until after you’ve decrypted the payload. Thus, the ESP Next Header field is placed after, rather than before, the payload. ESP’s authentication service covers only the payload itself, not the IP headers of its own packet as with the Authentication Header. And the confidentiality service covers only the payload itself; obviously, you can’t encrypt the IP headers of the packet intended to deliver the payload and still expect any intermediate routers to be able to process the packet. Of course, if you’re using tunneling, you can encrypt everything, but only everything in the tunneled packet itself.

Part 4: Cryptographic Algorithms and Deploying IPsec

Cryptographic Algorithms

Although there is no IPsec without encryption and authentication algorithms, which algorithms you use do not matter all that much–as long as the ones you use are secure. The fact is, IPsec was designed to allow entities to negotiate the appropriate security mechanisms from whatever algorithms each supports, using ISAKMP-based key and SA management protocols.

There is currently some controversy over which algorithms should be used in IPsec, and which should be considered basic parts of any IPsec implementation. The Data Encryption Standard, or DES, has recently proven to be vulnerable to relatively inexpensive brute-force attacks; there is a significant movement to have it deprecated for use in IPsec. At the same time, the US National Institute of Standards and Technology (NIST) is in the process of selecting DES’s successor algorithm, the Advanced Encryption Standard or AES.

Implementing and Deploying IPsec

The IPsec specification (found in RFC 2401) states there are several ways to implement IPsec in a host or in conjunction with a router or firewall:

Most organizations are likely to buy rather than build their IPsec implementation. VPN vendors usually claim to support IPsec, though some are more interoperable than others. Resources for checking interoperability include:

IPsec continues to evolve as research reveals new tools for security and new threats to security. To stay on top of the latest IETF standards developments, check:

.

There is no longer any question about whether or not the Internet will be important to your business; it already is. IPsec provides a framework within which you can use the Internet as your own, secure, virtual private network.

IPsec and related RFCs

• RFC 1320 The MD4 Message-Digest Algorithm
• RFC 1321 The MD5 Message-Digest Algorithm
• RFC 1828 IP Authentication using Keyed MD5
• RFC 1829 The ESP DES-CBC Transform
• RFC 2040 The RC5, RC5-CBC, RC5-CBC-Pad, and RC5-CTS Algorithms
• RFC 2085 HMAC-MD5 IP Authentication with Replay Prevention
• RFC 2104 HMAC: Keyed-Hashing for Message Authentication
• RFC 2144 The CAST-128 Encryption Algorithm
• RFC 2202 Test Cases for HMAC-MD5 and HMAC-SHA-1
• RFC 2268 A Description of the RC2(r) Encryption Algorithm
• RFC 2401 Security Architecture for the Internet Protocol
• RFC 2402 IP Authentication Header
• RFC 2403 The Use of HMAC-MD5-96 within ESP and AH
• RFC 2404 The Use of HMAC-SHA-1-96 within ESP and AH
• RFC 2405 The ESP DES-CBC Cipher Algorithm With Explicit IV
• RFC 2406 IP Encapsulating Security Payload (ESP)
• RFC 2407 The Internet IP Security Domain of Interpretation for ISAKMP
• RFC 2408 Internet Security Association and Key Management Protocol (ISAKMP)
• RFC 2409 The Internet Key Exchange (IKE)
• RFC 2410 The NULL Encryption Algorithm and Its Use With IPsec
• RFC 2411 IP Security Document Roadmap
• RFC 2412 The OAKLEY Key Determination Protocol
• RFC 2451 The ESP CBC-Mode Cipher Algorithms
• RFC 2631 Diffie-Hellman Key Agreement Method

Pete Loshin has written a dozen books on networking and the Internet, and is editor of the soon-to-be released “Big Book of IPsec RFCs: Internet Security Architecture” (Morgan Kaufmann 1999). Other books include “TCP/IP Clearly Explained” 3rd edition (Morgan Kaufmann 1999) and “Extranet Design and Implementation” (SYBEX 1998). You can reach him at pete@loshin.com or http://www.loshin.com.