Last time we talked about securing your virtual private networking connection with PPTP (point-to-point tunnelling protocol), which is relatively simple and quick to set up. Some security experts consider Microsoft's implementation of PPTP flawed, however, and thus compromisable in the hands of a cracker with time and patience.
For those who require more secure network connections, Microsoft offers the IETF (Internet Engineering Task Force) IP security standard - IPsec from now on - built into Windows XP Professional.
IPsec implements security at the packet level; that is, it protects the integrity of each data packet through a variety of means, including stringent authentication checks and encryption, making it ideal for safe communications over untrusted networks. It offers the ability to secure all traffic between some or all hosts, or traffic flowing between specific ports and/or using certain protocols. It can be used to create encrypted tunnels between networks, or simply to encrypt traffic between two endpoints or hosts.
However, this level of protection comes at a price: IPsec is a complex beast to understand and to set up. Get it wrong, and you lock down your computer too tightly, leaving it unable to communicate with the rest of the world.
The finer details of IPsec are beyond the scope of this brief article, so we'll stick to a practical example: how to set up an IPsec VPN connection using Layer 2 Tunnelling Protocol (L2TP).
Just like PPTP, L2TP provides an encrypted tunnel for VPN traffic; however, L2TP encrypts the traffic before the Point-To-Point (PPP) connection negotiation begins, making it much harder to conduct dictionary attacks on captured PPP packets.
L2TP with IPsec can be set up to use no encryption, providing Authentication Header (AH) only, which handles the authentication, integrity checking of packets, and checks the sequence numbers of these to ensure that the data hasn't been intercepted and discards packets that might have been added by hackers.
Encryption of the packet contents is provided by ESP (Encapsulation Security Pay-load); you can use AH and ESP on their own, or together. ESP takes more computing effort than just AH, so in performance-sensitive environments Microsoft's L2TP/IPsec implementation uses either a single or triple 56-bit DES keys for encryption in blocks of 64 bits each; the latter key length is considered very secure for the time being.
Caveats and certificates
Due to the packet integrity checking, it can be hard to make IPsec work in a Network Address Translation environment - naturally enough, packets with rewritten IP addresses in the headers are flagged as having been tampered with. Microsoft also says the Internet Key Exchange (IKE) necessary to set up Security Associations (SAs) for L2TP and PPP won't work with NAT and Windows XP; SAs are defined as the combination of negotiated key, security protocol and security parameters index. The latter is a unique value used to tell the receiving host which SA to use to process incoming packets.
However, IKE works with NAT using the downloadable VPN Client for Windows 98/NT, and Microsoft says it's planning to introduce the feature, called NAT Traversal, into XP soon.
Your router and firewall must be able to pass traffic on UDP port 500 (for IKE) and Internet Protocols number 50 (ESP) and 51 (AH) for IPsec to work. While UDP isn't a problem, not all routers understand IPsec protocols, so check this first.
Encryption and integrity checking coupled with tunnelling add overhead, so expect to give up 10 to 20 per cent of network performance in exchange for securing your connection.
Finally, you will need certificates at both ends of the connection. These can be obtained from a commercial issuer, or you can use the stand-alone Windows 2000 Server Certificate Authority to create them. Certificates can also be created using the Open Source OpenSSL tools, if you have access to these.
Make sure that both the RAS server and workstation connecting to it have a certificate issued.
And you're ready to connect!
Once you have sorted out certificates, the router/firewall, and tested that you can connect to the RAS server with plain old PPTP, you are ready to use the L2TP/IPsec VPN. The existing PPTP connection you set up earlier defaults to L2TP/IPsec if it's offered by the server, so there's no need to create a new one. You don't even need to create IPsec policies - Windows XP does it automatically for your VPN connection.