hopefully the last page ever on..

Configuring an IPSec Tunnel between Linux and Microsoft Windows 2000

rev. by: Fred Morris
rev. date: 21-Aug-2004
acknowledgements: Joe Chesak


Abstract

Other Documents

Nate Carlson's Document

Microsoft's Knowledge Base Article 252735

Architecture of the Test Network

Certificates

Linux (hera)

Copying certificates

Configuring the tunnels

Microsoft Windows 2000 (athena)

Copying certificates

Configuring the tunnels


Abstract

Fills in a couple of gaps in available documentation on setting up an IPSec VPN tunnel between Linux using FreeSWAN with the X.509 patch and Microsoft Windows 2000 using the IP Security Policy snap-in.

The tunnel described is host-to-host. Those with an appropriate understanding of how IPSec works should have no difficulties altering this to create a tunnel or tunnels to subnets served by the hosts which are the VPN tunnel endpoints.

Although this document specifically references Microsoft Windows 2000, the IP Security Policy snap-in exists in Microsoft Windows Server 2003 and Microsoft Windows XP.

Other Documents

You should read the following two documents before attempting to set up your tunnel.

Nate Carlson's Document

This document comes in two flavors, the current document which references OpenSWAN and the original document which references FreeSWAN.

This document will help you create a Certificate of Authority (CA) and the appropriate certificates on Linux, and provides instructions on installing the CA and certificates on your Microsoft Windows box.

It also provides a good starting point for setting up the Linux end of the tunnel. You should read the current document first, and then look at the original and see how they differ.

Microsoft's Knowledge Base Article 252735

Microsoft's Knowledge Base article 252735 (keywords: kb3rdparty kbhowto kbnetwork kb252735) describes how to use the IP Security Policy snap-in to create a tunnel endpoint.

It is specifically targeted at Windows 2000, but the snap-in exists on Microsoft Windows XP as well.

Architecture of the Test Network

The test network consists of two boxes:

host name hera.m3047.inwa.net athena.m3047.inwa.net
ip address 10.0.0.21 10.0.0.20
operating system SuSE Linux 8.2 Professional Microsoft Windows Server 2000 with Service Pack 4

SuSE includes the X.509 patch in the version of FreeSWAN which they ship. The notes on Nate Carlson's page indicate that Microsoft Windows 2000 requires Service Pack 2 or later.

Certificates

Nate Carlson's instructions on setting up certificates are bang-on. Follow his instructions.

Linux (hera)

Copying certificates

For the most part, Mr. Carlson's instructions are o.k. I recommend (and you will see below) that when you copy the public and private keys to /etc/ipsec.d that you don't distinguish in your naming convention between the public and private keys ( /etc/ipsec.d/*pem and /etc/ipsec.d/private/*key respectively) but that you just use the commonName, which should be the hostname (in the documents below I unfortunately used hostname.key); the location of the file will determine whether it is public or private, and you won't have to worry about whether you're referring to the public or private key this way.

Note: Both kinds of files are basically text, and you can tell them apart by examining the contents because they're different formats.

Another thing which will make life easier is if you use the same passphrase for the certificates as for the CA... security implications aside.

Configuring the tunnels

There are actually two symmetric tunnels, one going in each direction (but if you know enough about IPSec to be doing this, you knew that already).

Make sure you have IP Forwarding enabled in your kernel or network configuration...

The following directory listing shows the locations of the relevant configuration files under /etc:

 

-rw-r--r--    1 root     root         1135 Aug 15 21:48 ipsec.conf
-rw-------    1 root     root         3850 Aug 15 22:00 ipsec.secrets

ipsec.d:
total 28
drwxr-xr-x    5 root     root         4096 Aug 15 22:03 .
drwxr-xr-x   56 root     root         4096 Aug 17 19:53 ..
-rw-r--r--    1 root     root         3880 Aug 15 21:56 athena.m3047.inwa.net.key
drwxr-x---    2 root     root         4096 Aug 15 15:45 cacerts
drwxr-x---    2 root     root         4096 Mar 17  2003 crls
-rw-r--r--    1 root     root         3874 Aug 15 15:43 hera.m3047.inwa.net.key
drwx------    2 root     root         4096 Aug 15 21:55 private

ipsec.d/cacerts:
total 12
drwxr-x---    2 root     root         4096 Aug 15 15:45 .
drwxr-xr-x    5 root     root         4096 Aug 15 22:03 ..
-rwx------    1 root     root         1411 Aug 15 15:45 cacert.pem

ipsec.d/crls:
total 8
drwxr-x---    2 root     root         4096 Mar 17  2003 .
drwxr-xr-x    5 root     root         4096 Aug 15 22:03 ..

ipsec.d/private:
total 16
drwx------    2 root     root         4096 Aug 15 21:55 .
drwxr-xr-x    5 root     root         4096 Aug 15 22:03 ..
-rw-r--r--    1 root     root         1757 Aug 15 21:55 athena.m3047.inwa.net.key
-rw-r--r--    1 root     root         1753 Aug 15 15:38 hera.m3047.inwa.net.key

Note that I don't even have a CRL.... oh well!

Remember that this machine is hera.m3047.inwa.net as you examine the following two files. It's important to create an entry in ipsec.secrets with the passphrase for this machine's certificate:

# This file holds shared secrets or RSA private keys for inter-Pluto
# authentication.  See ipsec_pluto(8) manpage, and HTML documentation.
#
: RSA hera.m3047.inwa.net.key "you don't think this is my passphrase do you?"
#
# RSA private key for this host, authenticating it to any other host
# which knows the public part.  Suitable public keys, for ipsec.conf, DNS,
# or configuration of other implementations, can be extracted conveniently
# with "ipsec showhostkey".
: RSA	{
    # RSA 2048 bits   hera   Sat Aug 14 19:34:44 2004
    # for signatures only, UNSAFE FOR ENCRYPTION
    #pubkey=0sxxxx
    #IN KEY 0x4200 4 1 xxxx
    # (0x4200 = auth-only host-level, 4 = IPSec, 1 = RSA)
    Modulus: 0xxxxx
    PublicExponent: 0x03
    # everything after this point is secret
    PrivateExponent: 0xxxxx
    Prime1: 0xxxxx
    Prime2: 0xxxxx
    Exponent1: 0xxxxx
    Exponent2: 0xxxxx
    Coefficient: 0xxxxx
  }
# do not change the indenting of that "}"

Having done this, you can create the definition for the tunnels themselves in ipsec.conf. Note the left/rightrsasigkey and left/rightcert entries which are being used in ways which you will not find in the man page for ipsec.conf! These are X.509 extensions. You can find some additional informations about them if you install the source package for FreeSWAN from the SuSE distribution. Anyway, they will just work... just be sure to reference the correct certificates for your VPN endpoint hosts.

# /etc/ipsec.conf - FreeS/WAN IPsec configuration file
# More elaborate and more varied sample configurations can be found
# in FreeS/WAN's doc/examples file, and in the HTML documentation.

# basic configuration
config setup
    # THIS SETTING MUST BE CORRECT or almost nothing will work;
    # %defaultroute is okay for most simple cases.
    interfaces=%defaultroute
    # Debug-logging controls:  "none" for (almost) none, "all" for lots.
    klipsdebug=none
    plutodebug=all
    # Use auto= parameters in conn descriptions to control startup actions.
    plutoload=%search
    plutostart=%search
    # Close down old connection when new one using same ID shows up.
    uniqueids=yes
    # Enable NAT-Traversal
    #nat_traversal=yes

# defaults for subsequent connection descriptions
# (these defaults will soon go away)
conn %default
    keyingtries=0
    disablearrivalcheck=no
    authby=rsasig
conn hera-to-athena
    type=tunnel
    authby=rsasig
    left=10.0.0.20
    leftnexthop=%direct
    right=10.0.0.21
    rightnexthop=%direct
    leftrsasigkey=%cert
    rightrsasigkey=%cert
    leftcert=athena.m3047.inwa.net.key
    rightcert=hera.m3047.inwa.net.key
    keyingtries=0
    auto=start
    pfs=yes

Once you're done configuring the Microsoft Windows side of the tunnel, you can come back and invoke /etc/rc.d/ipsec start to start up IPSec. It will create an ipsecn device if all goes well, which you will see if you use the command /sbin/ifconfig. If you configure it as above with plutodebug=all, useful information will be written to /var/log/warn. The other extremely useful command is ipsec auto --status.

Microsoft Windows 2000 (athena)

Note: You don't need to download the ebootis VPN Tool, everything you need is already on the box.. you just have to find it!

Copying certificates

Again, follow Mr. Carlson's instructions for creating the certificates and .p12 file.

To get the Management Console snap-in so that you can anoint yourself as a Certificate Authority and install the certificate for your machine, just go to DOSHell and type MMC. Now follow the rest of his instructions.

Configuring the tunnels

The thing that's really missing from Microsoft's Knowledge Base 252735 novella is a checklist. At the conclusion of the process, you should have the following configured items:

IP Security Policy

VPN

an IP Security Policy with rules:

  • hera to athena
  • athena to hera
  • <Dynamic> Default Response disabled
IP Security Rules

hera to athena

  • filter list: hera to athena
  • filter action: FreeSWAN
  • authentication methods: Certificate Authority
  • tunnel setting: endpoint is 10.0.0.20
  • connection type: all network connections

athena to hera

  • filter list: athena to hera
  • filter action: FreeSWAN
  • authentication methods: Certificate Authority
  • tunnel setting: endpoint is 10.0.0.21
  • connection type: all network connections
Filter Lists (1 filter in each)

hera to athena

  • mirror: no
  • protocol: any
  • source port: any
  • source address/mask: 10.0.0.21/255.255.255.255
  • destination address/mask: 10.0.0.20/255.255.255.255

athena to hera

  • mirror: no
  • protocol: any
  • source port: any
  • source address/mask: 10.0.0.20/255.255.255.255
  • destination address/mask: 10.0.0.21/255.255.255.255
Filter Action

FreeSWAN

  • negotiate security
  • one item in list:
    • type: custom
    • AH integrity: none
    • ESP confidentiality: 3DES
    • ESP integrity: SHA1
    • key lifetime: 0/0
  • accept unsecured communication: disabled
  • allow unsecured communication: disabled
  • Perfect Forward Secrecy: enabled

Make the policy your default policy, and away you go! Once you have it configured and enabled, typing the DOSHell command IPSECMON should bring up a utility which will show you your tunnel and its activity. There isn't much else in the way of debugging assistance on Windows.


Fred Morris Consulting, Licensed in Seattle, WA, USA. since 1984

Document/Collaboration/Content Management Tools and Solutions

Better, Cheaper, Highly Adaptable, Less Hassles

Custom and Extraordinary Needs Data Processing Services

What else is on this web site?

An Internet Plumber... not a web cowboy

telephone: 206.297.6344
e-mail: x0xm3047x0xatx0xinwa.net