What is VPN IPsec and how they work together

Why use VPN Connections?

The Virtual Private Network,

• Cheaper Connections over public network
• Available anywhere the internet is available
• Heavily encrypted and secured – very secure and very difficult to break. VPN take lot more overhead on router for connecting of privet line, case it lot more secure and also encrypt anything. Like more processor utilization   
•  Many-to-Many connections – support many to many connections

The different styles of VPN connections: Cisco VPN styles

Site-to-Site (L2L) – connection between different location, provide a privet encrypt, secure connection over public network/internet.

Remote Access – Remote access allow to connect user by username/password, dialup etc. the remote access client usually are installed on PC/Laptop (Cisco vpn client, get from cisco site / in windows has built-in vpn client). (token, user/password, biometric method – like finger print, face recognize, retina scan etc ). Now have some small wireless routers that support VPN, which can be installed on client site (that also support Voice phone through VPN).

Other two new important technique to connect VPN for Remote Access, those are SSL VPN, WEB VPN. Those are same. Here the VPN supported router enables this feature and also provides like a web page for client to connect by using username/password or Token code (For Many users (50, 100) for Remote Access VPN support). Example – someone want to access remote network through VPN enable router that display a web page and asking the username/password or token. After verify user the router installs a mini VPN client tool to PC to established VPN connection. if you close the page, the VPN connection was gone.

What devices can use for VPN and Software :

What is IPSec?

The IPSec (Internet Protocol Security) is a protocol suite for securing IP (Internet Protocol) communications by Authenticating, Encrypting and Protecting each one IP packet of a communication session.  (http://en.wikipedia.org/wiki/IPsec)

IPSec Modes of Communication: two type of IPsec Mode are given below.

Transport Mode: Here IPSce is enable LAN environment, everything is encrypted in LAN.

The transport mode in IPsec will encrypts everything from the transport layer and above. So port number, TCP, UDP information at transport layer and all the application layer also encrypted by IPSec (From Diagram DATA, ESP). The IPsec enable VPN style security on LAN, if somebody can grub (wire shark) the data, he don’t do anything cause is encrypted. Because most of the time attacks are initiate from inside of the network.

Tunnel Mode: The tunnel mode to securely connect local network through internet.

Here both remote internal networks connected through VPN over Public Network.

The tunnel Mode in IPSec will encrypt everything from Network layer and above.

Example - (see diagram) Here (Router R1) encrypt everything DATA, IP (Private IP) and ESP, and add new IP header (Public IP) to send other site and when Router R2 receive the packet than decrypt everything (PATA, IP (Private), ESP) and also encrypt IP header (Public IP), MAC for this internal network (LAN2). Same thing happen when Router R2 send something to Router R1.

The pieces that build IPSec:

Now get little bit technical and into the protocols that give the power to do that. The VPN technology work with the protocol that known as IPsec. IPsec is the protocol that makes VPN possible. It is the security protocol that has all of heavy encryption protocol. The IPsec work with TCP/IP protocol (TCP/IP is protocol of communication over network) and IPsec is one of those protocol that actually work at Transport Layer, like TCP or UDP you have choose, so the VPN always choose IPsec to communicating over TCP/IP network.

IPSec is not just one protocol, it combination of many things. IPsec is taking care about Authentication, Data Integrity, and Confidentiality and Anti replay.  Every piece in IPSes be changeable, something new may come. It builds of four major categories of protocols:

Negotiation Protocol: This is the engine of IPSec. The negotiation protocol defines how VPN connection was built between two routers and how to implement (It tied to other method of IPsec) all of IPsec pieces like Authentications (MD5, SHA), Encryptions (DES, 3DES, AES) and Protections (also Encryption) (DH (1,2,5,7), RSA) with VPN. There some type of Negotiation Protocols, those are given bellow.

AH(Authentication Header) – AH is original engine that come out with IPsec. The problem is AH could not do Encryption. But support Authentication and Protection (Data integrity)

ESP (Encapsulating Security Payload) – ESP also the engine of IPsec, that allow Encryption (DES, 3DES and AES), Authentication (MD5, SHA), Protection (Data Integrity) (DH1,2,5,7 and RSA)

ESP₊AH – More power full than ESP also more overhead on devices.

Encryption Protocol: You can choose any one from the list for encryption to secure data

DES – (Symmetric encryption Algorithms) (Created by IBM, 56bit key (also can see 64bit key but only work 56bit key encryption )) One of the first encryption and also weak encryption protocol. Handle many connections with less processing (Note – 12 year old girl in swiden breaking the DES encryption formula)

3DES – (Symmetric encryption Algorithms) (56*3=168bit keys) is very secure and difficult to break. Uses three DES key on each block of data to create 168 bit key. Here have 3 key, so data encrypt by first key than second key encrypt the first one  and than third key encrypt the second key and send to other. That usage may old router (Netgear FVS318, they use 3DES by default) to create VPN connection. 3DES have 168 bit but not secure then AES.

AES (Advanced Encryption Standard) – (Symmetric encryption Algorithms) (USA Government approve standard, 128 bit, 192 bit, 256 bit keys) Newer, More Efficient algorithm and also Most powerful encryption protocol, provide more secure encryption and also more overhead. Its Symmetric and also more process utilized.

Authentication Protocol (Data Integrity): Its an hashing Algorithms that taking care about “Data Integrity”. Making sure data does not change from source to destination. (Prevent Man-in-the-middle(MIM) attack)

MD5 – (128 bit hash), is secure hashing, but MD5 is proven as unsecure. That not broken yet but may be possibility of broken some way.

SHA-1 – (160 bit hash), most secure hashing algorithm and also not breakable

Protection Protocol (Encryption): that allows you to do all of this over public network. Every source and destination has same encryption key to travel data between them. The encryption formula allows scrambling data before send it, and the destination also has same key and able to unencrypt that data. It is possible, an hacker (MIM ) can grub the key and also change data by having some encryption and decryption process.

So (they are very similar, Secure and also unbreakable) the Diffie-Hellman and RSA protection protocol prevents this type of attack.  

DH (Diffie-Hellman) – (Asymmetric encryption Algorithms) (Created formula at 1978 and came up 1997, DH1, DH2, DH5, DH7 (768bit, 1024bit, 1536bit or larger)). Generally used for VPN Connections to allow secure transfer of “Shared secret” keys and also helps to generate “Shared Secret” Keys. (See Details on Asymmetric sections)

RSA (Rivest – Shamir - Adleman)-  (Asymmetric encryption Algorithms) (Came up 2000 (512bit, 768bit, 1024bit or larger)). Use for “Miscellanies” encryption (SSH, Secure server, HTTPs, VPN on Cisco devices). Is less process consumed than DH. Like generate 512 bit key on both DH and RSA, here RSA is less process consumed.

How Security over a public Network using encryption:

how is it all work, how is it possible to get true security over public network when send those key to encryption each other and also decrypt.

Two type of encryption keys algorithm:

Symmetric encryption: “Each peer uses the same key to encrypt and decrypt data (Shared Secret)”. Symmetric is uses the same key (generate by router) to encrypt and decrypt data. That uses as known as “Shared Secret” Key (DES, 3DES, AES). Symmetric encryption is really first and less overhead. DES, 3DES, AES are form symmetric encryption cause they use same key to encrypt and decrypt. Example – Router R1 encrypt data using key and send, router R2 receive and uses same key to decrypt data (see diagram).

This is possibility of grub the key by attacker (MIM), so the Diffie-Hellman (DH1, HD2, DH5, DH7) Prevent this attack, that usages in Asymmetric Encryption process.

Asymmetric Encryption: “A peer uses one key to encrypt and another key to decrypt (Public/Private)”. Asymmetric usages Diffie-Hellman (DH) protection. Asymmetric uses two type to key, Public and Private (DH Privet and DH Public).  Here, Anything that it encrypt with public key that can be decrypted with privet and also anything encrypt with privet key that can be decrypted with public key. They do both but they completely opposite of each other. Here, Private Key never shares to each other, its use locally and the public key is use to encrypt “Shared Secret” key. The Diffie-Hellman is so sophisticated, secure, impossible to reverse/break the encryption.

Example – Like Site-to-Site VPN, when someone initiates VPN connection, firstly happen is the router receives the connection send a key.

R2 router want connect to R1 router, the R1 generate and send an “DH Public” key to R2 Router (The public key is plain text) and also Router R2 generate and send his “DH Public” Key to Router R1. Now Router R1 generate “Shared Secret” key and encrypt using R2 Public key and send the encrypt data to Router R2 over Internet. Also the Router R2 generate “Shared Secret” key and encrypt using R1 Public key and sent to Router R1 over internet. After exchange “Shared Secret” key, only the router decrypt those key by using the Private Key of the router (DH Private). The (DH Private) key never ever shares to other, cause it local. So both sites can use the “Share Secret” key for all future communication and also allowing secure communication over internet.

Once the VPN connection is done, the “Shared Secret” key was gone and next time the VPN connection happen than new “Shared Secret” key going to be generated.

The encryptions (Shared Secret) that are being use to encrypt all the data over VPN that are constantly changing, means always being regenerate and renew. At Site-to-Site VPN that always be connected, so after certain amount of time a new (Shared Secret) key will be regenerated and encrypt by using Diffie-Hellman or RSA key to securely exchange data over VPN.  

The VPN session has a lifetime either amount of Second (86400) or in Kilobyte (4Mb). When the session reach the amount of time and also send about 4Mb data over VPN the router flash the old “Shared Secret” key and generate new one.  

Why need “Shared Secret” Key in Asymmetric (DH)?

The Asymmetric are more overhead and more process consume, hundred time (if have dedicated hardware board inside the router) and it will be thousand times (if do not have dedicated hardware board inside the router) then symmetric and Symmetric is less overhead and Faster. Asymmetric key size is 1024 bit (Most Secure) and Symmetric key size 128 bit (Less Secure). Only thing, you can use those “DH” and other one “RSA” (RSA another standard of Asymmetric encryption) standard to encrypt “Shared Secret” key (the router now in less process). So both work together and provide more secure and faster VPN connection over Internet.

How devices authenticate to an IPSec VPN (Digital Signatures and PKT, The method for authenticating across VPN):

There are many way to authenticate VPN, those are describe below –

Using Username/Password, Biometric: (Remote Access)

 VPN allow you to authenticate using username/password or Biometric method. Means you have laptop so you can use like finger print, face recognize, retina scan etc to authenticate

One time password: (Remote Access)

You can see in credit card process, you may use the password one time to authenticate VPN and hit the button of credit card that say your password valid for like 1 or 2 minute after the time expired, you never use this password again cause your password no longer valid.

The above methods (Username/password, “Biometric” or One time Password) are for Remote Access VPN and In Site-to-Site VPN authentication have two primary methods (Pre-Shared key, Certificate). Those are describe below

Pre-Shared Key: (Site-to-Site)

The pre-shared key is just like a password. So same Pre-Shared key configured on both site to authenticate each other. It’s plain security and pretty good security.  In addition feature of Pre-shared Key also have the VPN is being tied with those physical address (Public IP).

Example: Here router (R1) wants to start VPN connection and send a packet with pre-shared key “cisco123” to Route (R2). Now Router (R2) check out own pre-shared key, it’s “cisco123”, so Router (R2) allowed the VPN connection with Router (R1).

In the Remote access VPN people are connected to route from anywhere by using username/password, Biometric, Dialup. The problem is that anyone could start attack (Using dictionary attack, trying with different keys) to trying getting access in VPN. So we have to configure trusted physical address (Public IP) list (Access-list) who is allow or not (No body on internet has the same Public IP address). So my key is “cisco123” but I only allow those who have in my trusted list. Here have another problem, what about “IP Spoofing”, people aren’t able to “IP spoofing” in IPsec VPN connection. Cause here to many layer of security happen in IPsec VPN.

The problem of Pre-Shared key on Scalability:

In the diagram we have 5 routers and configure Site-to-Site VPN with different Pre-Shared Key like R1-R2 (cisco1), R1-R5 (cisco2), R1-R4 (cisco3), R1-R3 (cisco4) so on. The Pre-Shared key is use to generate encryption key for VPN, if the same key keep for long time that would be possibility of broken the key by attacker. So the good practice is that those key need to be change periodically basic, once every month, once every six month or once every year.

This is a small network. In large network there has like 50 or 100 or more routers that will be very difficult to change Pre-Shared key to periodically basic. So here is the better solution for the system. Its call “Certificates”  

Certificates: (Site-to-Site) “what is Certificate”

In the system that has centralize Certificate Authority (CA) who provide certificate to authentication IPsec VPN connection. Everybody trusts to Certificate Authority. The Certificate authority issue and provides Certificate to everyone to established VPN between them.

Example: Router (R1) would like to stat VPN with you, this Route (R2) says do you have Pre-Shared key? Router (R1) answered to Router (R2) no and also I have something better that is “Certificate” that get from Certificate Authority (CA). So router (R2) trusted R1 and stat VPN. (How they doing that, Having they create two way trusted relationship between every single one of the router with pre-shared key.)

How Certificate works? Why the certificate most trusted?

 If Router has a Certificate that says other router about the certificate so other router trusted the certificate and allows access to the whole network.

Every single Router in network has the own certificate that are given by CA. Each Certificate has three (3) parts:

1. Public Key – Public key of the Router (R1). This is Asymmetric Encryption public key “DH Public” 1024-bit (In IPsec).

2. Device Signature – is the name of the Router (R1). The name is combined of “hostname+domain name” (R1.tramsit.com) fully qualified domain name, it’s (Router R1) going to sign this certificate with his name. You know in Asymmetric encryption each router has Private Key (DH Private Key) is encrypted with the Device name (R1.tramsit.com) (the private key not often using for encryption but in this case it happen) and create a Signature. What that does, it’s allow the router receiving Public key (its other router public key that encrypted with “Session key or Shared Secret” key) to using it (Public Key) to decrypt the signature, it’s look like the name of R1.tramsit.com. (Remember that, (Asymmetric Encryption) Anything that it encrypt with public key that can be decrypted with privet and also anything encrypt with privet key that can be decrypted with public key. They do both but they completely opposite of each other.)

3. CA Signature - The certificate Authority who gave you the Certificate (Puts stamp of approval) (it’s may windows 2003, Linux or having special CA Box). The CA is validate this certificate and the public key (its R1 Public key) and the Device Signature (Decrypt by CA, ok its R2.tramsit.com) of the router (R1). So any other router should trust the certificate of R2 cause it validates by CA.

The CA signature is the name of the CA (ca.cas.com). The name is encrypted with CA private key (that is not mean the private key of CA is sitting on the Certificate, nobody has that (Private Key) except the CA). Here everybody who is in the CA trusted list (all the member/route of VPN) they have the “Public key” of CA, that’s part of CA process.

So, everybody got the CA “Public Key” because they trust CA. the part of that trust relationship is to get public key of CA. when they get the certificate they decrypt (they decrypt using the CA “Public key”) the stamp and see the name of CA (ca.cas.com) that they all trust. No one else could sign it that and also encrypt that with “Private Key” of CA except CA.

One another thing if your CA is compromised (The CA “Public Key” and “Private Key” compromise by attacker), this is the solution is to change the CA.

Certificate Standards: industry standard

Public Key Cryptography Standard (PKCS) #7 - PKCS#7 is the standard for signing Certificate.  Is “CA Signature” of a Router that signed by CA (Stamp of approval).            

Public Key Cryptography Standard (PKCS) #10 - PKCS#10 is the standard format for sending certificate request. The end router send a certificate request using SCEP protocol to CA that request has to flow specific guideline. That has ask for certain thing, sign in certain way etc

RSA (Rivest-Shamir-Adleman) keys - RSA – is use for SSH, secure web services on router, also use for VPN Connection.

X.509 Certificates – Industry standard, this is the Certificate itself having public key of router, signature of router and stamp of approval from CA.

The Cisco routers support CA from the flowing entity:

Entrust, Baltimore, Verisign – well known, Windows 2000, 2003

The Certificate Enrollment Process:

This Process done by SCEP, more details at below:

About PKI(Public Key Infrastructure) – the whole idea of trusting of CA, that can be higher key of CA, the CA is trusted to another CA, if you trust that CA than that CA is well, is a big system that call PKI.

If you want to implementing Certificate Authority in your network so you have to fast enroll your devices with your CA. it’s basically done either on OOB (Out-of-Band) management network or you can do with flash drive (with USB has Certificate). Remember that if the process has compromise the whole system will be broken by attacker.

Example of Enroll process: First I point R1 to CA IP address and request for enroll/trust. The CA send his certificate to router R1 (this is CA certificate not the R1 certificate), the CA certificate has CA “Public key”, “CA Signature”. Now the router (R1) trusts the CA Certificate and adds this into the router configuration (Same thing for all). So all router have the CA certificate and “Public Key”, (that’s the first step) also all router trust the CA. Now CA will send the router (all) their own Certificate and the routers installed that (Second step). (That Describe in Certificate Section above)

Now Router R1 going to establish VPN with Router R2, Router R1 says to Router R2 I got the certificate (have Public key, Device Signature, Stamp of CA)from CA, so router R2 trusted Router R1 and also send him his Certificate. After exchange Certificate they both generate some “Session Key” using like DH formula based on both certificate exchanges and now they communicate using this session key for the VPN. All of this happens without typing “Pre-Shared key” between them and also they trusting the CA system.

Simple Certificate Enrollment Protocol (SCEP) Protocol:

SCEP is a standard, which uses an automated method to send certificates to end devices. By using SCEP protocol the CA sending certificate to the router, wireless, laptop etc.

That done by two Modes:

Manually – Its an manual process, where certificate approve by locale Administrator, nothing is automated. If someone wants to get certificate the administrator manually allow him to the trusted list of CA. this is very difficult process if there have large network.

Example: Router R1will use the SCEP says the CA I would like to trust you and could you send me certificate. Once the request comes in, the Certificate Authority (CA) administrator approve this request manually, here is your certificate.

Pre-Shared - when you have large network and want to approve many devices like Router, laptop etc, than you configure Pre-Shared key (cisco123). So any request comes with the same pre-shared key that will be automatically approved by CA. Once all devices on network have certificate and the pre-shared key is no longer “cisco123” than you might me change it manually. Os at initial setup pre-shared is better solution.  

Understanding VPN Architecture and how the IPSec VPN Negotiation process works:

How IPsec VPN Negotiation process work:

Here is the step by step process of how every single VPN connection is established and what happen make the VPN come up. Basically first three steps is the main configuration of VPN. Those steps describe below

Interesting Traffic Triggers VPN:

Is the traffic that matches on a router to transfer across the VPN between both sites. When I setup VPN, first we have to define which network traffic is considering as interesting traffic on both sites. Basically we configure access list to define interesting traffic.

Example: Here 172.16.10.0 is source address for Router R1 and destination address is 192.168.10.0 and also 192.168.10.0 is source address for Router R2 and destination address is 172.16.10.0. (if someone have another address (172.16.11.0) except those address (Like Internet host) that’s not bring up the VPN). So here 172.16.10.0 (R1 site) and 192.168.10.0 (R2 site) address is interesting traffic.

Interesting traffic decision – when router receives traffic that destines through VPN this have three choices:

Choice 1 (Encrypt Using IPSec) –

Route R1 configure access-list for interesting traffic is 172.16.10.0 and destination is 192.168.10.0 and also Router R2 configure access-list for interesting traffic is 192.168.10.0 and destination is 172.16.10.0, all of those IP address was encrypted for VPN connection on both sites.

The encryption has to be identically define on both sites connection otherwise the VPN connection was failed (like, in one site (R1) encrypted half the 172.16.10.0 and other site (R2) encrypted full the 172.16.10.0, so the connection was failed. They has to Identical (says Cryptomap), means this two are the same).

Choice 2 (Send In Clear Text) –

From the above description, the 172.16.10.0 and 192.168.10.0 is encrypted. So if I go with some IP (172.16.10.0 or 172.16.11.0) that not encrypted that will sent in clear text.

That is also calling like Split tunneling. The split tunneling is actually use for “Remote Access” VPN. The Split tunneling is pass traffic across VPN tunnel (site to site) and also split some traffic from tunnel for public internet, which does not match on cryptomap and the encryption list. So the unencrypted/normal traffic not goes through VPN tunnel.

If encrypt all of those traffic (traffic for search internet) which don’t need VPN tunnel, so those traffic first go other site using VPN tunnel and then goes to internet (just adding hops). So identify first what traffic need to be encrypted.

Choice 3 (Discard the Traffic) –

Router R2 has an encryption map (Cryptomap) but he receives a packet from Router R1 that not encrypted, So that will be discarded (Reason of IP Spoofing).

The VPN is not the one tunnel (Its tunnel inside of tunnel), it’s actually having 2 tunnels and the first one is IKE Phase 1 and second one IKE Phase 2 (also called IPsec tunnel).

IKE(Internet key Exchange) Phase 1:

IKE Phase 1, is the initial phase of security, its goal is to exchange VPN keys securely (AES 128bits) between two sites. The most important phase of everything IPSec VPN and it design to negotiate VPN.

In the IKE Phase 1exchange 3 types of message

Message 1 (Exchange and Negotiate Policy) – the first step/message that exchange and negotiate policy between two site. So you can make sure correctly configure policy on both site.

The router R1 may have site-to-sit VPN connections with Router R2 and R1 may also have other site-to-site VPN connections. So inside of that router (R1) define multiple policy lists, like Policy 1 (Pre-shared key, DH L5, AES 256bits), Policy 2 (Pre-shared key, DH L2, AES 256bits), Policy 3 (Pre-Shared Key, DH L5, DES) etc (you can create 60 thousand of policy), so the other site router (R2) must having policy that match with one policy of R1 to establish IKE Phase 1 otherwise the negotiation is failed.

You can create different policy for all of your VPN connections, those usage your policy as priority base.

Message 2 (Exchange Diffie Hellman Keys) – start this step after complete Message 1.

In the step both site exchange DH Public key to each other and they also use symmetric encryption algorithm (You know DH is asymmetric, so why use symmetric. The reason is less process consume).

Now they use the public key (magic of DH) on each site and combined with randomize number on both site and also each site will generate an identical symmetric encryption key. So that like the same AES (128-256) bits key will generate on both site and using this key, all future traffic pass through VPN.

Message 3 (Identity Verification) -

So what they do, Send the public key over to each other and verify identity, verify right pre-shared key if using, verify certificates if using that. (This step actually happen in Message 3)

IKE(Internet Key Exchange) Phase 2:

IKE Phase 2 is often called IPsec tunnel Phase/VPN Phase and use for actual data transfer; this is the tunnel inside of tunnel. The IKE Phase 2 is exchange keys (AES 128 bits) and transfer data after successfully complete IKE Phase 2.

“IKE Phase 2” is actually after exchange all your keys in “IKE Phase 1” verifying other site who they are and setup the secure tunnel for your key exchange. Now is start is called IPsec Transform set.

The IPsec Transform set is generate symmetric encryption keys, that would be use for rest of the session (here both site exchange DH public key and setup secure tunnel). Now generate symmetric key based on Transform set (that is actually configuration line), this transform set like AES 256 bits (that’s your Symmetric key) on both sites. The symmetric keys negotiate and exchange now they using those for all over data communicated between two routers. That’s the actually data transmission phase.  

VPN Teardown (Lifetime): how long the VPN tunnel will stay? After certain amount of idle time, no one use VPN anymore so it wills timeout until interesting traffic come again. It’s actually based on

Time – once the VPN reach the lifetime (86,400 seconds) (Still exchange data) they renegotiate the keys. The symmetric algorithm is very strong (Not like asymmetric key), so IPsec VPN will generate fresh symmetric key after certain amount of times to continuing VPN connection.

Data – once the certain amount of data exchange (like 4Mb) a fresh symmetric algorithm is come. After 4 Mb of data exchange between two routers the current symmetric key is trash and generate new symmetric key for next 4Mb to keep exchange data over VPN.

If someone tries to breaking key, may be it success after 100 years to breaking the key. There are 50 billion different keys, each time the router takes new key and renegotiation key after trashing old one. This processes same for both symmetric and asymmetric algorithm.

Example of IPSec VPN Configuration:
* How to Configure Site-to-Site IPSec VPN