- NG Firewall firmware versions 4.2.x, 5.0.x, 5.2.x
- netfence firmware versions 4.2.x
Why there is no need to use a hashing algorithm (e.g. MD5, SHA) when you are establishing vpn-tunnels with the tina protocol.
The ESP protocol provides packet authentication and packet encryption. Packet authentication is performed using a hashing algorithm (MD5, SHA, etc.) which is used to hash the packet spanning the ESP header, the encrypted ESP payload (The tunneled IP packet) and the payload padding. Packet Encryption only spans the encrypted ESP payload and the payload padding and NOT(!) the ESP header.
An ESP packet is only valid if the following checks are passed (order is important)
- the authentication using the hashing algorithm is correct
- the sequence number is larger than all sequence numbers of all received valid(!) ESP packets (replay protection)
- the encryption of the ESP payload is sucessful (this is performed by a padding check)
This method was used since 10 years ago hashing algorithms were much faster than encryption alogorithms. The intention was to authenticate the packet before decryption, in order to avoid an expensive decryption for unauthentic packets. With AES this assumption is no longer true. In fact AES is even fast than SHA.
The NoHash method is based on the following consideration:
Encrpytion can be used as authentication since only the VPN partner holding the same encryption session key may construct an ESP packet which will be correctly decrypted. The only problem by simply turning off the authentication would be that packets can be replayed using old (captured) ESP packet and replay it exchanging the sequence number with a larger one. The receiver would trust the packet since the sequence number is not part of the encryption. This way a Denial of Service could be achieved.
The solution to this problem is quite simple: By including the sequence number redundantly in the encryption data of the ESP packet, tampering of the sequence number as described above is not possible. After decryption the two sequence numbers are simply compared and the packet discarded on mismatch.
As a consequence ESP packets can be exchanged authenticated and encrypted with replay protection using only a single encryption step as long as the sequence number is part of the encrypted data. This lead to a significant performance improvement single the hasing operation can be skipped.
This information belongs to all transports (Hybrid, UDP, TCP and ESP).