Packet type detection is a crucial part of any traffic shaping technique. Traffic shaping is an essential part of the network traffic which allows time-sensitive packets meet their performance goals. Determining packet types is quite straightforward when the contents are not encrypted and the applications follow a certain pattern generating data packets. But things get much harder when we have to deal with an application which communicates in an encrypted manner and tries to hide from being detected by generating random pattern of packets (For example, Skype and Bit torrent clients are trying hard to avoid detection, because many service providers may benefit compromising their traffic).

There are many approaches to detect traffic such as Statistical Classification methods or Regular Expression pattern matching but these techniques can be fooled when newer version of the software reaches market and the traffic changes. These techniques also need time and human interference and may not necessarily work well with the newer versions of the software.

Therefore, we decided to introduce a new technique to detect packets in real-time. This technique uses the port scanning technique to communicate with the sender and the receiver of the packet flow and detect their services by trying to match a signature to their response. If the services are known then the types of the packet can be detected much easier. We tested our approach to detect HTTP, FTP and Skype packets as a course project and the results were very promising.

We created an application similar to nmap in Python which sends some packets to the sender and the receiver of each packet and tries to match their response to one of the signatures we created for the applications mentioned above and based on the matching we were able to output the packet type. Note that there are many nmap scripts available which can be used to detect many more types of service running on the servers.

The main advantages of this approach:

1. If the software remains backward compatible, our application will still be able to detect the packet type without the need for an update.
2. This approach works in both cases when the host’s connection is symmetric or asymmetric because at least one side of the connection should be active and respond to our request.
3. This technique will work when the source or destination are behind a port-restricted NAT because the port is already forwarded properly.

The disadvantages are as below:

1. The packets generated by our application adds to the network traffic and repeating the task for each flow on the network can be costly.
2. If the software refuses to communicate with our application or the response does not match our signature then this approach will be useless.
3. If the new version of the software is not backward compatible then the new signature will require human interference and automating this task is almost impossible.

I worked on this project with Mike Conley and he described the steps we had to follow to generate the signature for Skype here on his blog.