Python Sniffing URL

Question

Does anyone know how to write a real-time data analyzer in Python that retrieves the source IP address and the full URL that is accessed? I looked at extracting data from urlsnarf, however IPv6 is not supported (and connections will be on IPv6 hosts).

While I can extract data from tcpdump and greping for GET / POST that will leave me with a simple path on the web server and I would not get the associated FQDN. Unfortunately, using SQUID w / IPv6 TPROXY is not an option due to the configuration of the environment.

Does anyone have any ideas on how to do this with Python bindings for libpcap? Your help would be most appreciated :)

Thank:)

Answer 1

Unfortunately, with IPv6 you are stuck doing your own TCP build. The good news is that you are only interested in URL data, which should (in general) be in one or two packages.

You must get away from using pylibpcap for this. You want to use setfilter on your pcap object to make sure that you only look at TCP traffic. As you move forward in your pcap loop, you will apply some HTTP regular expressions to the payload. If you have something that looks like HTTP traffic, and try to parse the header to get the URL data. Hope you get the full url with line breaks until the end of the package. If not, you will have to do a small TCP build.

, socket.inet_ntop socket.getaddrinfo, IPv6.