We are very happy to announce our partnership with Farsight Security (Farsight) I would like to personally thank Bert Lathrop and Paul Vixie for their interest and support in our project. It's not every day you get to chat with an Internet legend.
We have now completed the integration of our machine learning algorithms with Farsight’s Passive DNS database, the DNSDB originally of ISC fame, which is arguably the most complete such dataset available to security analysts and practitioners in the market today.
This database is providing our project with enough breadth of data to create dozens of new features based on DNS and domain-related information, and also to create a whole new set of domain-oriented algorithms.
But what is this Passive DNS (pDNS) thing you say? How can it help in security monitoring and incident investigation?
Through Farsight’s superior Passive DNS data harvesting and value-added processing techniques, they monitor DNS queries and related responses in various points of the Internet, gathering information in real-time about domain names and what Internet infrastructure they resolve to, then collating and storing this information over time in DNSDB.
By storing information like this, you can have a pretty good picture of what historically were and presently are the actual DNS configurations on services and organizations in general. This can provide a much more complete and less biased dataset representing the relationships between domain names and IP addresses.
For example, if you try to resolve our host name on DNS, you will get an incomplete view as most CDNs and hosting providers will have round-robin setups and only return a few IP addresses as response.
These are very simple usage examples but, for those of you who have experience with data analysis, the potential of having this data at your fingertips should be immediately clear. It now becomes very easy to create relationship graphs between groups of IP addresses and domain names, and also possible to discover unusual trends in domain resolution and registration over time.