The FloVis System for Network Data Analysis

(Link to Full HD Video Demonstration).

NetFlow data is accumulated from the borders between private networks and the public internet. The data aggregate grows at the rate of gigabytes per day, stored in SiLK repositories. While tools exist to look for known malicious activities such as scanning, the volume of the data, its volatility, and the difficulty of establishing its ground truth preclude the use of conventional anomaly detection approaches for detecting malicious activity. To aid analysts in understanding this data, we at Computer Associates and Dalhousie University are developing a comprehensive and extensible set of visualization tools, integrated with the SiLK tool suite via a relational database that stores data such as sets and multisets (bags), derived from NetFlow and similar sources.

A Multi-view Architecture

Our approach incorporates visualizations of two views of the network: entity behaviors and entity interactions. An entity is a host or a grouping of hosts where the grouping is based on some common property such as an address in a given subnet or a physical location in a given country. It may also include behaviors, such as membership in a known botnet, having recently responded to a scan, exhibiting activity on a particular port, or maintaining connections with more than a given number of outside hosts. Multiple visualizations are used to represent entity relationships and behaviors. However, a major shortcoming of other visualization approaches is that they do not scale with large amounts of traffic. This project aims to overcome this by bundling flows or entities based at a variety of scales. The interactive displays provide drilldown and pivoting, allowing further investigation of features deemed of interest to the analyst.