Big Data Visualization for a Cybersecurity Startup
For one of our customers, a cybersecurity startup, we implemented a variety of traffic and security data visualizations.
This is an example of a 3D visualization with cybersecurity data. Here, our love for data materialized in the cluster that took the shape of a heart after applying manifold learning. We wish we could show more examples of real-time visualizations publicly, but they they cannot be shared publicly.
Visualization of cybersecurity data has a long history. Everybody remembers Norse map of worldwide attacks that not only was useless for any practical purposes, but it turned out to be fake (the website even kept showing active map of attacks after the company closed its doors). You can see it recorded on YouTube, for example, here.
For this company, using events generated by cybersecurity real-time systems, Aligned Research designed non-interactive real-time dashboards. What is our approach to the data visualization that we applied to this engagement? There are four most important principles to show security and network data:
Real-Time: ongoing change to show real-time aspect of the data collection and processing: the dashboard must change at least every second
Eye Candy: the dashboard must be well designed, both the layout of the elements but also in the design of each dashboard component. The dashboard must be fun to watch as events unfold and data changes.
Show Data Insights: the dashboard must provide value beyond entertainment and eye candy: too many dashboards are designed well and show real-time change, but there is no value in professionals in adding these dashboard to their screens. These eye-candy-only visualizations are only showed to potential customers but generate little value or respect within the organization.
Intelligent Focus: when visualized directly, big data produces messy images. Designers of the dashboard must find a perfect balance between showing a lot of data and to have humans be able to consume this information. We used a variety of data filtering, data aggregation, and ETL processing to prepare the data for visualization. Significant data reduction enables hosting the code in the cloud even when the primary data resides in the private cloud (with “push” only data transfer), thus protecting sensitive data while enabling innovation to flourish.
The value of data insights available as a result of continuous observation of the dashboards resulted in many data scientists security researchers adding these dashboards to their monitors (who were initially sceptical in the value beyond eye candy), even though they always had access to the data through static data analytics tools. This has been the ultimate praise we received for our efforts. SOC (Security Operation Center) also many of the visualizations we developed to the dashboards and TVs they use to monitor malicious activity and generate new ideas for improving their protection.
As organizations move to involve more of their staff to innovate, democratization of the data access is becoming critical. With advanced visualizations Aligned Research created for this customer, more field personnel (sales engineers, sales) became involved in seeing the data trends, which enabled them to be more effective in growing the business. The field was to initiate conversations with several large corporate customers that were difficult to convince in the real-time aspect of their data processing, and many of their customers even asked how these visualizations were created. In addition to the dashboard and non-interactive visualizations, Aligned Research Group created multiple interactive interfaces to enable human security analysts to explore the data in a visual way. Let us have a discussion about your data to see how we can make your big data available for visual interaction with your staff.